Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix wrong inputs/model placement when using a single core #725

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

JingyaHuang
Copy link
Collaborator

@JingyaHuang JingyaHuang commented Oct 25, 2024

What does this PR do?

It's a recent find that I did not experience before. Previously, when only a single neuron core was used, input tensors were allocated to the same core that the model has been loaded with torch.jit.load. But it seems that now the input tensors are always allocated to nc:0, whereas the model is loaded to a core with lower usage, so we could bump into unexpected flaky situations where they are in different cores...

2024-Oct-25 15:40:20.728276 13974:13974 ERROR  NMGR:dlr_check_tensor_set_on_same_vtpb_v2    Tensor input249 allocated on vnc 0 must be allocated on same vnc as model (1)

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests?

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants