Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[3.0] Operate on detached copies of tensors #147

Open
bchess opened this issue Jun 27, 2024 · 0 comments
Open

[3.0] Operate on detached copies of tensors #147

bchess opened this issue Jun 27, 2024 · 0 comments
Assignees
Milestone

Comments

@bchess
Copy link
Contributor

bchess commented Jun 27, 2024

Eta0 last week
This is more of a general comment as it doesn't block this PR, but as an optimization we should operate on detached copies of tensors to better control tensor lifetime and their garbage collection impact (by manually set_-ing them to empty buffers at the end of the serialization process). The handling of futures in this PR makes the object lifetime a little unclear, but depending on what function deallocation triggers in, it can reduce performance by a bit. Plus, deallocation can even be shunted to a background thread right before returning from the function that way.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants