Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[3.0] [maybe] More granular future dependencies #152

Open
bchess opened this issue Jun 27, 2024 · 0 comments
Open

[3.0] [maybe] More granular future dependencies #152

bchess opened this issue Jun 27, 2024 · 0 comments
Milestone

Comments

@bchess
Copy link
Contributor

bchess commented Jun 27, 2024

From @Eta0 in #127 (review)

Dependencies for later tasks are tracked as a single tensor_data_task. We should add differentiation for write hazards and read hazards, since storing it as one item doesn't allow two operations that only need to read tensor data to happen simultaneously (e.g. computing hashes while also writing data to disk, when encryption isn't active). This is less harmful to performance when queueing operations in batch order like now rather than per-tensor, but any time that there are enough threads (or few enough tensors) to handle tasks from multiple stages at once, this would unblock the later stages sooner.

@bchess bchess added this to the 3.0 milestone Jun 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant