[BBPBGLIB-712] Estimate memory usage for synapse and connection #22
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Enabled Target methods to return raw gids - Accumulate counts in Counter() object (from collections) - MPI gather all counts, merge them and display
…pses - Added script used to measure synapse memory usage - Dry run now calculates estimate of memory usage of synapses - Fix typos
ferdonline
reviewed
Aug 9, 2023
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
st4rl3ss
changed the title
This comment has been minimized.
This comment has been minimized.
|
Nitpick: In the benchmark script we could work in Kilobytes to avoid multiplying by 1024 bellow. |
|
Thanks for the review @ferdonline . I think also we can integrate the json export into a different MR. |
|
Logfiles from GitLab pipeline #143299 (:white_check_mark:) have been uploaded here! Status and direct links: |
ferdonline
approved these changes
Aug 10, 2023
WeinaJi
pushed a commit
that referenced
this pull request
Oct 14, 2024
As detailed in [BBPBGLIB-712](https://bbpteam.epfl.ch/project/issues/browse/BBPBGLIB-712), this MR aims to introduce memory estimation for synapses and relative connections. A basic counter is introduced that tracks all the synapses that get instantiated on all the ranks. Then memory usage is estimated using a LUT containing the approximate memory estimate for each type of synapse. The values were calculated using the script `synstat.py` included in the `_benchmarks` folder. Based on the counter and the memory estimate, an approximated figure for memory utilization can be reported to the user. --------- Co-authored-by: Fernando Pereira <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As detailed in BBPBGLIB-712, this MR aims to introduce memory estimation for synapses and relative connections.
A basic counter is introduced that tracks all the synapses that get instantiated on all the ranks. Then memory usage is estimated using a LUT containing the approximate memory estimate for each type of synapse. The values were calculated using the script
synstat.pyincluded in the_benchmarkfolder.Based on the counter and the memory estimate, an approximated figure for memory utilization can be reported to the user.