Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

memory creeping up #261

Open
ninjaahhh opened this issue Dec 10, 2018 · 5 comments
Open

memory creeping up #261

ninjaahhh opened this issue Dec 10, 2018 · 5 comments
Assignees

Comments

@ninjaahhh
Copy link
Contributor

memory is still creeping up on my 4g-mem machine. after running for the 3~4 days (after the keccak fix) the memory usage went from ~1.2g to ~3g.

master: ~700m
slave 1: ~1.1g
slave 2: ~1.1g

@ninjaahhh ninjaahhh self-assigned this Dec 10, 2018
@qcdll
Copy link
Contributor

qcdll commented Dec 10, 2018

it may be worthwhile to keep a multi-day cluster of python3 with #254

@ninjaahhh
Copy link
Contributor Author

agree with @qcdll. let's keep track of those machines

@qcdll
Copy link
Contributor

qcdll commented Dec 11, 2018

here is a slave left overnight:
https://gist.github.com/qcdll/fe22b6ab2c143b3bf8f13e8a3f156f6c
the log takes snapshot every 1 min and report differences, there is some steady increase from core.py which seems to relate to deserialization (mem allocated on those lines were never freed)

@qcdll
Copy link
Contributor

qcdll commented Dec 12, 2018

top mem usage by lineno and some traceback
this was obtained from a slave that ran 1+ days
https://gist.github.com/qcdll/8a6c1ce4fd5b675162fb6dfb1a0d7fe7

@qcdll
Copy link
Contributor

qcdll commented Dec 13, 2018

based on analysis above, adding new counts in #254 and will be launching clusters with python3 and #254 for testnet 23

also as @ninjaahhh suggests will also be running profiling with mining enabled

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants