Skip to content
This repository has been archived by the owner on Feb 15, 2022. It is now read-only.

Zenbot hanging #2763

Open
arthurwolf opened this issue Nov 19, 2021 · 5 comments
Open

Zenbot hanging #2763

arthurwolf opened this issue Nov 19, 2021 · 5 comments

Comments

@arthurwolf
Copy link

Have I written custom code (as opposed to using zenbot vanilla): Yes, a custom strategy, but see bellow
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 20.04, Fresh install in VM
Zenbot version (commit ref, or version): What github clones this day
Zenbot branch: Same
NodeJS version: v10.19.0
Python version (when using a python script): 2.7.17
Exact command to reproduce (include everything): node zenbot.js sim binance.1INCH-USDT --strategy=grid --period=10m --start=202012250000 --end=202101010000 --grid_high=0.001 --grid_low=0.0001 --grid_number=100
Did I make any changes to conf-sample.js?: no

Describe the problem

Zenbot just holds on a regular basis during simulations.

I added tons of console.logs to trace the issue, and it seems like I get to onCollectionCursorEnd in sim.js, with numTrades != 0 it goes into the "else", so the engine.exit(exitSim) isn't hit, and from there it just does nothing anymore.

What is this "else" for and what is supposed to happen once it's hit?

I do not think this has to do with my custom strategy, but I could very well be wrong. Would still appreciate any idea in debugging this so I can fix whatever is happening here.

here's the order things happen during a crash:

  1. enter collectionCursorStream.on('data') callback
  2. As totaltrades == numtrades, call onCollectionCursorEnd()
  3. Enter onCollectionCursorEnd()
  4. go into the "else"
  5. console.log([reversing, cursor, lastTrade, numTrades, totalTrades]) there shows:
    [ undefined,
    1609032111821,
    { _id: 'binance.1INCH-USDT-783797',
    trade_id: '783797',
    time: 1609032125360,
    size: 126,
    price: 1.4184,
    side: 'buy',
    id: 'binance.1INCH-USDT-783797',
    selector: 'binance.1INCH-USDT' },
    100,
    100 ]
  6. exit collectionEnd(), while calling getNext()
  7. enter getNext()
  8. do tradesCollection.find() on :
    { query:
    { selector: 'binance.1INCH-USDT',
    time: { '$lte': 1609459200000, '$gt': 1609032125360 } },
    sort: { time: 1 },
    limit: 100,
    timeout: false }
  9. exit collectionCursorStream.on('data') callback (presumably the one called in (1.), which is surprising to happen now, threads/promises/parralelism stuff?
  10. in getNext() still, we haven't left yet, collectionCursor.stream() and collectionCursor.count(true) gets executed at this time (it might be before 9.)
  11. Finally, we exit getNext()
  12. Nothing happens anymore from now, none of the console.log()s get called anymore, no CPU is used.
@arthurwolf
Copy link
Author

it's starting to look like this is a weird re-entry issue.

why is getNext() an async function but await isn't used when it's called?

@arthurwolf
Copy link
Author

a dirty fix to this was to change limit: 100 to limit: 100000000, it now works 100% of the time
it also had a weird side effect: execution of simulations is MASSIVELY faster
why was this limited to 100, was this built to be executed on a raspberry pi originally or something like that? it's a very small limit...
or maybe there's an operational reason why you need to do it by small groups? the reverse/symmetry thing?

@arthurwolf
Copy link
Author

note: setting limit to very high values does not solve the issue, it just makes it much much more difficult to reproduce
also, setting limit to 5 makes it much easier to reproduce

@arthurwolf
Copy link
Author

Replacing most of the getNext() function with simply:

    let cur = tradesCollection.find(opts.query).sort(opts.sort);
    let results = await cur.toArray();
    results.forEach((trade, i) => {eventBus.emit('trade', trade)});
    engine.exit(exitSim);

functions great (so far...)
I'm loosing the "reverse" feature, which I don't use/understand. If I understood what it is maybe I could re-implement it, will look into it.

@arthurwolf
Copy link
Author

Down to two lines, works perfectly. Doesn't have the "symmetry" feature, but I don't understand that well enough to implement it. Any explanations welcome:

        await tradesCollection.find(opts.query).batchSize(10).sort(opts.sort).forEach((trade, i) => {eventBus.emit('trade', trade)});
        engine.exit(exitSim);

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

2 participants