client/db: prune archived orders #2975
Conversation
Prune archived orders when there are more than 1000 (configurable).
buck54321
force-pushed
the
dump-archive
branch
from
43e8f54
to
4bab10f
Compare
| nOrds := uint64(archivedOB.Stats().BucketN - 1 /* BucketN includes top bucket */) | ||
| if nOrds <= archiveSizeLimit { | ||
| return nil | ||
| } |
There was a problem hiding this comment.
You could do this first and maybe avoid collecting the active orders sometimes.
|
|
||
| ArchiveSizeLimit uint64 `long:"archivesize" description:"the maximum number of orders to be archived before deleting the oldest"` |
There was a problem hiding this comment.
Could mention it cannot be zero, or error somewhere if it is.
That's a much bigger thing, since archived matches are not indexed on order ID. I do agree that we need a more sophisticated solution though with more knobs. The problem we have right now is that clients running mm bots are filling up their disks too quickly, and we need something for a patch release asap. |
| if _, found := oidsWithActiveMatches[oid]; found { | ||
| return nil | ||
| } |
There was a problem hiding this comment.
What if instead of just not deleting the oids with active matches you don't delete them with any matches. I think the problem is that the archives get clogged up with orders that just get created and cancelled after the market moves. Orders that actually matched shouldn't get deleted because people need to keep records of those. You could even just check the swap fees here, and not delete any trade that has swap fees > 0.
|
I'm going to close this and we can work out a more comprehensive solution later. |
Prune archived orders when there are more than 1000 (configurable).