libirmin: reuse a single eio scheduler across calls #2318
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
(builds on top of the other Eio PRs, only the last commit 121309c is new)
The port to Eio of
libirmin
inherited a defect from the previous integration with the Lwt scheduler: Every calls from C into OCaml land would start a fresh scheduler, run the Irmin function, stop the scheduler and return the result to the C world. Not only is it inefficient to start/stop the scheduler so often, stopping Eio will also close any opened database files thanks to its strict resource management... which was not an issue before as Lwt allowed us to leak the open files across calls!To address this, @talex5 recommended that we spawn a new domain to run the Eio scheduler on, with the C thread submitting request to it and blocking until the response comes back... but since
libirmin
doesn't expose any "async" operations to C, we would always have a single domain active at a time (either the C thread or the Eio one).So it was slightly easier and more fun to flex some effect handlers to achieve the same outcome without a second domain:
cc @kayceesrk "Eio dawg, I heard you like schedulers"