add queueing and multi process support

[Peddle]

• Adds ability to queue multiple requests for better local dev experience
• adds request id to all logs
• adds ability to configure number of processes

[dMARLAN]

🚗 drive by review 🚗💨

nice work, left comments 🍌

main thing is I think a lot of code can be extracted into private methods to make everything a lot cleaner to read & easier to maintain.

[erik-dunteman]

Doing some QA.

I am just observing behavior, not saying we should fix anything just yet.

[erik-dunteman]

If I run single worker, three background tasks w/ 5s sleep, they all queue (nice) but the 200 responses all log under the first request id.

[erik-dunteman]

Worker ID super helpful in logs. The order they consume from the queue makes sense. 3 bg calls to 2 workers, first 2 calls claimed immediately, third claimed by the worker that took # 1

[erik-dunteman]

Going to try with 100k processes, wish me luck

[erik-dunteman]

Got up to 32 workers (BERT) running locally, laptop struggled but seemingly no bugs.

I'd suggest

• cap quantity of workers so users don't footgun and Banana doesn't get loaded up with OOM-bombs
• showing some proof of a highly concurrent slam test passing

[erik-dunteman]

Tentative LGTM from me given above is addressed