Use more realistic request distribution

[howardjohn]

Current situation

Today, the benchmark deploys 64 emojivoto instances (3 pods each). It then calls instance 0 for X time, instance 1 for X time, .... Each call is done by sending 128 request instantaneously to the server, waiting a bit (depending on RPS configuration), then sending 128 requests again.

Impact

This does not align with real world scenarios. It is extremely unlikely to receive this 'bursty' traffic pattern in the real world, as is it uncommon to send requests to only one namespace at a time.

Ideal future situation

Requests are sent to all namespaces with a uniform request release rate. Most load clients can do this - IIRC from looking at it a while back the wrk2 fork used here actually introduces the batching behavior, while standard wrk2 does not perform like this.

An alternative would be to have multiple replicas within a single namespace, rather than 64 namespaces. For example, we could have 6 namespaces with 10 replicas of each pod, for example. This is both more realistic and gives an opportunity for better load balancing algorithms to shine.

[howardjohn]

This shows the difference of current traffic flow (left) vs a uniform distribution (right). Each dot represents 1 request and the Y axis is latency.

The difference is night and day