Statsd seems to be dropping metrics probably because of UDP

[olegbespalov]

Discussed in forum thread.

My last comment confirms that:

1. running between dockers is worse
2. fixing MTU in docker helps
3. lowering push interval (effectively always pushing smaller packets) helps more and effectively fixes it in my case, although this shouldn't be needed as the lib should be doing the same 🤷
4. updating datadog-go makes it slightly worse 🤦
5. k6.v0.29.0(random older version I tried) is even worse. 🤦

Possible actions:

1. lower the push interval - this seems like a good fast solution, given that we are sending metrics as udp packets and they have a small limit, k6 not doing it more regularly doesn't really help anything IMO
2. investigate this further, it might not be UDP related at all, at which point switching statsd libraries to something with possibly tcp support as well might not even help.
3. investigate datadog-go + aggregation in datadog-go ... maybe it will help 🤷

The issue ported from grafana/k6#2044