Use wave scenario in performance tests #1574
Labels
C-Low Hanging Fruit
Ideal issue for new contributors.
S-Nice to have
The bug fix or feature would be nice but doesn't currently have much negative impact.
T-Challenges
Involves the challenge scenarios - shorter games with objectives.
Z-Developer Experience
This issue seeks to make life easier for developers writing Scenarios or other Swarm code.
Z-Performance
This issue concerns memory or time efficiency of the Swarm game.
Is your feature request related to a problem? Please describe.
Our performance tests are out of date, we should measure realistic play scenarios.
Describe the solution you'd like
Let's use the Wave scenario (#1556) that has shown useful for measuring.
To truly stress the system and see how the number of running robots affects performance,
we should generate longer versions of the map. (e.g. 100, 300, 1000 robots).
Reusing the scenario description and just making longer maps should prove simpler
and more useful then writing the benchmarks by hand.
The advantage of using a scenario is that we could actually play it and see the difference.
Describe alternatives you've considered
We could write a new benchmark purely in Haskell, but that seems hard. (Terms can use TemplateHaskell though.)
We could also reuse the scenario except for the robots and place those dynamically like in other benchmarks.
Additional context
Ideally we would measure the overhead of drawing the map (terrain, entities and moving robots) as the base robot moves.
This could be achieved by serialising the visible area to
Text
in each tick, which could make it easier to inspect the benchmark.The text was updated successfully, but these errors were encountered: