test(vm): Refactor VM benchmarks

[slowli]

What ❔

• Integrates Prometheus metrics into criterion benches; removes the DIY benchmark correspondingly.
• Merges the main benchmark crate with the harness one.
• Includes benched bytecodes into the crate itself rather than reading them in runtime.

Why ❔

Makes VM benchmarks more maintainable.

Checklist

• PR title corresponds to the body of PR (we generate changelog entries from PRs).
• Tests for the changes have been added / updated.
• Documentation comments have been added / updated.
• Code has been formatted via zk fmt and zk lint.

[slowli]

Couple of general comments:

Iai improvements are probably caused by reducing the setup complexity. Could also to do with removing some deps that have static initialization logic like RocksDB (although it should be filtered out by iai calibration logic).

If removing the DIY benchmark / extending Criterion is controversial, I can split it off from this PR. IMO, having uniform benchmark framework for running locally and Prometheus reporting is good since it allows not to define same benches multiple times. The API looks relatively straightforward. It has its drawbacks though; e.g., collected stats include warm-up iterations (so, e.g. reported means slightly differ between Criterion and Prometheus).

Metrics for a test run are in the stage Prometheus (vm_benchmark_mean_timing_seconds etc.).

[joonazan]

Regarding Criterion vs. DIY: DIY gives more useful data. The average time reported by criterion is a very bad metric, especially in noisy CI. Because our programs don't use any randomized algorithms (except hashmap in old VM), the minimum time is a good measurement.

The newer Divan framework is as pleasant as Criterion and reports minima. I use it in the vm2 repo.

[slowli]

The average time reported by criterion is a very bad metric, especially in noisy CI.

My understanding is that in CI, we would primarily rely on metrics exported to Prometheus, which do feature the minimum iteration time (along with maximum, mean and median, and the number of iterations). They could be printed to stdout as well (added that in the latest commit). So if it's only a question of whether these values are displayed, it's not an issue.

[github-actions]

Detected VM performance changes

Benchmark name	change in estimated runtime	change in number of opcodes executed
decode_shl_sub	-4.9%	N/A
access_memory	-3.4%	N/A
deploy_simple_contract_legacy	-18.0%	N/A
call_far	-3.2%	N/A
write_and_decode	-4.1%	N/A
deploy_simple_contract	-11.9%	N/A
slot_hash_collision	-4.3%	N/A
call_far_legacy	-2.5%	N/A

Changes in number of opcodes executed indicate that the gas price of the benchmark has changed, which causes it run out of gas at a different time. Or that it is behaving completely differently.