Add fuzzer workflow #1240

daphneherlambda · 2023-06-15T17:45:29Z

Add fuzzer workflow

Description

This PR adds a worflow to run the fuzzers every day at 00:00

Checklist

Linked to Github Issue
Unit tests added
Integration tests added.
This change requires new documentation.
- Documentation has been added/updated.
- CHANGELOG has been updated.

codecov · 2023-06-15T22:06:14Z

Codecov Report

❗ No coverage uploaded for pull request base (cairo-fuzzer@29e1a1c). Click here to learn what that means.
The diff coverage is n/a.

@@               Coverage Diff               @@
##             cairo-fuzzer    #1240   +/-   ##
===============================================
  Coverage                ?   97.61%           
===============================================
  Files                   ?       89           
  Lines                   ?    36333           
  Branches                ?        0           
===============================================
  Hits                    ?    35466           
  Misses                  ?      867           
  Partials                ?        0

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

github-actions · 2023-06-16T14:11:37Z

Benchmark Results for unmodified programs 🚀

Command	Mean [s]	Min [s]	Max [s]	Relative
`base blake2s_integration_benchmark`	10.298 ± 0.153	10.167	10.552	1.01 ± 0.02
`head blake2s_integration_benchmark`	10.201 ± 0.088	10.098	10.378	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
`base compare_arrays_200000`	3.483 ± 0.043	3.440	3.576	1.00
`head compare_arrays_200000`	3.491 ± 0.046	3.442	3.597	1.00 ± 0.02

Command	Mean [s]	Min [s]	Max [s]	Relative
`base dict_integration_benchmark`	2.362 ± 0.030	2.323	2.416	1.00
`head dict_integration_benchmark`	2.375 ± 0.047	2.334	2.502	1.01 ± 0.02

Command	Mean [s]	Min [s]	Max [s]	Relative
`base factorial_multirun`	3.682 ± 0.045	3.635	3.763	1.00 ± 0.01
`head factorial_multirun`	3.681 ± 0.030	3.641	3.747	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
`base fibonacci_1000_multirun`	3.092 ± 0.039	3.056	3.191	1.00 ± 0.01
`head fibonacci_1000_multirun`	3.077 ± 0.013	3.063	3.095	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
`base field_arithmetic_get_square_benchmark`	1.832 ± 0.010	1.819	1.851	1.00 ± 0.01
`head field_arithmetic_get_square_benchmark`	1.831 ± 0.010	1.808	1.843	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
`base integration_builtins`	8.899 ± 0.171	8.748	9.356	1.00
`head integration_builtins`	8.995 ± 0.150	8.798	9.355	1.01 ± 0.03

Command	Mean [s]	Min [s]	Max [s]	Relative
`base keccak_integration_benchmark`	10.427 ± 0.178	10.157	10.728	1.01 ± 0.02
`head keccak_integration_benchmark`	10.273 ± 0.134	10.115	10.527	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
`base linear_search`	3.501 ± 0.018	3.467	3.541	1.00 ± 0.01
`head linear_search`	3.500 ± 0.018	3.472	3.532	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
`base math_cmp_and_pow_integration_benchmark`	2.507 ± 0.046	2.475	2.629	1.00 ± 0.03
`head math_cmp_and_pow_integration_benchmark`	2.504 ± 0.043	2.477	2.621	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
`base math_integration_benchmark`	2.303 ± 0.012	2.286	2.321	1.00 ± 0.02
`head math_integration_benchmark`	2.302 ± 0.033	2.283	2.395	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
`base memory_integration_benchmark`	2.017 ± 0.009	2.009	2.040	1.00
`head memory_integration_benchmark`	2.054 ± 0.087	2.005	2.293	1.02 ± 0.04

Command	Mean [s]	Min [s]	Max [s]	Relative
`base operations_with_data_structures_benchmarks`	2.286 ± 0.044	2.252	2.409	1.00 ± 0.02
`head operations_with_data_structures_benchmarks`	2.281 ± 0.009	2.269	2.298	1.00

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`base pedersen`	843.7 ± 2.2	840.4	846.2	1.00
`head pedersen`	847.2 ± 5.8	837.7	859.1	1.00 ± 0.01

Command	Mean [s]	Min [s]	Max [s]	Relative
`base poseidon_integration_benchmark`	1.571 ± 0.005	1.562	1.580	1.00
`head poseidon_integration_benchmark`	1.575 ± 0.015	1.557	1.600	1.00 ± 0.01

Command	Mean [s]	Min [s]	Max [s]	Relative
`base secp_integration_benchmark`	2.645 ± 0.025	2.622	2.708	1.00
`head secp_integration_benchmark`	2.646 ± 0.014	2.634	2.680	1.00 ± 0.01

Command	Mean [s]	Min [s]	Max [s]	Relative
`base set_integration_benchmark`	1.304 ± 0.019	1.286	1.334	1.01 ± 0.02
`head set_integration_benchmark`	1.296 ± 0.009	1.286	1.319	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
`base uint256_integration_benchmark`	6.524 ± 0.040	6.460	6.599	1.00
`head uint256_integration_benchmark`	6.540 ± 0.076	6.493	6.741	1.00 ± 0.01

MegaRedHand · 2023-06-23T19:11:00Z

.github/workflows/fuzzer.yml

+        sudo apt-get install -y build-essential
+        sudo apt-get install binutils-dev
+        sudo apt-get install libunwind-dev
+        sudo curl https://sh.rustup.rs -sSf | bash -s -- -y --default-toolchain nightly


You could use dtolnay's action for this

Yes, please do that.

* Create fuzzer * Remove cli * Update fuzzer * Change fuzzer name * Update Cargo.toml * Update Cargo.toml * Add fuzzer workflow (#1240) * add workflow to run fuzzers * delete extra changes * add pulling inputs folder from outside repo * fix paths * change test workflow to use cache and store the report * actualice workflow * Update fuzzer.yml * delete extra files * Delete Cargo.toml * Delete main.rs --------- Co-authored-by: dafifynn <[email protected]> * Update Cargo.toml --------- Co-authored-by: Juanma <[email protected]> Co-authored-by: daphneherlambda <[email protected]> Co-authored-by: dafifynn <[email protected]>

* Create fuzzer * Remove cli * Update fuzzer * Change fuzzer name * Update Cargo.toml * Update Cargo.toml * Add fuzzer workflow (lambdaclass#1240) * add workflow to run fuzzers * delete extra changes * add pulling inputs folder from outside repo * fix paths * change test workflow to use cache and store the report * actualice workflow * Update fuzzer.yml * delete extra files * Delete Cargo.toml * Delete main.rs --------- Co-authored-by: dafifynn <[email protected]> * Update Cargo.toml --------- Co-authored-by: Juanma <[email protected]> Co-authored-by: daphneherlambda <[email protected]> Co-authored-by: dafifynn <[email protected]>

dafifynn added 3 commits June 15, 2023 14:44

add workflow to run fuzzers

88f347d

delete extra changes

ad3e85a

delete extra changes

96b8d9e

daphneherlambda changed the base branch from main to cairo-fuzzer June 15, 2023 18:12

add pulling inputs folder from outside repo

5b4e819

fix paths

a2c6a2a

change test workflow to use cache and store the report

61bbf78

daphneherlambda added pipelines This PR/issue is exclusively about improving our CI coverage and removed pipelines This PR/issue is exclusively about improving our CI labels Jun 16, 2023

matias-gonz marked this pull request as ready for review June 16, 2023 20:27

matias-gonz requested review from igaray, Oppen, fmoletta, entropidelic, juanbono, pefontana and Juan-M-V as code owners June 16, 2023 20:27

matias-gonz changed the base branch from cairo-fuzzer to main June 16, 2023 20:35

pefontana marked this pull request as draft June 16, 2023 20:37

matias-gonz changed the base branch from main to cairo-fuzzer June 16, 2023 20:38

dafifynn and others added 6 commits June 16, 2023 19:14

actualice workflow

ae103a5

Update fuzzer.yml

5ab1bb7

delete extra files

820de90

Delete Cargo.toml

2d040c6

Delete main.rs

e7d36a9

Merge branch 'cairo-fuzzer' into add-fuzzer-workflow

de89bd1

daphneherlambda marked this pull request as ready for review June 23, 2023 15:25

MegaRedHand approved these changes Jun 23, 2023

View reviewed changes

Oppen approved these changes Jun 28, 2023

View reviewed changes

Oppen merged commit 185e82d into cairo-fuzzer Jun 28, 2023
33 checks passed

Oppen deleted the add-fuzzer-workflow branch June 28, 2023 18:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add fuzzer workflow #1240

Add fuzzer workflow #1240

daphneherlambda commented Jun 15, 2023

codecov bot commented Jun 15, 2023 •

edited

Loading

github-actions bot commented Jun 16, 2023 •

edited

Loading

MegaRedHand Jun 23, 2023

Oppen Jun 28, 2023

Add fuzzer workflow #1240

Add fuzzer workflow #1240

Conversation

daphneherlambda commented Jun 15, 2023

Add fuzzer workflow

Description

Checklist

codecov bot commented Jun 15, 2023 • edited Loading

Codecov Report

github-actions bot commented Jun 16, 2023 • edited Loading

MegaRedHand Jun 23, 2023

Choose a reason for hiding this comment

Oppen Jun 28, 2023

Choose a reason for hiding this comment

codecov bot commented Jun 15, 2023 •

edited

Loading

github-actions bot commented Jun 16, 2023 •

edited

Loading