Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Builtins] Make all builtins strict #6523

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

[Builtins] Make all builtins strict #6523

wants to merge 3 commits into from

Conversation

effectfully
Copy link
Contributor

I was investigating a weird behavior of a test and this is where it led me to. This PR is gonna be blocked by broken benchmarking though, but we really should make builtins strict, because laziness only reduces the quality of tests.

@effectfully effectfully self-assigned this Sep 27, 2024
@effectfully
Copy link
Contributor Author

/benchmark validation

@github-actions GitHub Actions
Copy link
Contributor

Click here to check the status of your benchmark.

@github-actions GitHub Actions
Copy link
Contributor

Comparing benchmark results of 'validation' on '6e469a346b' (base) and '2fb2d22dcb' (PR)

Results table
Script 6e469a3 2fb2d22 Change
auction_1-1 256.2 μs 258.6 μs +0.9%
auction_1-2 914.3 μs 614.8 μs -32.8%
auction_1-3 903.3 μs 620.4 μs -31.3%
auction_1-4 325.4 μs 228.3 μs -29.8%
auction_2-1 257.0 μs 177.9 μs -30.8%
auction_2-2 919.9 μs 628.5 μs -31.7%
auction_2-3 841.1 μs 822.8 μs -2.2%
auction_2-4 632.8 μs 623.1 μs -1.5%
auction_2-5 231.2 μs 229.1 μs -0.9%
crowdfunding-success-1 208.7 μs 207.6 μs -0.5%
crowdfunding-success-2 293.4 μs 207.3 μs -29.3%
crowdfunding-success-3 297.3 μs 211.6 μs -28.8%
currency-1 354.1 μs 244.4 μs -31.0%
escrow-redeem_1-1 493.0 μs 349.9 μs -29.0%
escrow-redeem_1-2 494.9 μs 491.0 μs -0.8%
escrow-redeem_2-1 582.8 μs 578.8 μs -0.7%
escrow-redeem_2-2 404.9 μs 578.3 μs +42.8%
escrow-redeem_2-3 419.5 μs 580.4 μs +38.4%
escrow-refund-1 156.1 μs 222.3 μs +42.4%
future-increase-margin-1 357.4 μs 352.3 μs -1.4%
future-increase-margin-2 578.7 μs 760.3 μs +31.4%
future-increase-margin-3 768.6 μs 704.2 μs -8.4%
future-increase-margin-4 695.4 μs 548.2 μs -21.2%
future-increase-margin-5 1.158 ms 1.160 ms +0.2%
future-pay-out-1 353.8 μs 350.7 μs -0.9%
future-pay-out-2 765.5 μs 758.4 μs -0.9%
future-pay-out-3 766.5 μs 759.4 μs -0.9%
future-pay-out-4 794.2 μs 850.6 μs +7.1%
future-settle-early-1 249.6 μs 244.9 μs -1.9%
future-settle-early-2 541.5 μs 531.5 μs -1.8%
future-settle-early-3 543.0 μs 532.6 μs -1.9%
future-settle-early-4 612.7 μs 868.6 μs +41.8%
game-sm-success_1-1 390.4 μs 537.3 μs +37.6%
game-sm-success_1-2 200.7 μs 195.3 μs -2.7%
game-sm-success_1-3 640.7 μs 831.9 μs +29.8%
game-sm-success_1-4 232.6 μs 330.2 μs +42.0%
game-sm-success_2-1 389.1 μs 548.7 μs +41.0%
game-sm-success_2-2 200.8 μs 284.7 μs +41.8%
game-sm-success_2-3 642.7 μs 906.9 μs +41.1%
game-sm-success_2-4 232.7 μs 329.9 μs +41.8%
game-sm-success_2-5 645.0 μs 908.5 μs +40.9%
game-sm-success_2-6 232.4 μs 329.2 μs +41.7%
multisig-sm-1 532.9 μs 558.7 μs +4.8%
multisig-sm-2 381.7 μs 543.9 μs +42.5%
multisig-sm-3 393.2 μs 552.0 μs +40.4%
multisig-sm-4 395.3 μs 555.8 μs +40.6%
multisig-sm-5 760.4 μs 648.3 μs -14.7%
multisig-sm-6 565.7 μs 560.7 μs -0.9%
multisig-sm-7 548.0 μs 543.5 μs -0.8%
multisig-sm-8 560.1 μs 554.5 μs -1.0%
multisig-sm-9 559.2 μs 557.3 μs -0.3%
multisig-sm-10 790.5 μs 785.8 μs -0.6%
ping-pong-1 470.1 μs 468.0 μs -0.4%
ping-pong-2 469.2 μs 467.7 μs -0.3%
ping-pong_2-1 284.5 μs 282.3 μs -0.8%
prism-1 240.3 μs 239.5 μs -0.3%
prism-2 408.2 μs 587.2 μs +43.9%
prism-3 369.4 μs 520.6 μs +40.9%
pubkey-1 143.1 μs 201.4 μs +40.7%
stablecoin_1-1 910.4 μs 1.271 ms +39.6%
stablecoin_1-2 195.9 μs 278.9 μs +42.4%
stablecoin_1-3 1.041 ms 1.464 ms +40.6%
stablecoin_1-4 297.2 μs 293.8 μs -1.1%
stablecoin_1-5 1.895 ms 1.880 ms -0.8%
stablecoin_1-6 255.1 μs 363.6 μs +42.5%
stablecoin_2-1 911.4 μs 1.271 ms +39.5%
stablecoin_2-2 195.7 μs 279.1 μs +42.6%
stablecoin_2-3 1.042 ms 1.463 ms +40.4%
stablecoin_2-4 207.8 μs 293.8 μs +41.4%
token-account-1 191.4 μs 187.6 μs -2.0%
token-account-2 340.9 μs 335.7 μs -1.5%
uniswap-1 419.9 μs 414.0 μs -1.4%
uniswap-2 227.3 μs 224.6 μs -1.2%
uniswap-3 1.762 ms 1.751 ms -0.6%
uniswap-4 359.1 μs 332.8 μs -7.3%
uniswap-5 1.157 ms 1.146 ms -1.0%
uniswap-6 339.3 μs 316.8 μs -6.6%
vesting-1 347.7 μs 350.2 μs +0.7%
6e469a3 2fb2d22 Change
TOTAL 40.95 ms 44.04 ms +7.6%

@effectfully
Copy link
Contributor Author

The benchmarking machine is still sick.

@effectfully
Copy link
Contributor Author

/benchmark validation

@github-actions GitHub Actions
Copy link
Contributor

Click here to check the status of your benchmark.

@github-actions GitHub Actions
Copy link
Contributor

Comparing benchmark results of 'validation' on '9b4e76a9a' (base) and '0a644b5af' (PR)

Results table
Script 9b4e76a 0a644b5 Change
auction_1-1 254.8 μs 257.3 μs +1.0%
auction_1-2 908.7 μs 910.4 μs +0.2%
auction_1-3 899.8 μs 906.4 μs +0.7%
auction_1-4 329.1 μs 331.5 μs +0.7%
auction_2-1 254.5 μs 257.1 μs +1.0%
auction_2-2 913.5 μs 909.8 μs -0.4%
auction_2-3 1.184 ms 1.188 ms +0.3%
auction_2-4 900.1 μs 900.8 μs +0.1%
auction_2-5 329.7 μs 332.9 μs +1.0%
crowdfunding-success-1 297.5 μs 299.2 μs +0.6%
crowdfunding-success-2 296.6 μs 299.0 μs +0.8%
crowdfunding-success-3 297.6 μs 299.0 μs +0.5%
currency-1 356.3 μs 357.4 μs +0.3%
escrow-redeem_1-1 496.5 μs 498.1 μs +0.3%
escrow-redeem_1-2 496.7 μs 503.3 μs +1.3%
escrow-redeem_2-1 582.3 μs 582.5 μs +0.0%
escrow-redeem_2-2 397.3 μs 581.5 μs +46.4%
escrow-redeem_2-3 407.6 μs 577.9 μs +41.8%
escrow-refund-1 155.9 μs 222.3 μs +42.6%
future-increase-margin-1 251.5 μs 360.2 μs +43.2%
future-increase-margin-2 539.0 μs 765.5 μs +42.0%
future-increase-margin-3 542.4 μs 774.7 μs +42.8%
future-increase-margin-4 494.4 μs 694.6 μs +40.5%
future-increase-margin-5 812.3 μs 1.153 ms +41.9%
future-pay-out-1 251.4 μs 356.4 μs +41.8%
future-pay-out-2 540.4 μs 778.8 μs +44.1%
future-pay-out-3 777.8 μs 766.0 μs -1.5%
future-pay-out-4 1.153 ms 1.153 ms 0.0%
future-settle-early-1 356.7 μs 356.3 μs -0.1%
future-settle-early-2 765.7 μs 765.1 μs -0.1%
future-settle-early-3 764.5 μs 768.2 μs +0.5%
future-settle-early-4 858.2 μs 859.9 μs +0.2%
game-sm-success_1-1 551.5 μs 554.8 μs +0.6%
game-sm-success_1-2 283.9 μs 287.3 μs +1.2%
game-sm-success_1-3 905.8 μs 912.1 μs +0.7%
game-sm-success_1-4 331.6 μs 333.8 μs +0.7%
game-sm-success_2-1 403.9 μs 557.1 μs +37.9%
game-sm-success_2-2 241.9 μs 286.7 μs +18.5%
game-sm-success_2-3 690.6 μs 913.2 μs +32.2%
game-sm-success_2-4 274.8 μs 268.1 μs -2.4%
game-sm-success_2-5 910.0 μs 634.7 μs -30.3%
game-sm-success_2-6 300.9 μs 232.8 μs -22.6%
multisig-sm-1 563.4 μs 392.9 μs -30.3%
multisig-sm-2 546.1 μs 382.3 μs -30.0%
multisig-sm-3 384.9 μs 387.3 μs +0.6%
multisig-sm-4 470.1 μs 393.0 μs -16.4%
multisig-sm-5 795.6 μs 553.4 μs -30.4%
multisig-sm-6 558.3 μs 393.3 μs -29.6%
multisig-sm-7 546.6 μs 382.8 μs -30.0%
multisig-sm-8 391.0 μs 389.5 μs -0.4%
multisig-sm-9 392.9 μs 395.2 μs +0.6%
multisig-sm-10 557.8 μs 557.7 μs -0.0%
ping-pong-1 329.9 μs 327.9 μs -0.6%
ping-pong-2 328.1 μs 464.0 μs +41.4%
ping-pong_2-1 201.2 μs 286.4 μs +42.3%
prism-1 169.3 μs 227.7 μs +34.5%
prism-2 413.2 μs 407.1 μs -1.5%
prism-3 370.7 μs 382.5 μs +3.2%
pubkey-1 142.0 μs 142.5 μs +0.4%
stablecoin_1-1 907.9 μs 1.283 ms +41.3%
stablecoin_1-2 195.8 μs 281.4 μs +43.7%
stablecoin_1-3 1.046 ms 1.483 ms +41.8%
stablecoin_1-4 209.1 μs 302.9 μs +44.9%
stablecoin_1-5 1.344 ms 1.903 ms +41.6%
stablecoin_1-6 258.4 μs 367.7 μs +42.3%
stablecoin_2-1 907.5 μs 1.282 ms +41.3%
stablecoin_2-2 196.1 μs 280.7 μs +43.1%
stablecoin_2-3 1.130 ms 1.025 ms -9.3%
stablecoin_2-4 295.9 μs 226.1 μs -23.6%
token-account-1 186.3 μs 267.9 μs +43.8%
token-account-2 335.6 μs 477.5 μs +42.3%
uniswap-1 417.0 μs 588.4 μs +41.1%
uniswap-2 220.6 μs 314.6 μs +42.6%
uniswap-3 1.776 ms 2.491 ms +40.3%
uniswap-4 336.2 μs 486.5 μs +44.7%
uniswap-5 1.161 ms 1.649 ms +42.0%
uniswap-6 318.9 μs 459.4 μs +44.1%
vesting-1 347.5 μs 495.5 μs +42.6%
9b4e76a 0a644b5 Change
TOTAL 41.51 ms 46.88 ms +12.9%

@effectfully
Merge branch 'master' of https://github.com/input-output-hk/plutus in…
3b689ae 
…to effectfully/builtins/make-all-builtins-strict
@effectfully effectfully force-pushed the effectfully/builtins/make-all-builtins-strict branch from 0a644b5 to 3b689ae Compare November 7, 2024 11:57
@effectfully
Copy link
Contributor Author

/benchmark validation

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 7, 2024

Click here to check the status of your benchmark.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 7, 2024

Comparing benchmark results of 'validation' on '2411c01a8' (base) and '3b689ae88' (PR)

Results table
Script 2411c01 3b689ae Change
auction_1-1 178.9 μs 185.6 μs +3.7%
auction_1-2 639.7 μs 664.4 μs +3.9%
auction_1-3 634.2 μs 661.8 μs +4.4%
auction_1-4 241.2 μs 240.4 μs -0.3%
auction_2-1 186.6 μs 186.0 μs -0.3%
auction_2-2 661.4 μs 669.4 μs +1.2%
auction_2-3 872.7 μs 872.7 μs 0.0%
auction_2-4 658.8 μs 660.6 μs +0.3%
auction_2-5 241.1 μs 240.4 μs -0.3%
crowdfunding-success-1 218.6 μs 223.7 μs +2.3%
crowdfunding-success-2 219.4 μs 218.3 μs -0.5%
crowdfunding-success-3 219.4 μs 218.3 μs -0.5%
currency-1 353.9 μs 260.3 μs -26.4%
escrow-redeem_1-1 499.7 μs 364.1 μs -27.1%
escrow-redeem_1-2 498.7 μs 363.1 μs -27.2%
escrow-redeem_2-1 583.4 μs 426.3 μs -26.9%
escrow-redeem_2-2 582.5 μs 425.7 μs -26.9%
escrow-redeem_2-3 584.2 μs 426.4 μs -27.0%
escrow-refund-1 224.1 μs 161.0 μs -28.2%
future-increase-margin-1 355.5 μs 260.5 μs -26.7%
future-increase-margin-2 767.9 μs 560.1 μs -27.1%
future-increase-margin-3 768.8 μs 559.5 μs -27.2%
future-increase-margin-4 691.6 μs 505.8 μs -26.9%
future-increase-margin-5 1.102 ms 850.1 μs -22.9%
future-pay-out-1 260.5 μs 260.1 μs -0.2%
future-pay-out-2 563.6 μs 559.5 μs -0.7%
future-pay-out-3 558.9 μs 568.2 μs +1.7%
future-pay-out-4 842.0 μs 846.7 μs +0.6%
future-settle-early-1 260.8 μs 260.0 μs -0.3%
future-settle-early-2 562.7 μs 636.8 μs +13.2%
future-settle-early-3 560.1 μs 534.1 μs -4.6%
future-settle-early-4 628.9 μs 806.3 μs +28.2%
game-sm-success_1-1 493.6 μs 548.1 μs +11.0%
game-sm-success_1-2 202.3 μs 209.5 μs +3.6%
game-sm-success_1-3 641.6 μs 663.3 μs +3.4%
game-sm-success_1-4 236.6 μs 242.8 μs +2.6%
game-sm-success_2-1 391.3 μs 402.0 μs +2.7%
game-sm-success_2-2 206.1 μs 208.0 μs +0.9%
game-sm-success_2-3 657.9 μs 665.5 μs +1.2%
game-sm-success_2-4 244.9 μs 243.2 μs -0.7%
game-sm-success_2-5 759.6 μs 663.2 μs -12.7%
game-sm-success_2-6 235.6 μs 243.4 μs +3.3%
multisig-sm-1 397.4 μs 408.3 μs +2.7%
multisig-sm-2 386.8 μs 398.4 μs +3.0%
multisig-sm-3 391.1 μs 402.9 μs +3.0%
multisig-sm-4 412.9 μs 404.9 μs -1.9%
multisig-sm-5 577.4 μs 575.7 μs -0.3%
multisig-sm-6 410.4 μs 406.9 μs -0.9%
multisig-sm-7 400.4 μs 396.8 μs -0.9%
multisig-sm-8 559.9 μs 401.8 μs -28.2%
multisig-sm-9 559.6 μs 405.5 μs -27.5%
multisig-sm-10 791.7 μs 575.5 μs -27.3%
ping-pong-1 427.2 μs 345.0 μs -19.2%
ping-pong-2 339.4 μs 341.4 μs +0.6%
ping-pong_2-1 208.1 μs 209.0 μs +0.4%
prism-1 177.0 μs 176.5 μs -0.3%
prism-2 428.4 μs 431.2 μs +0.7%
prism-3 384.4 μs 526.9 μs +37.1%
pubkey-1 148.1 μs 190.0 μs +28.3%
stablecoin_1-1 940.2 μs 1.274 ms +35.5%
stablecoin_1-2 205.2 μs 201.6 μs -1.8%
stablecoin_1-3 1.084 ms 1.094 ms +0.9%
stablecoin_1-4 218.2 μs 216.0 μs -1.0%
stablecoin_1-5 1.399 ms 1.395 ms -0.3%
stablecoin_1-6 270.0 μs 267.4 μs -1.0%
stablecoin_2-1 939.7 μs 937.0 μs -0.3%
stablecoin_2-2 205.0 μs 203.5 μs -0.7%
stablecoin_2-3 1.084 ms 1.080 ms -0.4%
stablecoin_2-4 218.3 μs 216.5 μs -0.8%
token-account-1 216.8 μs 194.8 μs -10.1%
token-account-2 335.8 μs 347.1 μs +3.4%
uniswap-1 413.0 μs 430.0 μs +4.1%
uniswap-2 221.9 μs 230.0 μs +3.7%
uniswap-3 1.768 ms 1.834 ms +3.7%
uniswap-4 344.1 μs 349.9 μs +1.7%
uniswap-5 1.161 ms 1.204 ms +3.7%
uniswap-6 382.0 μs 332.2 μs -13.0%
vesting-1 492.9 μs 360.9 μs -26.8%
2411c01 3b689ae Change
TOTAL 39.19 ms 37.53 ms -4.2%

@effectfully
Merge branch 'master' of https://github.com/input-output-hk/plutus in…
1cfcd3e 
…to effectfully/builtins/make-all-builtins-strict
@effectfully
Copy link
Contributor Author

/benchmark validation

@github-actions GitHub Actions
Copy link
Contributor

Click here to check the status of your benchmark.

@github-actions GitHub Actions
Copy link
Contributor

Comparing benchmark results of 'validation' on '9b27bf75f1' (base) and '1cfcd3e5ec' (PR)

Results table
Script 9b27bf7 1cfcd3e Change
auction_1-1 255.8 μs 255.8 μs 0.0%
auction_1-2 911.6 μs 917.8 μs +0.7%
auction_1-3 903.3 μs 916.0 μs +1.4%
auction_1-4 331.4 μs 330.6 μs -0.2%
auction_2-1 256.2 μs 256.2 μs 0.0%
auction_2-2 912.2 μs 916.8 μs +0.5%
auction_2-3 1.194 ms 1.197 ms +0.3%
auction_2-4 905.2 μs 911.1 μs +0.7%
auction_2-5 330.8 μs 331.8 μs +0.3%
crowdfunding-success-1 301.8 μs 299.5 μs -0.8%
crowdfunding-success-2 303.9 μs 302.6 μs -0.4%
crowdfunding-success-3 303.6 μs 302.6 μs -0.3%
currency-1 359.8 μs 360.5 μs +0.2%
escrow-redeem_1-1 499.2 μs 498.5 μs -0.1%
escrow-redeem_1-2 505.2 μs 504.0 μs -0.2%
escrow-redeem_2-1 588.4 μs 588.9 μs +0.1%
escrow-redeem_2-2 587.4 μs 588.3 μs +0.2%
escrow-redeem_2-3 588.9 μs 589.4 μs +0.1%
escrow-refund-1 225.6 μs 223.2 μs -1.1%
future-increase-margin-1 360.5 μs 361.0 μs +0.1%
future-increase-margin-2 770.5 μs 772.5 μs +0.3%
future-increase-margin-3 768.7 μs 769.6 μs +0.1%
future-increase-margin-4 697.8 μs 696.2 μs -0.2%
future-increase-margin-5 1.159 ms 1.155 ms -0.3%
future-pay-out-1 359.2 μs 360.9 μs +0.5%
future-pay-out-2 765.8 μs 773.9 μs +1.1%
future-pay-out-3 767.0 μs 769.7 μs +0.4%
future-pay-out-4 1.151 ms 1.151 ms 0.0%
future-settle-early-1 359.6 μs 360.3 μs +0.2%
future-settle-early-2 771.7 μs 770.5 μs -0.2%
future-settle-early-3 769.0 μs 769.1 μs +0.0%
future-settle-early-4 864.2 μs 869.5 μs +0.6%
game-sm-success_1-1 555.6 μs 555.9 μs +0.1%
game-sm-success_1-2 289.4 μs 287.5 μs -0.7%
game-sm-success_1-3 910.9 μs 911.1 μs +0.0%
game-sm-success_1-4 335.5 μs 334.5 μs -0.3%
game-sm-success_2-1 555.6 μs 556.7 μs +0.2%
game-sm-success_2-2 291.1 μs 286.5 μs -1.6%
game-sm-success_2-3 910.8 μs 912.4 μs +0.2%
game-sm-success_2-4 336.9 μs 335.0 μs -0.6%
game-sm-success_2-5 909.9 μs 911.4 μs +0.2%
game-sm-success_2-6 336.4 μs 334.2 μs -0.7%
multisig-sm-1 562.9 μs 559.3 μs -0.6%
multisig-sm-2 551.5 μs 546.8 μs -0.9%
multisig-sm-3 555.9 μs 552.6 μs -0.6%
multisig-sm-4 561.1 μs 559.1 μs -0.4%
multisig-sm-5 793.4 μs 792.6 μs -0.1%
multisig-sm-6 561.5 μs 557.8 μs -0.7%
multisig-sm-7 549.4 μs 547.5 μs -0.3%
multisig-sm-8 553.5 μs 552.3 μs -0.2%
multisig-sm-9 561.4 μs 557.5 μs -0.7%
multisig-sm-10 792.4 μs 792.0 μs -0.1%
ping-pong-1 464.6 μs 468.0 μs +0.7%
ping-pong-2 466.3 μs 468.9 μs +0.6%
ping-pong_2-1 286.3 μs 286.6 μs +0.1%
prism-1 243.9 μs 242.8 μs -0.5%
prism-2 588.6 μs 588.2 μs -0.1%
prism-3 530.0 μs 531.5 μs +0.3%
pubkey-1 203.9 μs 202.5 μs -0.7%
stablecoin_1-1 1.284 ms 1.300 ms +1.2%
stablecoin_1-2 282.4 μs 280.3 μs -0.7%
stablecoin_1-3 1.486 ms 1.507 ms +1.4%
stablecoin_1-4 301.0 μs 299.7 μs -0.4%
stablecoin_1-5 1.913 ms 1.937 ms +1.3%
stablecoin_1-6 371.0 μs 369.4 μs -0.4%
stablecoin_2-1 1.283 ms 1.289 ms +0.5%
stablecoin_2-2 282.1 μs 281.2 μs -0.3%
stablecoin_2-3 1.486 ms 1.495 ms +0.6%
stablecoin_2-4 300.7 μs 297.8 μs -1.0%
token-account-1 268.9 μs 267.7 μs -0.4%
token-account-2 477.2 μs 478.1 μs +0.2%
uniswap-1 589.2 μs 591.0 μs +0.3%
uniswap-2 315.3 μs 316.9 μs +0.5%
uniswap-3 2.514 ms 2.518 ms +0.2%
uniswap-4 485.2 μs 481.8 μs -0.7%
uniswap-5 1.650 ms 1.652 ms +0.1%
uniswap-6 458.5 μs 457.2 μs -0.3%
vesting-1 494.6 μs 496.0 μs +0.3%
9b27bf7 1cfcd3e Change
TOTAL 50.33 ms 50.42 ms +0.2%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@effectfully effectfully

Labels
Builtins Evaluation
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@effectfully