hca: non-deterministic codesize results

[kjetilos]

When running analysis using the hca script on a small binary I see that the code size saving varies between executions of the script. My expectations are that the optimizations/transformations are deterministic and that the code size results should be the same each time. Here is sample output from the test:

$ for i in {1..20}; do "C:/Python/Python37/Python.exe" tools/HCA/hca -a c_sext_b c_sext_h c_zext_b c_zext_h c_zext_w beqi bnei muli c_mul c_not c_neg c_tblj c_tbljal c_tbljalm c_mva01s07 pushpop pushpop,both c_sb c_sh c_lhu c_lbu c_lh c_lb sp,lbu sp,lhu sp,sb sp,sh size -f binaries/blink_rv32imac_no_save_restore.out | head -1 ; done
blink_rv32imac_no_save_restore.out, 9134.0 , 7642.0 , 16.33%
blink_rv32imac_no_save_restore.out, 9134.0 , 7648.0 , 16.27%
blink_rv32imac_no_save_restore.out, 9134.0 , 7628.0 , 16.49%
blink_rv32imac_no_save_restore.out, 9134.0 , 7634.0 , 16.42%
blink_rv32imac_no_save_restore.out, 9134.0 , 7632.0 , 16.44%
blink_rv32imac_no_save_restore.out, 9134.0 , 7628.0 , 16.49%
blink_rv32imac_no_save_restore.out, 9134.0 , 7668.0 , 16.05%
blink_rv32imac_no_save_restore.out, 9134.0 , 7642.0 , 16.33%
blink_rv32imac_no_save_restore.out, 9134.0 , 7636.0 , 16.40%
blink_rv32imac_no_save_restore.out, 9134.0 , 7630.0 , 16.47%
blink_rv32imac_no_save_restore.out, 9134.0 , 7650.0 , 16.25%
blink_rv32imac_no_save_restore.out, 9134.0 , 7664.0 , 16.09%
blink_rv32imac_no_save_restore.out, 9134.0 , 7638.0 , 16.38%
blink_rv32imac_no_save_restore.out, 9134.0 , 7666.0 , 16.07%
blink_rv32imac_no_save_restore.out, 9134.0 , 7630.0 , 16.47%
blink_rv32imac_no_save_restore.out, 9134.0 , 7652.0 , 16.23%
blink_rv32imac_no_save_restore.out, 9134.0 , 7638.0 , 16.38%
blink_rv32imac_no_save_restore.out, 9134.0 , 7662.0 , 16.12%
blink_rv32imac_no_save_restore.out, 9134.0 , 7636.0 , 16.40%
blink_rv32imac_no_save_restore.out, 9134.0 , 7648.0 , 16.27%

As we see the results vary between 7628 and 7668. Is there a way to make this deterministic so we get the same results each time we run the script?

[abukharmeh]

Thank you for reporting this, it should be indeed deterministic, I will investigate this.

[abukharmeh]

Could you please confirm that if you remove BEQI and BNEI, the results would be always deterministic.

[kjetilos]

Yes, that is correct. When I remove both BEQI and BNEI the results are deterministic, if I have any of them applied then the result is non-deterministic.

[abukharmeh]

I have fixed this, but I need to verify that the changes I made make sense across my benchmarking suite. This might take a while, I will keep this Issue open until we push changes.