Patches and test results over the latest PRs

[valassi]

Hi Olivier, Stefan, Stephan, this is a comprehensive patch including several changes over the latest PRs. This is based on the current master including all PRs in the last two months, with some fixes that should be non controversial (clang format, build warnings, build errors etc), and other changes that instead need closer review (e.g. they may break Olivier's Mac).

It also includes new test results for 'my standard processes', using a new Alma9 machine (including downfall mitigation, which seems to have no effect anyway), using the August codebase and the most recent codebase from this branch.

I try to give a summary below. Can you please review (I suggest that you focus on CODEGEN only and on the parts that are potentially controversial). Thanks!

---

Summary of changes (in addition to or including minor fixes already in PRs #778 and #779).

Non-controversial? changes:

• fix build error for eemumu float by adding missing cxsmpl<float> operator*
• fix build warning in counters.cc after Remove smatrix counters and regenerate all processes #762
• fix clang format around Ccoeff following the fix for unary minus wrong type argument to unary minus (FFV functions should not use "-COUPs" as argument) #628
• add copyright and license, add comments, remove too many empty lines in python etc

More controversial changes, can you please review

• reenable lines commented out by Olivier in patchMad.sh: do this only in a special 'tmad mode' called from generateAndCompare.sh, and/or move the lines directly to the latter script (this in principle should also be non controversial and should not affect the full workflow?)
• go back from a ln to a cp for counters.cc and ompthreads.cc (Stephan, maybe yours was better: I just wanted to minimise differences so I got back my working version... but if you prefer and it works, I will try the ln)
• always dump stdout and stderr from subprocess.popen for patchMad.sh (eventually we may remove this maybe, but now I need this to more easily check the logs for errors and avoid silent failures... and personally I prefr to have it dumped anyway as it is a tricky part still)
• in cudacpp.mk, remove 'include ../../Source/make_opts' and remove '-fopenmp' (Olivier this is for you, why do you need that? for me it works without it)
• add a comment that in MatrixElementKernels.cc we should add the correct ifdef for Mac
• in launch_plugin.py, do not raise an Exception in reset_simd, but just print a warning (otherwise my generation fails: by the way I still use _cudacpp, not simd and not cuda)
• in patch.common, reenable some makefile fragments that had been commented out by Olivier (why? Mac?)

---

Coming to the performance and functionality tests.

I have the latest logs in these two commits:
ee83b62
and
536ad49

1. The first log has 'tput' (check.exe, gcheck.exe, runTests.exe) results. This is a summary

Functionality:
** all NaN's have disappeared everywhere!
-- eemumu, ggtt, ggttg, ggttgg, ggttggg: all ok
** gqttq: some failures in runTest.exe

Performance:
-- eemumu: cuda ~20% slower (and 20% increase in registers used)
** eemumu: simd a factor 2 (inl0) to a factor 4 (inl1) slower
-- ggtt: cuda up to 10% slower
-- ggtt: simd ~10-15% faster (inl0), similar (inl1)
-- ggttg, ggttgg, ggttggg: generally similar performance, some minor speedups

Rephasing:

• GOOD: no NaNs anymore! (I would not be surprised if Three floating point exceptions in CPP launch of nobm_pp_ttW (FPE in COUP values in VVV1P0_1) #733 will also disappear)
• TO DO: gqttq failures
• (bad? or who cares?... eemumu performance is degraded, I think by using ixxx instead of imzx)

2. The second log has 'tmad' tests (madevent executable: check xsec and lhe files). This is a summary

Functionality:

• ggttggg: madevent crashes
• gqttq: xsec differs again

Performance (Fortran overhead):
-- all very similar

Performance (MEs):
-- eemumu: fortran 20% faster, cuda slightly slower, simd a factor 2 to 3 slower
-- ggtt: fortran and simd 20% faster, cuda similar
-- ggttg: fortran 10% faster, simd and cuda similar
-- ggttgg: all very similar

+ERROR! ' ./madevent_fortran < /tmp/avalassi/input_ggttggg_x1_fortran > /tmp/avalassi/output_ggttggg_x1_fortran' failed
+d R # 5 > -0.0 -0.0 -0.0 0.4 0.4
+d R # 6 > -0.0 -0.0 -0.0 -0.0 0.4
+s min # 3> 0.0119716.0 29929.0 29929.0 0.0
+s min # 4> 0.0 0.0 29929.0 29929.0 0.0
+s min # 5> 0.0 0.0 0.0 0.0 0.0
+s min # 6> 0.0 0.0 0.0 0.0 0.0
+xqcutij # 3> 0.0 0.0 0.0 0.0 0.0
+xqcutij # 4> 0.0 0.0 0.0 0.0 0.0
+xqcutij # 5> 0.0 0.0 0.0 0.0 0.0
+xqcutij # 6> 0.0 0.0 0.0 0.0 0.0

+ERROR! xsec from fortran (0.26050333309703716) and cpp (1.2757941949814184) differ by more than 2E-14 (3.8974198518457603)

Rephrasing:

• performances are not surprising
• (I was expecting to see the throughput increase reported by Stephan in PR Remove smatrix counters and regenerate all processes #762 ? also because I did have the impression I saw it at some point... or are we sure that this is not again a problem on one specific machine using system calls for timing as in High CPU use in clock_gettime (TSC clocksource unavailable on itscrd03) #116 and the TSC source?)
• gqttq xsec is wrong again? I would therefore reopen xsec from fortran and cpp differ in gq_ttq tmad tests (wrong order of couplings) #748... but Stefan please check (Stefan note: I am kind of sure this had been fixed by PR WIP: if needed fix the order of couplings  #757, but it was broken again by PR fix running couplings ordering #761?)
• I think there is a new problem with ggttggg, I will open a new bug

---

Voila that's all. Thanks for reviewing!

[oliviermattelaer]

I fix the last issue with the pointer. (it was brew that messed up with something, so I have removed fully brew from my system).

But I confirm that I do need to have make_opts to have it compiling on mac.
(More precisely, I need to have gfortran setup as FC which is one of the things done in make_opts).

So one hacky solution to have this PR pushed, is to hardcode FC=gfortran within your makefile (but this is obviously not sustainable). I have pushed that. and will open an issue to track andrea issue with make_opts

[oliviermattelaer]

So the make_opts issue is tracked here: #787

[valassi]

Ok I have the impression that we converged :-)
Thanks @oliviermattelaer!

There is one pending issue in #790 about reset_simd, which is not understood. I assign this to you.

About make_opts, I fixed #787.
Just one point on Mac and gfortran: GNU make assumes FC=f77 if FC is not defined. So indeed in my Linux environment I explicitly export FC=gfortran. But I think this is the correct way
https://www.gnu.org/software/make/manual/html_node/Implicit-Variables.html

As discussed, I then consider this as ready to go.
(@roiser and @hageboeck, last chance for comments!).
I will regenerate all processes and launch all tests first.
So I will merge this evening I think.

[roiser]

NB: I'm have not looked at makefile changes as those will probably be changed soon again

[valassi]

@oliviermattelaer I just realised there are two bugs here

• It should be ifeq not ifneq
• It should be UNAME_S not UNAME
  That is it should read else ifeq ($(UNAME_S), 'Darwin')
  The way it is done now disables OpenMP on Mac AND ALSO on Linux (except icpx and clang)
  I will fix this

[valassi]

Ouf. There were three errors (I noticed because the CI for b647330 failed). No quotes needed. It should read else ifeq ($(UNAME_S),Darwin). Fixing this again

[valassi]

All tests in the CI including Mac are now good. Reviews addressesd. I am finally self merging.