-
Notifications
You must be signed in to change notification settings - Fork 32
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Call the patchMad.sh script from within the plugin's finalize #747
Conversation
…he the patches dir also contains files not necessary for the patching via upstream madgraph)
…ry (NB the the patches dir also contains files not necessary for the patching via upstream madgraph)" This reverts commit d4a2532.
…e the patches dir also contains files not necessary for the patching via upstream madgraph)
0af388d
to
30df485
Compare
…CUDACPP_SA_OUTPUT/MG5aMC_patches/PROD to CODEGEN/MG5aMC_patches/PROD These two files are ONLY needed by the madgraph4gpu generateAndCompare.sh script, they do not need to be exposed in upstream madgraph
…atchMad.sh involved)
…ow called inside the plugin's finalize
…things changed or are missing? - will revert *NB* The log clearly shows that there was a problem running patchMad.sh: DEBUG: 'Entering PLUGIN_ProcessExporter.finalize', self.in_madevent_mode = Entering PLUGIN_ProcessExporter.finalize True [output.py at line 204] Usage: /data/avalassi/GPU2023/MG5aMC/ghav-mg5amcnlo/PLUGIN/CUDACPP_SA_OUTPUT/patchMad.sh <process.[madonly|mad]> <vecsize> <patch_dir> [--nopatch|--upstream]
…d.sh) Revert "[patchMadsh] try to regenerate ggtt.mad, it does not look good, many things changed or are missing? - will revert" This reverts commit b34e3ec.
…ts new location inside the plugin, as it seems not to have worked in finalize
….sh from generateAndCompare (the one inside finalize still fails)
…ain (try instead to fix it inside finalize) Revert "[patchMadsh] in CODEGEN/generateAndCompare.sh, run patchMad.sh from its new location inside the plugin, as it seems not to have worked in finalize" This reverts commit 37c4fa9.
…leanup from that of plugins - and omit plugin cleanup at the end
I am doing further integration tests and patches, to make sure that calling this from generateAndCompare.sh also still works. @roiser I am confused however because, unless I did something different from you, the patchMad.sh scripts is not called from finalize (or better, it is called but it does nothing because it fails and prints an error)
In particular, I will add an error check if possible. And also try to fix this |
…hMad.sh failed This correctly stops code generation DEBUG: 'Entering PLUGIN_ProcessExporter.finalize', self.in_madevent_mode = Entering PLUGIN_ProcessExporter.finalize True [output.py at line 204] Usage: /data/avalassi/GPU2023/MG5aMC/ghav-mg5amcnlo/PLUGIN/CUDACPP_SA_OUTPUT/patchMad.sh <process.[madonly|mad]> <vecsize> <patch_dir> [--nopatch|--upstream] Command "import /data/avalassi/GPU2023/MG5aMC/ghav-mg5amcnlo/CODEGEN_mad_gg_tt.mg" interrupted in sub-command: "output madevent CODEGEN_mad_gg_tt --hel_recycling=False --vector_size=16384 --me_exporter=standalone_cudacpp" with error: Exception : ERROR! the O/S call to patchMad.sh failed Please report this bug on https://bugs.launchpad.net/mg5amcnlo More information is found in 'MG5_debug'. Please attach this file to your report.
…directory is called <proc>.mad or <proc>.madonly to allow the script to be called inside finalize
…till (silent) errors, and some differences with respect to the desired result > cp: cannot stat '/data/avalassi/GPU2023/MG5aMC/ghav-mg5amcnlo/PLUGIN/CUDACPP_SA_OUTPUT/PLUGIN/CUDACPP_SA_OUTPUT/madgraph/iolibs/template_files/.clang-format': No such file or directory > DEBUG: cd /data/avalassi/GPU2023/MG5aMC/ghav-mg5amcnlo/CODEGEN_mad_gg_tt; patch -p4 -i /data/avalassi/GPU2023/MG5aMC/ghav-mg5amcnlo/PLUGIN/CUDACPP_SA_OUTPUT/MG5aMC_patches/PROD/patch.common > patching file Source/genps.inc > patching file Source/makefile > patching file Source/vector.inc > patching file SubProcesses/makefile > patching file Source/make_opts > patching file bin/internal/banner.py > Hunk #1 succeeded at 4187 (offset 23 lines). > patching file bin/internal/gen_ximprove.py > patching file bin/internal/madevent_interface.py > cp: cannot stat '/data/avalassi/GPU2023/MG5aMC/ghav-mg5amcnlo/PLUGIN/CUDACPP_SA_OUTPUT/PLUGIN/CUDACPP_SA_OUTPUT/madgraph/iolibs/template_files/gpu/timer.h': No such file or directory > DEBUG: cd /data/avalassi/GPU2023/MG5aMC/ghav-mg5amcnlo/CODEGEN_mad_gg_tt/SubProcesses/P1_gg_ttx; patch -p6 -i /data/avalassi/GPU2023/MG5aMC/ghav-mg5amcnlo/PLUGIN/CUDACPP_SA_OUTPUT/MG5aMC_patches/PROD/patch.P1 > patching file auto_dsig1.f > patching file driver.f > patching file matrix1.f
…iate exit on errors The script now correctly fails INFO: cp: cannot stat '/data/avalassi/GPU2023/MG5aMC/ghav-mg5amcnlo/PLUGIN/CUDACPP_SA_OUTPUT/PLUGIN/CUDACPP_SA_OUTPUT/madgraph/iolibs/template_files/.clang-format': No such file or directory Command "import /data/avalassi/GPU2023/MG5aMC/ghav-mg5amcnlo/CODEGEN_mad_gg_tt.mg" interrupted in sub-command: "output madevent CODEGEN_mad_gg_tt --hel_recycling=False --vector_size=16384 --me_exporter=standalone_cudacpp" with error: Exception : ERROR! the O/S call to patchMad.sh failed Please report this bug on https://bugs.launchpad.net/mg5amcnlo More information is found in 'MG5_debug'. Please attach this file to your report. quit
…mat (this is in the plugin since a long time anyway...) The script now fails a bit later cp: cannot stat '/data/avalassi/GPU2023/MG5aMC/ghav-mg5amcnlo/PLUGIN/CUDACPP_SA_OUTPUT/PLUGIN/CUDACPP_SA_OUTPUT/madgraph/iolibs/template_files/gpu/timer.h': No such file or directory Command "import /data/avalassi/GPU2023/MG5aMC/ghav-mg5amcnlo/CODEGEN_mad_gg_tt.mg" interrupted in sub-command: "output madevent CODEGEN_mad_gg_tt --hel_recycling=False --vector_size=16384 --me_exporter=standalone_cudacpp" with error: Exception : ERROR! the O/S call to patchMad.sh failed Please report this bug on https://bugs.launchpad.net/mg5amcnlo More information is found in 'MG5_debug'. Please attach this file to your report.
…mpts at regenerating git checkout upstream/master gg_tt.mad
…hich is already copied via the plugin) Generation now seems to succeed
…ster, with a few differences Details - banner.py has additional text (I guess this is intended in the upstream change of gpucpp) - param_card.inc now has some values sorted in a different way, but everything is there - run_card.inc now has changed SDE_STRAT=1 to SDE_STRAT=2
…_STRAT = 2 to 1 instead (#419)
…ut this is not yet in patchMad.sh)
…erateAndCompare.sh to patchMad.sh
…ctor.inc (will eventually use runcards instead)
…ow simpler but still has 16384)
…via the hidden env variable CUDACPP_CODEGEN_PATCHLEVEL
… new MG5AMC patches as-is (in case, this should go to the plugin...)
…e first time in the new patch location (NB also remove vector.inc which no longer needs patches) ./CODEGEN/generateAndCompare.sh gg_tt --mad --nopatch sed -i 's/DEFAULT_F2PY_COMPILER=f2py3.*/DEFAULT_F2PY_COMPILER=f2py3/' gg_tt.mad/Source/make_opts git diff --no-ext-diff -R gg_tt.mad/Source/makefile gg_tt.mad/Source/dsample.f gg_tt.mad/Source/genps.inc gg_tt.mad/SubProcesses/makefile > CODEGEN/PLUGIN/CUDACPP_SA_OUTPUT/MG5aMC_patches/PROD/patch.common git diff --no-ext-diff -R gg_tt.mad/Source/make_opts gg_tt.mad/bin/internal/banner.py gg_tt.mad/bin/internal/gen_ximprove.py gg_tt.mad/bin/internal/madevent_interface.py >> CODEGEN/PLUGIN/CUDACPP_SA_OUTPUT/MG5aMC_patches/PROD/patch.common git diff --no-ext-diff -R gg_tt.mad/SubProcesses/P1_gg_ttx/auto_dsig1.f gg_tt.mad/SubProcesses/P1_gg_ttx/driver.f gg_tt.mad/SubProcesses/P1_gg_ttx/matrix1.f > CODEGEN/PLUGIN/CUDACPP_SA_OUTPUT/MG5aMC_patches/PROD/patch.P1 git checkout gg_tt.mad
I have also fixed all other pending issues
Now rerunning tests, should be able to merge soon |
I will wait for the tests running tonight to succeed, and will merge tomorrow. This will close issue #656. Note that the strategy for integrating patchMad.sh patches upstream has changed:
|
…madevent+cudacpp mode (also move to self.in_madevent_mode in one place only)
… which does not work where I tried to use it)
Also checked that these are identical to those created in-situ in MG5AMC, except for a few minor points - some .gitignore and CODEGEN logs are only in madgraph4gpu - the VECSIZE_MEMMAX depends on whatever was used upstream... - the Cards/me5_configuration.txt may have some differences - SubProcesses/proc_characteristics has different values of bias_modules (I noted in the genAndComp script that this is irrelevant) - the order of parameters in Cards/ident_card.dat is different (oliviermattelaer/mg5amc_test#2)
…anges - changes only in .mad python (mainly for combine events)
(Note: will rerun tput due to nsight config errors from AFS expired, #750) STARTED AT Mon Aug 14 22:01:13 CEST 2023 ENDED AT Tue Aug 15 02:20:34 CEST 2023 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_eemumu_mad/log_eemumu_mad_d_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_eemumu_mad/log_eemumu_mad_f_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_eemumu_mad/log_eemumu_mad_m_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggttggg_mad/log_ggttggg_mad_d_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggttggg_mad/log_ggttggg_mad_f_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggttggg_mad/log_ggttggg_mad_m_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggttgg_mad/log_ggttgg_mad_d_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggttgg_mad/log_ggttgg_mad_f_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggttgg_mad/log_ggttgg_mad_m_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggttg_mad/log_ggttg_mad_d_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggttg_mad/log_ggttg_mad_f_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggttg_mad/log_ggttg_mad_m_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggtt_mad/log_ggtt_mad_d_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggtt_mad/log_ggtt_mad_f_inl0_hrd0.txt 24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggtt_mad/log_ggtt_mad_m_inl0_hrd0.txt 0 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_gqttq_mad/log_gqttq_mad_d_inl0_hrd0.txt 0 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_gqttq_mad/log_gqttq_mad_f_inl0_hrd0.txt 0 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_gqttq_mad/log_gqttq_mad_m_inl0_hrd0.txt
STARTED AT Tue Aug 15 07:47:04 CEST 2023 ./tput/teeThroughputX.sh -mix -hrd -makej -eemumu -ggtt -ggttg -ggttgg -gqttq -ggttggg -makeclean ENDED(1) AT Tue Aug 15 08:14:00 CEST 2023 [Status=0] ./tput/teeThroughputX.sh -flt -hrd -makej -eemumu -ggtt -ggttgg -inlonly -makeclean ENDED(2) AT Tue Aug 15 08:23:38 CEST 2023 [Status=0] ./tput/teeThroughputX.sh -makej -eemumu -ggtt -ggttg -gqttq -ggttgg -ggttggg -flt -bridge -makeclean ENDED(3) AT Tue Aug 15 08:33:37 CEST 2023 [Status=0] ./tput/teeThroughputX.sh -eemumu -ggtt -ggttgg -flt -rmbhst ENDED(4) AT Tue Aug 15 08:36:51 CEST 2023 [Status=0] ./tput/teeThroughputX.sh -eemumu -ggtt -ggttgg -flt -curhst ENDED(5) AT Tue Aug 15 08:40:02 CEST 2023 [Status=0]
41b378d
to
2cb7e11
Compare
The CI tests passsed - I am self merging |
first version of calling the patch script
patch script and patch files moved into the plugin directory