Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Option to produce multi-build (fortran, cuda, cpp) gridpack #945

Open
valassi opened this issue Aug 2, 2024 · 0 comments · May be fixed by #948
Open

Option to produce multi-build (fortran, cuda, cpp) gridpack #945

valassi opened this issue Aug 2, 2024 · 0 comments · May be fixed by #948
Assignees

Comments

@valassi
Copy link
Member

valassi commented Aug 2, 2024

I am looking at #943: why does CMS see speedups in DY+4jets but not DY+3 jets

The way I would like to address this is to produce a multi-build (fortran, cuda, cpp) gridpack

That is to say:

  • do a 'make -j bldall' once, which includes fortran, cuda, cpp builds
  • do a vegas optimisation once (possibly in cuda which is the fastest)
  • then compare only event generation in the various ME implementations

This is related to #674 about madevent_fortran, madevent_cuda, madevent_cpp.

  • Our current strategy is about having a symlink to madevent which is then used everywhere. Another idea I had in mind for some time (not sure it is even written in 674 or elsewhere yet) is to have madevent as a script which then calls the appropriate madevent_cuda etc executable in its build directory. This may be more flexible.
  • A second question, described in madeventfortran vs madeventcpp vs madeventcuda #674, is the usual thing of vecsize_memmax etc. Strictly speaking, an optimal fortran or cpp should have vecsize 32, not larger. Instead cuda has typically 8192. So one would need separate build directories (or dynamic memory allocations) etc. Anyway for the moment I'd be happy even to just compare cuda, fortran, cpp all with 8192 vecsizes.
@valassi valassi self-assigned this Aug 2, 2024
valassi added a commit to valassi/madgraph4gpu that referenced this issue Aug 7, 2024
…ILDDIR=1 to make_opts (part of madgraph5#945)

Then regenerate gg_tt.mad: this only affects launch_plugin.py itself

Launching lauX.sh on ggtt fails as follows

Error detected in "generate_events -f"
write debug file /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/gg_tt.mad/run_01_tag_1_debug.log
If you need help with this issue please contact us on https://answers.launchpad.net/mg5amcnlo
str : A compilation Error occurs when trying to compile /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/gg_tt.mad/SubProcesses/P1_gg_ttx.
        The compilation fails with the following output message:
            make USEGTEST=0 BACKEND=cuda build.auto_m_inl0_hrd0/madevent_cuda
            make[1]: Entering directory '/data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/gg_tt.mad/SubProcesses/P1_gg_ttx'
            make[1]: *** No rule to make target 'build.auto_m_inl0_hrd0/madevent_cuda'.  Stop.
            make[1]: Leaving directory '/data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/gg_tt.mad/SubProcesses/P1_gg_ttx'
            make: *** [makefile:163: madevent_cuda_link] Error 2

        Please try to fix this compilations issue and retry.
        Help might be found at https://answers.launchpad.net/mg5amcnlo.
        If you think that this is a bug, you can report this at https://bugs.launchpad.net/mg5amcnlo
valassi added a commit to valassi/madgraph4gpu that referenced this issue Aug 7, 2024
… madevent_xxx_link' in args[0][0] to support USEBUILDDIR=1 (part of madgraph5#945)

Then regenerate gg_tt.mad

However lauX.sh still fails
valassi added a commit to valassi/madgraph4gpu that referenced this issue Aug 7, 2024
…][1] to 'make madevent_xxx_link' to support USEBUILDDIR=1 (part of madgraph5#945)

Then regenerate gg_tt.mad
valassi added a commit to valassi/madgraph4gpu that referenced this issue Aug 7, 2024
…two patches, will fix gg_tt.mad makefile instead

Revert "[runcard] in CODEGEN launch_plugin.py, append 'BACKEND=xxx' as args[0][1] to 'make madevent_xxx_link' to support USEBUILDDIR=1 (part of madgraph5#945)"
This reverts commit ce67fb5.

Revert "[runcard] in CODEGEN launch_plugin.py, prepend 'BACKEND=xxx' to 'make madevent_xxx_link' in args[0][0] to support USEBUILDDIR=1 (part of madgraph5#945)"
This reverts commit b51a59a.
valassi added a commit to valassi/madgraph4gpu that referenced this issue Aug 7, 2024
valassi added a commit to valassi/madgraph4gpu that referenced this issue Aug 7, 2024
…targets instead of having this in a python script (part of madgraph5#945)
valassi added a commit to valassi/madgraph4gpu that referenced this issue Aug 7, 2024
…_opts configuration, this will be hardcoded in link targets (part of madgraph5#945)"

Revert "[runcard] in CODEGEN launch_plugin.py, configure compile to add USEBUILDDIR=1 to make_opts (part of madgraph5#945)"
This reverts commit 853a20e.
valassi added a commit to valassi/madgraph4gpu that referenced this issue Aug 7, 2024
…(and propagate FPTYPE from make_opts) in _link targets for madgraph5#945

The only files that still need to be patched are
- 3 in patch.common: Source/makefile, Source/genps.inc, SubProcesses/makefile
- 3 in patch.P1: auto_dsig1.f, driver.f, matrix1.f

./CODEGEN/generateAndCompare.sh gg_tt --mad --nopatch
git diff --no-ext-diff -R gg_tt.mad/Source/makefile gg_tt.mad/Source/genps.inc gg_tt.mad/SubProcesses/makefile > CODEGEN/PLUGIN/CUDACPP_SA_OUTPUT/MG5aMC_patches/PROD/patch.common
git diff --no-ext-diff -R gg_tt.mad/SubProcesses/P1_gg_ttx/auto_dsig1.f gg_tt.mad/SubProcesses/P1_gg_ttx/driver.f gg_tt.mad/SubProcesses/P1_gg_ttx/matrix1.f > CODEGEN/PLUGIN/CUDACPP_SA_OUTPUT/MG5aMC_patches/PROD/patch.P1
git checkout gg_tt.mad
valassi added a commit to valassi/madgraph4gpu that referenced this issue Aug 7, 2024
valassi added a commit to valassi/madgraph4gpu that referenced this issue Aug 7, 2024
…s and remove it from _link targets madgraph5#945

(Also fix the cpp targets where I had forgotten one part of my previous patch)

The only files that still need to be patched are
- 3 in patch.common: Source/makefile, Source/genps.inc, SubProcesses/makefile
- 3 in patch.P1: auto_dsig1.f, driver.f, matrix1.f

./CODEGEN/generateAndCompare.sh gg_tt --mad --nopatch
git diff --no-ext-diff -R gg_tt.mad/Source/makefile gg_tt.mad/Source/genps.inc gg_tt.mad/SubProcesses/makefile > CODEGEN/PLUGIN/CUDACPP_SA_OUTPUT/MG5aMC_patches/PROD/patch.common
git diff --no-ext-diff -R gg_tt.mad/SubProcesses/P1_gg_ttx/auto_dsig1.f gg_tt.mad/SubProcesses/P1_gg_ttx/driver.f gg_tt.mad/SubProcesses/P1_gg_ttx/matrix1.f > CODEGEN/PLUGIN/CUDACPP_SA_OUTPUT/MG5aMC_patches/PROD/patch.P1
git checkout gg_tt.mad
valassi added a commit to valassi/madgraph4gpu that referenced this issue Aug 7, 2024
valassi added a commit to valassi/madgraph4gpu that referenced this issue Aug 7, 2024
valassi added a commit to valassi/madgraph4gpu that referenced this issue Aug 7, 2024
…to grid

Fix conflicts:	epochX/cudacpp/tlau/lauX.sh
valassi added a commit to valassi/madgraph4gpu that referenced this issue Aug 7, 2024
valassi added a commit to valassi/madgraph4gpu that referenced this issue Aug 7, 2024
valassi added a commit to valassi/madgraph4gpu that referenced this issue Oct 7, 2024
…ranch grid

This includes in summary
- madgraph5#700 add cudacpp_helinl, cudacpp_hrdcod, rename cudacpp_fptype
- madgraph5#945 add cudacpp_bldall for multi-backend gridpacks
- madgraph5#957 enhance tlau tests: instrument python code for gridpack profiling, add test logs and scripts to parse them
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant