New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

remove gXXX.cu symlinks (build XXX_cu.o from XXX.cc) #368

Merged

valassi merged 19 commits into madgraph5:master from valassi:makefiles

Feb 5, 2024

Member

valassi commented Feb 1, 2022

Hi @roiser @oliviermattelaer I would eventually go in this direction, is this ok for you?
(I would only keep the names check.exe and gcheck.exe for now, this is the only one starting with g... it is easier to find to run).
Thanks Andrea

valassi added 2 commits

February 1, 2022 18:09


          [makefiles] remove g*.cu symlinks (except for gcheck_sa.cu), build as…

1c8c16b

… *_cu.o from *.cc


          [makefiles] remove also gcheck_sa.cu, build check_sa_cu.o from check_…

eda6253

…sa.cc

valassi marked this pull request as draft

February 1, 2022 17:18

valassi force-pushed the makefiles branch from eda6253 to aae0105 Compare

November 21, 2023 14:22

valassi added 3 commits

November 21, 2023 16:54


          [makefiles] revert the two commits from Feb 2022 - will merge in upst…

cd46fa4

…ream/master and recreate them from scratch

Revert "[makefiles] remove also gcheck_sa.cu, build check_sa_cu.o from check_sa.cc"
This reverts commit eda6253.

Revert "[makefiles] remove g*.cu symlinks (except for gcheck_sa.cu), build as *_cu.o from *.cc"
This reverts commit 1c8c16b.


          Merge remote-tracking branch 'upstream/master' into makefiles

f8a517b


          [makefiles] in gg_tt.mad, remove g*.cu symlinks (including gcheck_sa.…

f3e7012

…cu), build as *_cu.o from *.cc"

This is the port to gg_tt.mad over Nov 2023 upstream/master of the changes originally committed in gg_tt over the Feb 2022 code base

valassi force-pushed the makefiles branch from aae0105 to f3e7012 Compare

November 21, 2023 16:14

valassi added 2 commits

November 22, 2023 11:03


          [makefiles] backport removal of g*.cu symlinks from gg_tt.mad to CODEGEN

7b8ed4d


          [makefiles] regenerate gg_tt.mad, check that all is ok

f819518

valassi requested a review from hageboeck

November 22, 2023 10:10

valassi changed the title ~~WIP: remove gXXX.cu symlinks (build XXX_cu.o from XXX.cc)~~ remove gXXX.cu symlinks (build XXX_cu.o from XXX.cc)

valassi self-assigned this

valassi marked this pull request as ready for review

November 22, 2023 10:10

Member Author

valassi commented Nov 22, 2023

Hi @hageboeck @roiser as discussed this morning, this is a very simple MR that removes g*.cu symlinks and builds instead xxx_cu.o directly from xxx.c

The only exception (for now?) of a gxxx file is the gcheck.exe executable (built from check_sa_cu.o).

Any feedback? Thanks

Member

roiser commented Nov 22, 2023

Thanks @valassi I think its a good move and @hageboeck would probably integrate this into his ongoing work for the Makefiles

Member Author

valassi commented Dec 16, 2023

I have just merged #706, I will modify this PR to merge in the new master

valassi added 3 commits

December 16, 2023 09:34


          [makefiles] ** COMPLETE MAKEFILES (but need to merge in upstream/mast…

4d5b341

…er including PR madgraph5#706) ** regenerate all processes, removing g*.cu symlinks


          Merge remote-tracking branch 'upstream/master' (including merged PR m…

a1599a9

…adgraph5#706) into makefiles

Fix conflicts in CODEGEN logs by checking them out from upstream/master
git checkout upstream/master $(git ls-tree --name-only HEAD */CODEGEN*txt)


          [makefiles] ** COMPLETE MAKEFILES ** regenerate all processes (after …

4485c54

…merging in upstream/master including PR madgraph5#706) - ok, changes are only in codegen logs

valassi force-pushed the makefiles branch from fcf4cd4 to 4485c54 Compare

December 16, 2023 08:42

This was referenced Jan 27, 2024

Jorgen's addition of HIP/CUDA abstraction to codegen and all processes (with extra fixes and tests: "jt774" replacing PR #774) #801

Merged

Allow cucomplex and thrust only for CUDA but not for HIP? #810

Closed

valassi added 7 commits

February 2, 2024 13:26


          [makefiles] prepare to merge upstream/master: go back to last common …

7c28f76

…ancestor for generated code

git merge-base --fork-point upstream/master
80ff716

git checkout 80ff716 $(git ls-tree --name-only HEAD *.mad *.sa)


          Merge remote-tracking branch 'upstream/master' (as of after HIP PR ma…

90dd567

…dgraph5#801 and gpucpp PR madgraph5#811) into makefiles

Fix conflicts in epochX/cudacpp/CODEGEN/PLUGIN/CUDACPP_SA_OUTPUT/madgraph/iolibs/template_files/gpu/cudacpp.mk


          [makefiles] regenerate gg_tt.mad (now without gXXX.cu links) - tput a…

38f9ccc

…nd tmad tests look ok


          [makefiles] regenerate all processes (without gXXX.cu links)

ef8c2ae


          [makefiles] rerun 78 tput tests on itscrd90, all ok

d1c0bac

STARTED  AT Fri Feb  2 01:56:53 PM CET 2024
./tput/teeThroughputX.sh -mix -hrd -makej -eemumu -ggtt -ggttg -ggttgg -gqttq -ggttggg -makeclean
ENDED(1) AT Fri Feb  2 04:49:47 PM CET 2024 [Status=0]
./tput/teeThroughputX.sh -flt -hrd -makej -eemumu -ggtt -ggttgg -inlonly -makeclean
ENDED(2) AT Fri Feb  2 05:06:05 PM CET 2024 [Status=0]
./tput/teeThroughputX.sh -makej -eemumu -ggtt -ggttg -gqttq -ggttgg -ggttggg -flt -bridge -makeclean
ENDED(3) AT Fri Feb  2 05:16:09 PM CET 2024 [Status=0]
./tput/teeThroughputX.sh -eemumu -ggtt -ggttgg -flt -rmbhst
ENDED(4) AT Fri Feb  2 05:19:30 PM CET 2024 [Status=0]
./tput/teeThroughputX.sh -eemumu -ggtt -ggttgg -flt -curhst
ENDED(5) AT Fri Feb  2 05:22:51 PM CET 2024 [Status=0]


          [makefiles] rerun 18 tmad tests on itscrd90, all ok (note, performanc…

a61dab0

…e may be degraded by other activities on the node)

STARTED AT Fri Feb  2 05:26:15 PM CET 2024
ENDED   AT Fri Feb  2 09:48:14 PM CET 2024

Status=0

24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_eemumu_mad/log_eemumu_mad_d_inl0_hrd0.txt
24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_eemumu_mad/log_eemumu_mad_f_inl0_hrd0.txt
24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_eemumu_mad/log_eemumu_mad_m_inl0_hrd0.txt
24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggttggg_mad/log_ggttggg_mad_d_inl0_hrd0.txt
24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggttggg_mad/log_ggttggg_mad_f_inl0_hrd0.txt
24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggttggg_mad/log_ggttggg_mad_m_inl0_hrd0.txt
24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggttgg_mad/log_ggttgg_mad_d_inl0_hrd0.txt
24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggttgg_mad/log_ggttgg_mad_f_inl0_hrd0.txt
24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggttgg_mad/log_ggttgg_mad_m_inl0_hrd0.txt
24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggttg_mad/log_ggttg_mad_d_inl0_hrd0.txt
24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggttg_mad/log_ggttg_mad_f_inl0_hrd0.txt
24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggttg_mad/log_ggttg_mad_m_inl0_hrd0.txt
24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggtt_mad/log_ggtt_mad_d_inl0_hrd0.txt
24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggtt_mad/log_ggtt_mad_f_inl0_hrd0.txt
24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggtt_mad/log_ggtt_mad_m_inl0_hrd0.txt
24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_gqttq_mad/log_gqttq_mad_d_inl0_hrd0.txt
24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_gqttq_mad/log_gqttq_mad_f_inl0_hrd0.txt
24 /data/avalassi/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_gqttq_mad/log_gqttq_mad_m_inl0_hrd0.txt


          [makefiles] rerun 78 tput tests on LUMI, all as expected

6e90ff8

(1) build on the login node

./tput/allTees.sh -hip -makeonly

STARTED  AT Fri 02 Feb 2024 02:56:07 PM EET
./tput/teeThroughputX.sh -mix -hrd -makej -eemumu -ggtt -ggttg -ggttgg -gqttq -ggttggg -makeclean  -makeonly
ENDED(1) AT Sat 03 Feb 2024 01:13:14 PM EET [Status=0]
./tput/teeThroughputX.sh -flt -hrd -makej -eemumu -ggtt -ggttgg -inlonly -makeclean  -makeonly
ENDED(2) AT Sat 03 Feb 2024 01:30:24 PM EET [Status=0]
./tput/teeThroughputX.sh -makej -eemumu -ggtt -ggttg -gqttq -ggttgg -ggttggg -flt -bridge -makeclean  -makeonly
ENDED(3) AT Sat 03 Feb 2024 01:42:44 PM EET [Status=0]
./tput/teeThroughputX.sh -eemumu -ggtt -ggttgg -flt -rmbhst  -makeonly
ENDED(4) AT Sat 03 Feb 2024 01:44:56 PM EET [Status=0]
./tput/teeThroughputX.sh -eemumu -ggtt -ggttgg -flt -rorhst  -makeonly
ENDED(5) AT Sat 03 Feb 2024 01:47:06 PM EET [Status=0]

(2) test step on the worker node

./tput/allTees.sh

STARTED  AT Sat 03 Feb 2024 06:05:42 PM EET
./tput/teeThroughputX.sh -mix -hrd -makej -eemumu -ggtt -ggttg -ggttgg -gqttq -ggttggg -makeclean
ENDED(1) AT Sat 03 Feb 2024 06:58:27 PM EET [Status=2]
./tput/teeThroughputX.sh -flt -hrd -makej -eemumu -ggtt -ggttgg -inlonly -makeclean
ENDED(2) AT Sat 03 Feb 2024 07:15:46 PM EET [Status=0]
./tput/teeThroughputX.sh -makej -eemumu -ggtt -ggttg -gqttq -ggttgg -ggttggg -flt -bridge -makeclean
ENDED(3) AT Sat 03 Feb 2024 07:34:49 PM EET [Status=2]
./tput/teeThroughputX.sh -eemumu -ggtt -ggttgg -flt -rmbhst
ENDED(4) AT Sat 03 Feb 2024 07:38:37 PM EET [Status=0]
./tput/teeThroughputX.sh -eemumu -ggtt -ggttgg -flt -curhst
ENDED(5) AT Sat 03 Feb 2024 07:40:57 PM EET [Status=0]

./tput/logs_gqttq_mad/log_gqttq_mad_d_inl0_hrd0_bridge.txt:Backtrace for this error:
./tput/logs_gqttq_mad/log_gqttq_mad_d_inl0_hrd0_bridge.txt:ERROR! Fortran calculation (F77/CUDA) crashed
./tput/logs_gqttq_mad/log_gqttq_mad_m_inl0_hrd1.txt:Backtrace for this error:
./tput/logs_gqttq_mad/log_gqttq_mad_m_inl0_hrd1.txt:ERROR! Fortran calculation (F77/CUDA) crashed
./tput/logs_gqttq_mad/log_gqttq_mad_d_inl0_hrd1.txt:Backtrace for this error:
./tput/logs_gqttq_mad/log_gqttq_mad_d_inl0_hrd1.txt:ERROR! Fortran calculation (F77/CUDA) crashed
./tput/logs_gqttq_mad/log_gqttq_mad_d_inl0_hrd0.txt:Backtrace for this error:
./tput/logs_gqttq_mad/log_gqttq_mad_d_inl0_hrd0.txt:ERROR! Fortran calculation (F77/CUDA) crashed
./tput/logs_gqttq_mad/log_gqttq_mad_m_inl0_hrd0.txt:Backtrace for this error:
./tput/logs_gqttq_mad/log_gqttq_mad_m_inl0_hrd0.txt:ERROR! Fortran calculation (F77/CUDA) crashed
./tput/logs_gqttq_mad/log_gqttq_mad_f_inl0_hrd1.txt:Backtrace for this error:
./tput/logs_gqttq_mad/log_gqttq_mad_f_inl0_hrd1.txt:ERROR! Fortran calculation (F77/CUDA) crashed
./tput/logs_gqttq_mad/log_gqttq_mad_f_inl0_hrd0.txt:Backtrace for this error:
./tput/logs_gqttq_mad/log_gqttq_mad_f_inl0_hrd0.txt:ERROR! Fortran calculation (F77/CUDA) crashed
./tput/logs_gqttq_mad/log_gqttq_mad_f_inl0_hrd0_bridge.txt:Backtrace for this error:
./tput/logs_gqttq_mad/log_gqttq_mad_f_inl0_hrd0_bridge.txt:ERROR! Fortran calculation (F77/CUDA) crashed

valassi added 2 commits

February 4, 2024 21:53


          [makefiles] rerun 18 tmad tests on LUMI worker nodes, all as good as …

5cc9724

…it gets (gqttq issues)

(1) Step 1 all but ggttggg

STARTED AT Sat 03 Feb 2024 07:49:25 PM EET
ENDED   AT Sat 03 Feb 2024 08:13:27 PM EET

(2) Step 2 only ggttggg (in parallel, finished last)

STARTED AT Sat 03 Feb 2024 07:52:26 PM EET
ENDED   AT Sat 03 Feb 2024 10:31:15 PM EET

Status=0

16 /users/valassia/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_eemumu_mad/log_eemumu_mad_d_inl0_hrd0.txt
16 /users/valassia/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_eemumu_mad/log_eemumu_mad_f_inl0_hrd0.txt
16 /users/valassia/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_eemumu_mad/log_eemumu_mad_m_inl0_hrd0.txt
16 /users/valassia/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggttggg_mad/log_ggttggg_mad_d_inl0_hrd0.txt
16 /users/valassia/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggttggg_mad/log_ggttggg_mad_f_inl0_hrd0.txt
16 /users/valassia/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggttggg_mad/log_ggttggg_mad_m_inl0_hrd0.txt
16 /users/valassia/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggttgg_mad/log_ggttgg_mad_d_inl0_hrd0.txt
16 /users/valassia/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggttgg_mad/log_ggttgg_mad_f_inl0_hrd0.txt
16 /users/valassia/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggttgg_mad/log_ggttgg_mad_m_inl0_hrd0.txt
16 /users/valassia/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggttg_mad/log_ggttg_mad_d_inl0_hrd0.txt
16 /users/valassia/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggttg_mad/log_ggttg_mad_f_inl0_hrd0.txt
16 /users/valassia/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggttg_mad/log_ggttg_mad_m_inl0_hrd0.txt
16 /users/valassia/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggtt_mad/log_ggtt_mad_d_inl0_hrd0.txt
16 /users/valassia/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggtt_mad/log_ggtt_mad_f_inl0_hrd0.txt
16 /users/valassia/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_ggtt_mad/log_ggtt_mad_m_inl0_hrd0.txt
12 /users/valassia/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_gqttq_mad/log_gqttq_mad_d_inl0_hrd0.txt
12 /users/valassia/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_gqttq_mad/log_gqttq_mad_f_inl0_hrd0.txt
12 /users/valassia/GPU2023/madgraph4gpuX/epochX/cudacpp/tmad/logs_gqttq_mad/log_gqttq_mad_m_inl0_hrd0.txt


          [makefiles] ** COMPLETE MAKEFILES ** go back to tput and tmad logs fr…

bf3d262

…om itscrd90

git checkout a61dab0 tput tmad

Member Author

valassi commented Feb 5, 2024

Hi @oliviermattelaer this other PR is also ready for merging.

This simplifies generated code (by removing gXXX.cu symlinks to XXX.cc files) and simplifies makefiles (by only foreseeing the build of cuda from .cc files using "-x cu", rather than relying on a .cu suffix: previously both of these were supported for different files).

The main change is this one 7b8ed4d

Can you please @oliviermattelaer review and approve for merging?
Thanks
Andrea

valassi requested a review from oliviermattelaer

February 5, 2024 08:15

oliviermattelaer approved these changes

View reviewed changes

Member

oliviermattelaer left a comment

Sounds a nice move indeed.

Thanks,

Member Author

valassi commented Feb 5, 2024

Thanks @oliviermattelaer . Merging... one PR less to go.

valassi merged commit 9f8085e into madgraph5:master

57 checks passed

This was referenced Feb 5, 2024

add support for ROCRAND (via HIPRAND) #809

Merged

Add fbridgesequence_nomultichannel #796

Merged

valassi added a commit to valassi/madgraph4gpu that referenced this pull request


          Merge remote-tracking branch 'upstream/master' (including PR madgraph…

3548fbd

…5#368 adding _cu.o and removing gXXX.cu links) into rocrand

Fix conflicts:
	epochX/cudacpp/CODEGEN/PLUGIN/CUDACPP_SA_OUTPUT/madgraph/iolibs/template_files/gpu/cudacpp.mk
	epochX/cudacpp/CODEGEN/PLUGIN/CUDACPP_SA_OUTPUT/model_handling.py

The conflicts were in the interplay of adding Hiprand and adding _cu.o (so for instance gHipRandRandomNumberKernel.o becomes HipRandRandomNumberKernel_cu.o)

valassi added a commit to valassi/madgraph4gpu that referenced this pull request


          [rocrand] in CODEGEN, bug fix in my previous conflict fix after mergi…

61cf075

…ng gXXX.cu removal PR madgraph5#368

valassi added a commit to valassi/madgraph4gpu that referenced this pull request


          Merge remote-tracking branch 'upstream/master' (including PR madgraph…

1d59c1c

…5#368) into mch

valassi added a commit to valassi/madgraph4gpu that referenced this pull request


          [mch] **COMPLETE MCH (after merging madgraph5#368 plus minor changes)…

064a900

… ** manually copy the updated testmisc.cc to all generated processes

for f in $(git ls-tree --name-only HEAD */SubProcesses/testmisc.cc); do \cp gg_tt.mad/SubProcesses/testmisc.cc $f; done

valassi added a commit to valassi/madgraph4gpu that referenced this pull request


          [mch] **COMPLETE MCH (after merging madgraph5#368 plus minor changes)…

bd581e7

… ** improve a comment in CODEGEN and generated code for testmisc.cc (boolean vector times float vector, not integer vector times float vector)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet