Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[RFC] [NANO] Implement a check on the number of scale variation weights in GenWeightsTableProducer #46573

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

[RFC] [NANO] Implement a check on the number of scale variation weights in GenWeightsTableProducer #46573

wants to merge 1 commit into from

Conversation

hqucms
Copy link
Contributor

@hqucms hqucms commented Oct 31, 2024 •
edited
Loading

PR description:

This PR implements a possible mitigation of the LHEScaleSumw issue, as discussed in the PPD general meeting today. Feedback is welcome!

Basically a configurable option allowedNumScaleWeights is introduced to GenWeightsTableProducer, which can take a list of numbers, and the number of scale variation weights identified by parsing the initrwgt header of LHERunInfoProduct must match one of the values provided in allowedNumScaleWeights, otherwise a LogicError is thrown. The default value of allowedNumScaleWeights is set to 9 as it is the expected number of scale variations in the standard setup. An empty list for allowedNumScaleWeights means any number of scale variation weights is allowed and no check is performed.

This PR also fixes the parsing of certain MadGraph headers where it fails to pick up the central weight, as in some cases the central weight is outside any weight group. This has led to some NanoAOD samples containing only 8 (instead of 9) scale weights.

Related issue: #43784

PR validation:

Passes local tests with:

  • a normal MadGraph file with the expected, 9 scale weights:
    • /store/mc/Run3Summer22EEMiniAODv4/DYGto2LG-1Jets_MLL-50_PTG-10to50_TuneCP5_13p6TeV_amcatnloFXFX-pythia8/MINIAODSIM/130X_mcRun3_2022_realistic_postEE_v6-v2/30000/0fd00136-3752-4096-8e82-4b6fcd656da4.root
  • a buggy MadGraph file with no PDF/scale weights and missing theinitrwgt header :
    • /store/mc/Run3Summer22EEMiniAODv4/DYGto2LG-1Jets_MLL-50_PTG-10to50_TuneCP5_13p6TeV_amcatnloFXFX-pythia8/MINIAODSIM/130X_mcRun3_2022_realistic_postEE_v6-v2/40000/c22ada93-447e-4db9-bee9-fcb3ab3e7fa9.root
  • another MG file where only 8 scale weights were identified before the change:
    • /store/mc/RunIISummer20UL16MiniAODAPVv2/TTZ-ZToBB-TTTo2L_TuneCP5_13TeV-amcatnlo-pythia8/MINIAODSIM/106X_mcRun2_asymptotic_preVFP_v11-v2/2560000/02890624-0027-E345-880C-C8372B42C942.root

@cmsbuild cmsbuild added this to the CMSSW_14_2_X milestone Oct 31, 2024
@cmsbuild
Copy link
Contributor

cmsbuild commented Oct 31, 2024 •
edited
Loading

cms-bot internal usage

@hqucms
Copy link
Contributor Author

hqucms commented Oct 31, 2024

FYI -- @cms-sw/generators-l2 @cms-sw/pdmv-l2 @cms-sw/ppd-l2

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46573/42462

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @hqucms for master.

It involves the following packages:

  • PhysicsTools/NanoAOD (xpog)

@cmsbuild, @ftorrresd, @hqucms can you please review it and eventually sign? Thanks.
@AnnikaStein, @gpetruc this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

@hqucms
Copy link
Contributor Author

hqucms commented Oct 31, 2024

enable nano

@hqucms
Copy link
Contributor Author

hqucms commented Oct 31, 2024

please test

@cmsbuild
Copy link
Contributor

-1

Failed Tests: RelVals-INPUT
Size: This PR adds an extra 28KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8feffa/42518/summary.html
COMMIT: eada7b2
CMSSW: CMSSW_14_2_X_2024-10-30-2300/el8_amd64_gcc12
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/46573/42518/install.sh to create a dev area with all the needed externals and cmssw changes.

  • DAS Queries: The DAS query tests failed, see the summary page for details.

RelVals-INPUT

  • 2024.0000012024.000001_RunJetMET02024D_10k/step1_dasquery.log
  • 2024.0010012024.001001_RunZeroBias2024D_10k/step1_dasquery.log
  • 2024.1000012024.100001_RunJetMET02024C_10k/step1_dasquery.log
Expand to see more relval errors ...
  • 2024.101001
  • 2024.000001
  • 2024.001001
  • 2024.100001
  • 2024.101001

Comparison Summary

Summary:

NANO Comparison Summary

Summary:

  • You potentially added 992 lines to the logs
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 21
  • DQMHistoTests: Total histograms compared: 55313
  • DQMHistoTests: Total failures: 0
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 55313
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 20 files compared)
  • Checked 102 log files, 58 edm output root files, 21 DQM output files
  • TriggerResults: no differences found

Nano size comparison Summary:

Sample kb/ev ref kb/ev diff kb/ev ev/s/thd ref ev/s/thd diff rate mem/thd ref mem/thd
2500.001 3.012 3.012 0.000 ( +0.0% ) 3.00 2.98 +0.7% 2.350 2.332
2500.002 3.124 3.124 0.000 ( +0.0% ) 2.66 2.65 +0.6% 2.787 2.360
2500.003 3.065 3.065 0.000 ( +0.0% ) 2.79 2.77 +0.5% 2.762 2.374
2500.011 1.591 1.591 0.000 ( +0.0% ) 4.68 4.61 +1.6% 2.455 2.453
2500.012 2.113 2.113 0.000 ( +0.0% ) 2.81 2.81 +0.2% 2.637 2.264
2500.013 1.936 1.936 0.000 ( +0.0% ) 3.99 3.95 +1.0% 2.550 2.519
2500.021 0.022 0.022 0.000 ( +0.0% ) 0.93 0.91 +1.6% 2.415 2.413
2500.022 0.022 0.022 0.000 ( +0.0% ) 0.89 0.88 +1.1% 2.413 2.406
2500.023 0.022 0.022 0.000 ( +0.0% ) 0.90 0.88 +2.3% 2.264 2.276
2500.024 0.022 0.022 0.000 ( +0.0% ) 0.68 0.68 -0.3% 2.509 2.507
2500.031 0.035 0.035 0.000 ( +0.0% ) 0.83 0.81 +1.9% 2.476 2.479
2500.032 0.036 0.036 0.000 ( +0.0% ) 0.85 0.84 +1.6% 2.446 2.438
2500.033 0.037 0.037 0.000 ( +0.0% ) 0.75 0.75 +0.9% 2.514 2.523
2500.034 0.036 0.036 0.000 ( +0.0% ) 0.77 0.76 +0.8% 2.507 2.504
2500.101 2.766 2.766 0.000 ( +0.0% ) 7.62 7.52 +1.3% 2.585 2.590
2500.111 1.419 1.419 0.000 ( +0.0% ) 17.34 17.10 +1.4% 2.288 2.292
2500.112 1.838 1.838 0.000 ( +0.0% ) 14.21 13.86 +2.5% 2.360 2.375
2500.131 0.747 0.747 0.000 ( +0.0% ) 18.20 17.80 +2.2% 1.493 1.479
2500.201 2.595 2.595 0.000 ( +0.0% ) 6.44 6.07 +6.1% 2.153 2.152
2500.211 1.757 1.757 0.000 ( +0.0% ) 15.32 15.07 +1.6% 2.353 2.361
2500.212 2.152 2.152 0.000 ( +0.0% ) 12.41 12.43 -0.2% 2.442 2.444
2500.221 2.016 2.016 0.000 ( +0.0% ) 6.50 6.56 -0.9% 2.072 2.073
2500.222 3.392 3.392 0.000 ( +0.0% ) 6.32 6.18 +2.4% 2.163 2.163
2500.223 9.357 9.357 0.000 ( +0.0% ) 1.82 1.78 +2.2% 2.226 2.177
2500.224 6.040 6.040 0.000 ( +0.0% ) 0.60 0.59 +0.7% 2.216 2.172
2500.225 6.072 6.072 0.000 ( +0.0% ) 0.56 0.56 +0.4% 2.251 2.238
2500.226 3.086 3.086 0.000 ( +0.0% ) 6.58 6.35 +3.5% 2.152 2.160
2500.227 1.437 1.437 0.000 ( +0.0% ) 11.91 11.42 +4.3% 1.425 1.429
2500.231 1.429 1.429 0.000 ( +0.0% ) 12.49 12.32 +1.4% 2.255 2.259
2500.232 2.412 2.412 0.000 ( +0.0% ) 11.75 11.14 +5.5% 2.351 2.354
2500.233 4.904 4.904 0.000 ( +0.0% ) 2.74 2.74 +0.0% 2.420 2.361
2500.234 3.670 3.670 0.000 ( +0.0% ) 0.76 0.75 +1.1% 2.156 2.147
2500.235 3.689 3.689 0.000 ( +0.0% ) 0.72 0.72 +0.9% 2.206 2.198
2500.236 2.202 2.202 0.000 ( +0.0% ) 12.56 12.06 +4.1% 2.358 2.350
2500.237 1.016 1.016 0.000 ( +0.0% ) 17.49 17.02 +2.8% 1.453 1.459
2500.241 9.404 9.404 0.000 ( +0.0% ) 3.50 3.65 -4.3% 1.918 1.919
2500.242 10.331 10.331 0.000 ( +0.0% ) 0.83 0.88 -6.2% 1.709 1.705
2500.243 2.712 2.712 0.000 ( +0.0% ) 8.60 8.48 +1.5% 1.074 1.074
2500.244 485.976 485.976 0.000 ( +0.0% ) 0.58 0.55 +4.3% 1.692 1.675
2500.245 823.202 823.202 0.000 ( +0.0% ) 0.74 0.73 +1.6% 1.659 1.666
2500.901 1.777 1.777 0.000 ( +0.0% ) 21.67 21.08 +2.8% 1.423 1.415
2500.902 1.626 1.626 0.000 ( +0.0% ) 22.19 21.51 +3.2% 1.324 1.320
2500.911 13.995 13.995 0.000 ( +0.0% ) 3.87 2.50 +54.8% 1.092 1.089
2500.912 0.240 0.199 0.041 ( +20.3% ) 1.08 1.33 -19.4% 0.977 0.976
2500.913 0.110 0.110 0.000 ( +0.0% ) 1.14 1.06 +6.9% 0.975 0.978

@vlimant
Copy link
Contributor

vlimant commented Nov 1, 2024

the configuration is sample dependent, how often will it have to be different than 9 ?

@vlimant
Copy link
Contributor

vlimant commented Nov 1, 2024

assign ppd

@cmsbuild
Copy link
Contributor

cmsbuild commented Nov 1, 2024

New categories assigned: ppd

@malbouis,@vlimant you have been requested to review this Pull request/Issue and eventually sign? Thanks

@hqucms
Copy link
Contributor Author

hqucms commented Nov 1, 2024

the configuration is sample dependent, how often will it have to be different than 9 ?

Maybe @cms-sw/generators-l2 can comment on that.

@DickyChant
Copy link

On the PdmV front:

https://sqian.web.cern.ch/sqian/requests_list.html

here is a list of request that is in “submitted” status and used MadGraph_aMCatNLO as generator and is a wmLHE request that will save those weights.

if @cms-sw/generators-l2 are ok I will rewind them, revalidate and resubmit

@vlimant
Copy link
Contributor

vlimant commented Nov 4, 2024

go ahead with resetting @DickyChant

@vlimant
Copy link
Contributor

vlimant commented Nov 4, 2024

with regards to sample dependent configuration, we will have to look at this extremely carefully, as to avoid unmanageable overhead.

@vlimant
Copy link
Contributor

vlimant commented Nov 4, 2024

instead of throwing an exception, we could fill the table with well identifiable values (0?) that can be skipped at analysis.
this could also be handled at the merging step https://github.com/cms-sw/cmssw/blob/bf7e479f0d484aa781778709dd0e6abef4e89a34/DataFormats/NanoAOD/interface/MergeableCounterTable.h#L25C29-L25C51 by doing the += on a fresh default-valued object instead of the one failing the compatibility test

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
code-checks-approved orp-pending pending-signatures ppd-pending tests-rejected xpog-pending
Projects
None yet
Milestone
CMSSW_14_2_X
Development

Successfully merging this pull request may close these issues.
4 participants
@hqucms @cmsbuild @vlimant @DickyChant