[NGT] Backport - Implement new strategy for Tau Validation at HLT and RECO in Phase-2#51152
Open
elenavernazza wants to merge 14 commits into
Open
[NGT] Backport - Implement new strategy for Tau Validation at HLT and RECO in Phase-2#51152elenavernazza wants to merge 14 commits into
elenavernazza wants to merge 14 commits into
Conversation
Co-authored-by: Spyros Merianos <spyros.merianos@cern.ch>
elenavernazza
changed the title
Contributor
|
cms-bot internal usage |
Contributor
|
+code-checks Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51152/49677 |
Contributor
|
A new Pull Request was created by @elenavernazza for CMSSW_17_0_X. It involves the following packages:
@civanch, @cmsbuild, @ctarricone, @davidlange6, @fabiocos, @ftenchini, @gabrielmscampos, @kpedro88, @mandrenguyen, @mdhildreth, @rseidita can you please review it and eventually sign? Thanks. cms-bot commands are listed here |
Contributor
|
backport of #51092 |
Contributor
|
@cmsbuild, please test |
cmsbuild
added
operations-approved
tests-approved
backport-ok
and removed
operations-pending
tests-started
backport
labels
cmsbuild
added
operations-pending
tests-pending
code-checks-pending
and removed
operations-approved
tests-approved
code-checks-approved
labels
Contributor
Author
|
This additional commit contains the bug fix introduced in #51198 |
Contributor
|
test parameters:
|
Contributor
|
backport of #51198 |
Contributor
|
@cmsbuild, please test |
Contributor
|
+code-checks Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-51152/49723 |
Contributor
|
Pull request #51152 was updated. @civanch, @ctarricone, @davidlange6, @fabiocos, @ftenchini, @gabrielmscampos, @kpedro88, @mandrenguyen, @mdhildreth, @rseidita can you please check and sign again. |
Contributor
|
+1 Size: This PR adds an extra 28KB to repository Comparison SummarySummary:
Max Memory Comparisons exceeding threshold@cms-sw/core-l2 , I found 39 workflow step(s) with memory usage exceeding the error threshold: Expand to see workflows ...
|
cmsbuild
added
operations-approved
tests-approved
and removed
operations-pending
tests-started
labels
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR description:
This PR introduces a new high-level validation strategy for hadronic tau reconstruction at both HLT and RECO in Phase-2.
The validation framework evaluates the reconstruction performance by matching GEN taus to reconstructed (HLT/RECO) taus and computing the following metrics:
All metrics can be studied as a function of the tau kinematics ($p_T$ , $\eta$ , $\phi$ , mass).$\Delta R=0.3$ ). This complementary two-level validation approach was discussed in the Tau Algo Meeting (link).
As opposed to the low-level validation, which relies on detailed hit-by-hit associations, this high-level validation adopts a simplified matching strategy based on a configurable spatial requirement (default:
Technical changes
This PR includes:
HLTTauValandHLTTauPostValinto the Phase-2 HLT validation and post-validation sequencesThe core of the implementation is the new
TauValidator.ccEDAnalyzer, which produces centralised DQM histograms for all defined metrics that can be used for more automatised RelVal studies. It supports both HLT and RECO collections, handling both RECO and PAT formats (required for Tau ID algorithms such as DeepTau, available only from the PAT step for the offline reconstruction).Additionally, plotting utilities are provided to compare:
Some exemplary plots in the following:
All plots for 90K events from a TenTau sample in 150 PU can be found at this link.
Instruction to reproduce validation plots
Note: Before this PR, the HLT DeepTau ID was not persisted in the
FEVTDEBUGHLTevent content, so running only the RECO step on an old RelVal sample could result in empty distributions of the ID values at HLT.Choose
STEP=HLT/RECO(or omit for both). By default, it will consider the validation obtained by requiring WP vs Jet >0 (CutWP_VSjet0); change folder name (SUB_DIR,OUTDIR_SUFFIX,LABEL_TEXT) if needed.Choose
MODE=ID/WP/DeltaRandSTEP=HLT/RECO.PR validation:
This PR has been extensively tested with the commands listed above, based on a TenTau RelVal sample. The configurations based on NGT scouting and HLT timing menus have also been successfully validated.
It has also been successfully validated on a general Phase-2 workflow
34434.0:The DQM plots are correctly produced but there are only few GEN taus in the tt-bar sample so the performance is not fully representative.
If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:
This PR is a backport of #51092, in order for the validation tool to be available for Run-3 workflows.