A cautionary tale of decorrelating theory uncertainties

Ghosh, Aishik; Nachman, Benjamin

doi:10.1140/epjc/s10052-022-10012-w

A cautionary tale of decorrelating theory uncertainties

Journal Article · Tue Jan 18 00:00:00 UTC 2022 · European Physical Journal. C, Particles and Fields

DOI:https://doi.org/10.1140/epjc/s10052-022-10012-w· OSTI ID:1848067

Ghosh, Aishik ^[1]; ^[1]

Univ. of California, Berkeley, CA (United States); Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)

A variety of techniques have been proposed to train machine learning classifiers that are independent of a given feature. While this can be an essential technique for enabling background estimation, it may also be useful for reducing uncertainties. We carefully examine theory uncertainties, which typically do not have a statistical origin. We will provide explicit examples of two-point (fragmentation modeling) and continuous (higher-order corrections) uncertainties where decorrelating significantly reduces the apparent uncertainty while the true uncertainty is much larger. These results suggest that caution should be taken when using decorrelation for these types of uncertainties as long as we do not have a complete decomposition into statistically meaningful components.

View Accepted Manuscript (DOE)

Research Organization:: Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)

Sponsoring Organization:: USDOE Office of Science (SC)

Grant/Contract Number:: AC02-05CH11231; SC0009920

OSTI ID:: 1848067

Alternate ID(s):: OSTI ID: 23123835

Journal Information:: European Physical Journal. C, Particles and Fields, Journal Name: European Physical Journal. C, Particles and Fields Journal Issue: 1 Vol. 82; ISSN 1434-6044

Publisher:: SpringerCopyright Statement

Country of Publication:: United States

Language:: English

References (70)

Jet trimming Krohn, David; Thaler, Jesse; Wang, Lian-Tao Journal of High Energy Physics, Vol. 2010, Issue 2 https://doi.org/10.1007/JHEP02(2010)084	journal	February 2010
Maximizing boosted top identification by minimizing N-subjettiness Thaler, Jesse; Van Tilburg, Ken Journal of High Energy Physics, Vol. 2012, Issue 2 https://doi.org/10.1007/JHEP02(2012)093	journal	February 2012
DELPHES 3: a modular framework for fast simulation of a generic collider experiment de Favereau, J.; Delaere, C.; Demin, P. Journal of High Energy Physics, Vol. 2014, Issue 2 https://doi.org/10.1007/JHEP02(2014)057	journal	February 2014
Identifying boosted objects with N-subjettiness Thaler, Jesse; Van Tilburg, Ken Journal of High Energy Physics, Vol. 2011, Issue 3 https://doi.org/10.1007/JHEP03(2011)015	journal	March 2011
Measurement of the inclusive cross-sections of single top-quark and top-antiquark t-channel production in pp collisions at s = 13 $$ \sqrt{s}=13 $$ TeV with the ATLAS detector Aaboud, M.; Aad, G.; Abbott, B. Journal of High Energy Physics, Vol. 2017, Issue 4 https://doi.org/10.1007/JHEP04(2017)086	journal	April 2017
Enhancing searches for resonances with machine learning and moment decomposition Kitouni, Ouail; Nachman, Benjamin; Weisser, Constantin Journal of High Energy Physics, Vol. 2021, Issue 4 https://doi.org/10.1007/JHEP04(2021)070	journal	April 2021
Soft drop Larkoski, Andrew J.; Marzani, Simone; Soyez, Gregory Journal of High Energy Physics, Vol. 2014, Issue 5 https://doi.org/10.1007/JHEP05(2014)146	journal	May 2014
Thinking outside the ROCs: Designing Decorrelated Taggers (DDT) for jet substructure Dolen, James; Harris, Philip; Marzani, Simone Journal of High Energy Physics, Vol. 2016, Issue 5 https://doi.org/10.1007/JHEP05(2016)156	journal	May 2016
Convolved substructure: analytically decorrelating jet substructure observables Moult, Ian; Nachman, Benjamin; Neill, Duff Journal of High Energy Physics, Vol. 2018, Issue 5 https://doi.org/10.1007/JHEP05(2018)002	journal	May 2018
Energy correlation functions for jet substructure Larkoski, Andrew J.; Salam, Gavin P.; Thaler, Jesse Journal of High Energy Physics, Vol. 2013, Issue 6 https://doi.org/10.1007/JHEP06(2013)108	journal	June 2013
The automated computation of tree-level and next-to-leading order differential cross sections, and their matching to parton shower simulations Alwall, J.; Frederix, R.; Frixione, S. Journal of High Energy Physics, Vol. 2014, Issue 7 https://doi.org/10.1007/JHEP07(2014)079	journal	July 2014
Jet-images — deep learning edition de Oliveira, Luke; Kagan, Michael; Mackey, Lester Journal of High Energy Physics, Vol. 2016, Issue 7 https://doi.org/10.1007/JHEP07(2016)069	journal	July 2016
Towards an understanding of jet substructure Dasgupta, Mrinal; Fregoso, Alessandro; Marzani, Simone Journal of High Energy Physics, Vol. 2013, Issue 9 https://doi.org/10.1007/JHEP09(2013)029	journal	September 2013
Power counting to better jet observables Larkoski, Andrew J.; Moult, Ian; Neill, Duff Journal of High Energy Physics, Vol. 2014, Issue 12 https://doi.org/10.1007/JHEP12(2014)009	journal	December 2014
Reducing the Dependence of the Neural Network Function to Systematic Uncertainties in the Input Space Wunsch, Stefan; Jörger, Simon; Wolf, Roger Computing and Software for Big Science, Vol. 4, Issue 1 https://doi.org/10.1007/s41781-020-00037-9	journal	February 2020
Optimal Statistical Inference in the Presence of Systematic Uncertainties Using Neural Network Optimization Based on Binned Poisson Likelihoods with Nuisance Parameters Wunsch, Stefan; Jörger, Simon; Wolf, Roger Computing and Software for Big Science, Vol. 5, Issue 1 https://doi.org/10.1007/s41781-020-00049-5	journal	January 2021
MadMiner: Machine Learning-Based Inference for Particle Physics Brehmer, Johann; Kling, Felix; Espejo, Irina Computing and Software for Big Science, Vol. 4, Issue 1 https://doi.org/10.1007/s41781-020-0035-2	journal	January 2020
Parton fragmentation and string dynamics Andersson, B.; Gustafson, G.; Ingelman, G. Physics Reports, Vol. 97, Issue 2-3 https://doi.org/10.1016/0370-1573(83)90080-7	journal	July 1983
A QCD model for jet fragmentation including soft gluon interference Webber, B. R. Nuclear Physics B, Vol. 238, Issue 3 https://doi.org/10.1016/0550-3213(84)90333-X	journal	June 1984
Jet fragmentation of multiparton configurations in a string framework Sjöstrand, Torbjörn Nuclear Physics B, Vol. 248, Issue 2 https://doi.org/10.1016/0550-3213(84)90607-2	journal	December 1984
A brief introduction to PYTHIA 8.1 Sjöstrand, Torbjörn; Mrenna, Stephen; Skands, Peter Computer Physics Communications, Vol. 178, Issue 11 https://doi.org/10.1016/j.cpc.2008.01.036	journal	June 2008
INFERNO: Inference-Aware Neural Optimisation de Castro, Pablo; Dorigo, Tommaso Computer Physics Communications, Vol. 244 https://doi.org/10.1016/j.cpc.2019.06.007	journal	November 2019
The distance correlation t -test of independence in high dimension Székely, Gábor J.; Rizzo, Maria L. Journal of Multivariate Analysis, Vol. 117 https://doi.org/10.1016/j.jmva.2013.02.012	journal	May 2013
QBDT, a new boosting decision tree method with systematical uncertainties into training for High Energy Physics Xia, Li-Gang Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, Vol. 930 https://doi.org/10.1016/j.nima.2019.03.088	journal	June 2019
Parton distributions with LHC data Ball, Richard D.; Bertone, Valerio; Carrazza, Stefano Nuclear Physics B, Vol. 867, Issue 2 https://doi.org/10.1016/j.nuclphysb.2012.10.003	journal	February 2013
Dispelling the N3 myth for the kt jet-finder Cacciari, Matteo; Salam, Gavin P. Physics Letters B, Vol. 641, Issue 1 https://doi.org/10.1016/j.physletb.2006.08.037	journal	September 2006
Jet substructure at the Large Hadron Collider: A review of recent advances in theory and machine learning Larkoski, Andrew J.; Moult, Ian; Nachman, Benjamin Physics Reports https://doi.org/10.1016/j.physrep.2019.11.001	journal	November 2019
Machine learning at the energy and intensity frontiers of particle physics Radovic, Alexander; Williams, Mike; Rousseau, David Nature, Vol. 560, Issue 7716 https://doi.org/10.1038/s41586-018-0361-2	journal	August 2018
Mining gold from implicit models to improve likelihood-free inference Brehmer, Johann; Louppe, Gilles; Pavez, Juan Proceedings of the National Academy of Sciences, Vol. 117, Issue 10 https://doi.org/10.1073/pnas.1915980117	journal	February 2020
PYTHIA 6.4 physics and manual Sjöstrand, Torbjörn; Mrenna, Stephen; Skands, Peter Journal of High Energy Physics, Vol. 2006, Issue 05 https://doi.org/10.1088/1126-6708/2006/05/026	journal	May 2006
The anti- k _t jet clustering algorithm Cacciari, Matteo; Salam, Gavin P.; Soyez, Gregory Journal of High Energy Physics, Vol. 2008, Issue 04 https://doi.org/10.1088/1126-6708/2008/04/063	journal	April 2008
Event generation with SHERPA 1.1 Gleisberg, T.; Höche, S.; Krauss, F. Journal of High Energy Physics, Vol. 2009, Issue 02 https://doi.org/10.1088/1126-6708/2009/02/007	journal	February 2009
Direct optimization of the discovery significance in machine learning for new physics searches in particle colliders Elwood, A.; Krücker, D.; Shchedrolosiev, M. Journal of Physics: Conference Series, Vol. 1525 https://doi.org/10.1088/1742-6596/1525/1/012110	journal	April 2020
DELPHES 3: A modular framework for fast-simulation of generic collider experiments Selvaggi, Michele Journal of Physics: Conference Series, Vol. 523 https://doi.org/10.1088/1742-6596/523/1/012033	journal	June 2014
New features in Delphes 3 Mertens, Alexandre Journal of Physics: Conference Series, Vol. 608 https://doi.org/10.1088/1742-6596/608/1/012045	journal	May 2015
New approaches for boosting to uniformity Rogozhnikov, A.; Bukva, A.; Gligorov, V. Journal of Instrumentation, Vol. 10, Issue 03 https://doi.org/10.1088/1748-0221/10/03/T03002	journal	March 2015
Identification of heavy, energetic, hadronically decaying particles using machine-learning techniques Sirunyan, A. M.; Tumasyan, A.; Adam, W. Journal of Instrumentation, Vol. 15, Issue 06 https://doi.org/10.1088/1748-0221/15/06/P06005	journal	June 2020
uBoost: a boosting method for producing uniform selection efficiencies from multivariate classifiers Stevens, J.; Williams, M. Journal of Instrumentation, Vol. 8, Issue 12 https://doi.org/10.1088/1748-0221/8/12/P12013	journal	December 2013
A deep neural network to search for new long-lived particles decaying to jets No authors listed Machine Learning: Science and Technology, Vol. 1, Issue 3, 035012 https://doi.org/10.1088/2632-2153/ab9023	journal	August 2020
Adversarial domain adaptation to reduce sample bias of a high energy physics event classifier ^* Clavijo, J. M.; Glaysher, P.; Jitsev, J. Machine Learning: Science and Technology, Vol. 3, Issue 1 https://doi.org/10.1088/2632-2153/ac3dde	journal	December 2021
Nuisance hardened data compression for fast likelihood-free inference Alsing, Justin; Wandelt, Benjamin Monthly Notices of the Royal Astronomical Society, Vol. 488, Issue 4 https://doi.org/10.1093/mnras/stz1900	journal	July 2019
Review of Particle Physics Zyla, P. A.; Barnett, R. M.; Beringer, J. Progress of Theoretical and Experimental Physics, Vol. 2020, Issue 8 https://doi.org/10.1093/ptep/ptaa104	journal	August 2020
Automating the ABCD method with machine learning Kasieczka, Gregor; Nachman, Benjamin; Schwartz, Matthew D. Physical Review D, Vol. 103, Issue 3 https://doi.org/10.1103/PhysRevD.103.035021	journal	February 2021
Uncertainty-aware machine learning for high energy physics Ghosh, Aishik; Nachman, Benjamin; Whiteson, Daniel Physical Review D, Vol. 104, Issue 5 https://doi.org/10.1103/PhysRevD.104.056026	journal	September 2021
Recombination algorithms and jet substructure: Pruning as a tool for heavy particle searches Ellis, Stephen D.; Vermilion, Christopher K.; Walsh, Jonathan R. Physical Review D, Vol. 81, Issue 9 https://doi.org/10.1103/PhysRevD.81.094023	journal	May 2010
Decorrelated jet substructure tagging using adversarial neural networks Shimmin, Chase; Sadowski, Peter; Baldi, Pierre Physical Review D, Vol. 96, Issue 7 https://doi.org/10.1103/PhysRevD.96.074034	journal	October 2017
What is the machine learning? Chang, Spencer; Cohen, Timothy; Ostdiek, Bryan Physical Review D, Vol. 97, Issue 5 https://doi.org/10.1103/PhysRevD.97.056009	journal	March 2018
A guide to constraining effective field theories with machine learning Brehmer, Johann; Cranmer, Kyle; Louppe, Gilles Physical Review D, Vol. 98, Issue 5 https://doi.org/10.1103/PhysRevD.98.052004	journal	September 2018
Jet Substructure as a New Higgs-Search Channel at the Large Hadron Collider Butterworth, Jonathan M.; Davison, Adam R.; Rubin, Mathieu Physical Review Letters, Vol. 100, Issue 24 https://doi.org/10.1103/PhysRevLett.100.242001	journal	June 2008
Constraining Effective Field Theories with Machine Learning Brehmer, Johann; Cranmer, Kyle; Louppe, Gilles Physical Review Letters, Vol. 121, Issue 11 https://doi.org/10.1103/PhysRevLett.121.111801	journal	September 2018
Robust Jet Classifiers through Distance Correlation Kasieczka, Gregor; Shih, David Physical Review Letters, Vol. 125, Issue 12 https://doi.org/10.1103/PhysRevLett.125.122001	journal	September 2020
Machine learning and the physical sciences Carleo, Giuseppe; Cirac, Ignacio; Cranmer, Kyle Reviews of Modern Physics, Vol. 91, Issue 4 https://doi.org/10.1103/RevModPhys.91.045002	journal	December 2019
Automatic physical inference with information maximizing neural networks Charnock, Tom; Lavaux, Guilhem; Wandelt, Benjamin D. Physical Review D, Vol. 97, Issue 8 https://doi.org/10.1103/physrevd.97.083004	journal	April 2018
Herwig++ physics and manual Bähr, Manuel; Gieseke, Stefan; Gigg, Martyn A. The European Physical Journal C, Vol. 58, Issue 4 https://doi.org/10.1140/epjc/s10052-008-0798-9	journal	November 2008
FastJet user manual: (for version 3.0.2) Cacciari, Matteo; Salam, Gavin P.; Soyez, Gregory The European Physical Journal C, Vol. 72, Issue 3 https://doi.org/10.1140/epjc/s10052-012-1896-2	journal	March 2012
Herwig 7.0/Herwig++ 3.0 release note Bellm, Johannes; Gieseke, Stefan; Grellscheid, David The European Physical Journal C, Vol. 76, Issue 4 https://doi.org/10.1140/epjc/s10052-016-4018-8	journal	April 2016
Parameterized neural networks for high-energy physics Baldi, Pierre; Cranmer, Kyle; Faucett, Taylor The European Physical Journal C, Vol. 76, Issue 5 https://doi.org/10.1140/epjc/s10052-016-4099-4	journal	April 2016
Machine learning uncertainties with adversarial neural networks Englert, Christoph; Galler, Peter; Harris, Philip The European Physical Journal C, Vol. 79, Issue 1 https://doi.org/10.1140/epjc/s10052-018-6511-8	journal	January 2019
Performance of top-quark and $$\varvec{W}$$ W -boson tagging with ATLAS in Run 2 of the LHC Aaboud, M.; Aad, G.; Abbott, B. The European Physical Journal C, Vol. 79, Issue 5 https://doi.org/10.1140/epjc/s10052-019-6847-8	journal	April 2019
Measurement of differential cross sections and charge ratios for t-channel single top quark production in proton–proton collisions at $$\sqrt{s}=13$$ $$\,\text {Te}\text {V}$$ Sirunyan, A. M.; Tumasyan, A.; Adam, W. The European Physical Journal C, Vol. 80, Issue 5 https://doi.org/10.1140/epjc/s10052-020-7858-1	journal	May 2020
A modified cluster-hadronisation model Winter, J. -C.; Krauss, F.; Soff, G. The European Physical Journal C, Vol. 36, Issue 3 https://doi.org/10.1140/epjc/s2004-01960-8	journal	August 2004
Machine and deep learning applications in particle physics Bourilkov, Dimitri International Journal of Modern Physics A, Vol. 34, Issue 35 https://doi.org/10.1142/S0217751X19300199	journal	December 2019
Deep Learning and Its Application to LHC Physics Guest, Dan; Cranmer, Kyle; Whiteson, Daniel Annual Review of Nuclear and Particle Science, Vol. 68, Issue 1 https://doi.org/10.1146/annurev-nucl-101917-021019	journal	October 2018
Measuring and testing dependence by correlation of distances Székely, Gábor J.; Rizzo, Maria L.; Bakirov, Nail K. The Annals of Statistics, Vol. 35, Issue 6 https://doi.org/10.1214/009053607000000505	journal	December 2007
Brownian distance covariance Székely, Gábor J.; Rizzo, Maria L. The Annals of Applied Statistics, Vol. 3, Issue 4 https://doi.org/10.1214/09-AOAS312	journal	December 2009
Partial distance correlation with methods for dissimilarities Székely, Gábor J.; Rizzo, Maria L. The Annals of Statistics, Vol. 42, Issue 6 https://doi.org/10.1214/14-AOS1255	journal	December 2014
The Machine Learning landscape of top taggers Kasieczka, Gregor; Plehn, Tilman; Butter, Anja SciPost Physics, Vol. 7, Issue 1 https://doi.org/10.21468/SciPostPhys.7.1.014	journal	January 2019
Event generation with Sherpa 2.2 Bothmann, Enrico; Singh Chahal, Gurpreet; Höche, Stefan SciPost Physics, Vol. 7, Issue 3 https://doi.org/10.21468/SciPostPhys.7.3.034	journal	January 2019
Mass agnostic jet taggers Bradshaw, Layne; Mishra, Rashmish K.; Mitridate, Andrea SciPost Physics, Vol. 8, Issue 1 https://doi.org/10.21468/SciPostPhys.8.1.011	journal	January 2020
A guide for deploying Deep Learning in LHC searches: How to achieve optimality and account for uncertainty Nachman, Benjamin SciPost Physics, Vol. 8, Issue 6 https://doi.org/10.21468/SciPostPhys.8.6.090	journal	January 2020

Similar Records

Advanced neutron source thermal-hydraulic uncertainty analysis

Journal Article · Sun Dec 31 04:00:00 UTC 1995 · Transactions of the American Nuclear Society · OSTI ID:186609

Inertial Fusion Energy program within OFES

Technical Report · Sun Dec 01 04:00:00 UTC 2019 · OSTI ID:1592880

The effects of phase decorrelation on the dynamics of the bump-on-tail instability

Journal Article · Sat Aug 15 04:00:00 UTC 2015 · Physics of Plasmas · OSTI ID:22490028

Related Subjects

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS

A cautionary tale of decorrelating theory uncertainties

Citation Formats

References (70)

Similar Records

Related Subjects