Automating the ABCD method with machine learning

Kasieczka, Gregor; Nachman, Benjamin; Schwartz, Matthew D.; Shih, David

doi:10.1103/PhysRevD.103.035021

Title: Automating the ABCD method with machine learning

Abstract

The ABCD method is one of the most widely used data-driven background estimation techniques in high energy physics. Cuts on two statistically independent classifiers separate signal and background into four regions, so that background in the signal region can be estimated simply using the other three control regions. Typically, the independent classifiers are chosen "by hand" to be intuitive and physically motivated variables. Here, we explore the possibility of automating the design of one or both of these classifiers using machine learning. We show how to use state-of-the-art decorrelation methods to construct powerful yet independent discriminators. Along the way, we uncover a previously unappreciated aspect of the ABCD method: its accuracy hinges on having low signal contamination in control regions not just overall, but relative to the signal fraction in the signal region. Whye demonstrate the method with three examples: a simple model consisting of three-dimensional Gaussians; boosted hadronic top jet tagging; and a recasted search for paired dijet resonances. In all cases, automating the ABCD method with machine learning significantly improves performance in terms of ABCD closure, background rejection, and signal contamination.

Authors:

Kasieczka, Gregor;

; Schwartz, Matthew D.; Shih, David

Publication Date:: Mon Feb 22 00:00:00 EST 2021

Research Org.:: Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)

Sponsoring Org.:: USDOE Office of Science (SC); German Research Foundation (DFG)

OSTI Identifier:: 1766649

Alternate Identifier(s):: OSTI ID: 1811514

Grant/Contract Number:: AC02-05CH11231; SC0013607; DOE-SC0010008; SC0010008; 390833306

Resource Type:: Published Article

Journal Name:: Physical Review D

Additional Journal Information:: Journal Name: Physical Review D Journal Volume: 103 Journal Issue: 3; Journal ID: ISSN 2470-0010

Publisher:: American Physical Society

Country of Publication:: United States

Language:: English

Subject:: 72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS; hypothetical particle physics models; artificial neural networks; machine learning

Citation Formats


                    Kasieczka, Gregor, Nachman, Benjamin, Schwartz, Matthew D., and Shih, David. Automating the ABCD method with machine learning.  United States: N. p., 2021. 
Web.  doi:10.1103/PhysRevD.103.035021.

Copy to clipboard


                    Kasieczka, Gregor, Nachman, Benjamin, Schwartz, Matthew D., & Shih, David. Automating the ABCD method with machine learning.  United States.  https://doi.org/10.1103/PhysRevD.103.035021

Copy to clipboard


                    Kasieczka, Gregor, Nachman, Benjamin, Schwartz, Matthew D., and Shih, David. Mon .  
"Automating the ABCD method with machine learning".  United States.  https://doi.org/10.1103/PhysRevD.103.035021.

Copy to clipboard


                    
@article{osti_1766649,

  title        = {Automating the ABCD method with machine learning},

  author       = {Kasieczka, Gregor and Nachman, Benjamin and Schwartz, Matthew D. and Shih, David},

  abstractNote = {The ABCD method is one of the most widely used data-driven background estimation techniques in high energy physics. Cuts on two statistically independent classifiers separate signal and background into four regions, so that background in the signal region can be estimated simply using the other three control regions. Typically, the independent classifiers are chosen "by hand" to be intuitive and physically motivated variables. Here, we explore the possibility of automating the design of one or both of these classifiers using machine learning. We show how to use state-of-the-art decorrelation methods to construct powerful yet independent discriminators. Along the way, we uncover a previously unappreciated aspect of the ABCD method: its accuracy hinges on having low signal contamination in control regions not just overall, but relative to the signal fraction in the signal region. Whye demonstrate the method with three examples: a simple model consisting of three-dimensional Gaussians; boosted hadronic top jet tagging; and a recasted search for paired dijet resonances. In all cases, automating the ABCD method with machine learning significantly improves performance in terms of ABCD closure, background rejection, and signal contamination.},

  doi          = {10.1103/PhysRevD.103.035021},

  journal      = {Physical Review D},

  number       = 3,

  volume       = 103,

  place        = {United States},

  year         = {Mon Feb 22 00:00:00 EST 2021},

  month        = {Mon Feb 22 00:00:00 EST 2021}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Publisher's Version of Record
https://doi.org/10.1103/PhysRevD.103.035021

Other availability

Search WorldCat to find libraries that may hold this journal

Save / Share:

Export Metadata

Save to My Library

Works referenced in this record:

Evolution of average multiplicities of quark and gluon jets
journal, February 2000

Capella, A.; Dremin, I. M.; Gary, J. W.
Physical Review D, Vol. 61, Issue 7
DOI: 10.1103/PhysRevD.61.074009

Optimal Statistical Inference in the Presence of Systematic Uncertainties Using Neural Network Optimization Based on Binned Poisson Likelihoods with Nuisance Parameters
journal, January 2021

Wunsch, Stefan; Jörger, Simon; Wolf, Roger
Computing and Software for Big Science, Vol. 5, Issue 1
DOI: 10.1007/s41781-020-00049-5

Learning representations of irregular particle-detector geometry with distance-weighted graph networks
journal, July 2019

Qasim, Shah Rukh; Kieseler, Jan; Iiyama, Yutaro
The European Physical Journal C, Vol. 79, Issue 7
DOI: 10.1140/epjc/s10052-019-7113-9

Search for pair-produced resonances decaying to jet pairs in proton–proton collisions at $\sqrt{s} = 8 TeV$
journal, July 2015

Khachatryan, V.; Sirunyan, A. M.; Tumasyan, A.
Physics Letters B, Vol. 747
DOI: 10.1016/j.physletb.2015.04.045

Deep-learned Top Tagging with a Lorentz Layer
journal, January 2018

Butter, Anja; Kasieczka, Gregor; Plehn, Tilman
SciPost Physics, Vol. 5, Issue 3
DOI: 10.21468/SciPostPhys.5.3.028

Electromagnetic showers beyond shower shapes
journal, January 2020

de Oliveira, Luke; Nachman, Benjamin; Paganini, Michela
Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, Vol. 951
DOI: 10.1016/j.nima.2019.162879

Energy flow networks: deep sets for particle jets
journal, January 2019

Komiske, Patrick T.; Metodiev, Eric M.; Thaler, Jesse
Journal of High Energy Physics, Vol. 2019, Issue 1
DOI: 10.1007/JHEP01(2019)121

How much information is in a jet?
journal, June 2017

Datta, Kaustuv; Larkoski, Andrew
Journal of High Energy Physics, Vol. 2017, Issue 6
DOI: 10.1007/JHEP06(2017)073

End-to-End Physics Event Classification with CMS Open Data: Applying Image-Based Deep Learning to Detector Data for the Direct Classification of Collision Events at the LHC
journal, March 2020

Andrews, M.; Paulini, M.; Gleyzer, S.
Computing and Software for Big Science, Vol. 4, Issue 1
DOI: 10.1007/s41781-020-00038-8

Learning new physics from a machine
journal, January 2019

D’Agnolo, Raffaele Tito; Wulzer, Andrea
Physical Review D, Vol. 99, Issue 1
DOI: 10.1103/PhysRevD.99.015014

Deep learning in color: towards automated quark/gluon jet discrimination
journal, January 2017

Komiske, Patrick T.; Metodiev, Eric M.; Schwartz, Matthew D.
Journal of High Energy Physics, Vol. 2017, Issue 1
DOI: 10.1007/JHEP01(2017)110

Boosted $W$ and $Z$ tagging with jet charge and deep learning
journal, March 2020

Chen, Yu-Chen Janice; Chiang, Cheng-Wei; Cottin, Giovanna
Physical Review D, Vol. 101, Issue 5
DOI: 10.1103/PhysRevD.101.053001

Interaction networks for the identification of boosted $H \to b \bar{b}$ decays
journal, July 2020

Moreno, Eric A.; Nguyen, Thong Q.; Vlimant, Jean-Roch
Physical Review D, Vol. 102, Issue 1
DOI: 10.1103/PhysRevD.102.012010

FastJet user manual: (for version 3.0.2)
journal, March 2012

Cacciari, Matteo; Salam, Gavin P.; Soyez, Gregory
The European Physical Journal C, Vol. 72, Issue 3
DOI: 10.1140/epjc/s10052-012-1896-2

Topology Classification with Deep Learning to Improve Real-Time Event Selection at the LHC
journal, August 2019

Nguyen, T. Q.; Weitekamp, D.; Anderson, D.
Computing and Software for Big Science, Vol. 3, Issue 1
DOI: 10.1007/s41781-019-0028-1

Energy flow polynomials: a complete linear basis for jet substructure
journal, April 2018

Komiske, Patrick T.; Metodiev, Eric M.; Thaler, Jesse
Journal of High Energy Physics, Vol. 2018, Issue 4
DOI: 10.1007/JHEP04(2018)013

Identification of heavy, energetic, hadronically decaying particles using machine-learning techniques
journal, June 2020

Sirunyan, A. M.; Tumasyan, A.; Adam, W.
Journal of Instrumentation, Vol. 15, Issue 06
DOI: 10.1088/1748-0221/15/06/P06005

Uncovering latent jet substructure
journal, September 2019

Dillon, Barry M.; Faroughy, Darius A.; Kamenik, Jernej F.
Physical Review D, Vol. 100, Issue 5
DOI: 10.1103/PhysRevD.100.056002

Calorimetry with deep learning: particle simulation and reconstruction for collider physics
journal, July 2020

Belayneh, Dawit; Carminati, Federico; Farbin, Amir
The European Physical Journal C, Vol. 80, Issue 7
DOI: 10.1140/epjc/s10052-020-8251-9

Anomaly detection with density estimation
journal, April 2020

Nachman, Benjamin; Shih, David
Physical Review D, Vol. 101, Issue 7
DOI: 10.1103/PhysRevD.101.075042

Identifying boosted objects with N-subjettiness
journal, March 2011

Thaler, Jesse; Van Tilburg, Ken
Journal of High Energy Physics, Vol. 2011, Issue 3
DOI: 10.1007/JHEP03(2011)015

Jet substructure at the Large Hadron Collider: A review of recent advances in theory and machine learning
journal, November 2019

Larkoski, Andrew J.; Moult, Ian; Nachman, Benjamin
Physics Reports
DOI: 10.1016/j.physrep.2019.11.001

Fast convolutional neural networks for identifying long-lived particles in a high-granularity calorimeter
journal, December 2020

Alimena, J.; Iiyama, Y.; Kieseler, J.
Journal of Instrumentation, Vol. 15, Issue 12
DOI: 10.1088/1748-0221/15/12/P12006

Mass agnostic jet taggers
journal, January 2020

Bradshaw, Layne; Mishra, Rashmish K.; Mitridate, Andrea
SciPost Physics, Vol. 8, Issue 1
DOI: 10.21468/SciPostPhys.8.1.011

Extending the search for new resonances with machine learning
journal, January 2019

Collins, Jack H.; Howe, Kiel; Nachman, Benjamin
Physical Review D, Vol. 99, Issue 1
DOI: 10.1103/PhysRevD.99.014038

Dispelling the <mml:math altimg="si1.gif" overflow="scroll" xmlns:xocs="http://www.elsevier.com/xml/xocs/dtd" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.elsevier.com/xml/ja/dtd" xmlns:ja="http://www.elsevier.com/xml/ja/dtd" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:tb="http://www.elsevier.com/xml/common/table/dtd" xmlns:sb="http://www.elsevier.com/xml/common/struct-bib/dtd" xmlns:ce="http://www.elsevier.com/xml/common/dtd" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:cals="http://www.elsevier.com/xml/common/cals/dtd"><mml:msup><mml:mi>N</mml:mi><mml:mn>3</mml:mn></mml:msup></mml:math> myth for the <mml:math altimg="si2.gif" overflow="scroll" xmlns:xocs="http://www.elsevier.com/xml/xocs/dtd" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.elsevier.com/xml/ja/dtd" xmlns:ja="http://www.elsevier.com/xml/ja/dtd" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:tb="http://www.elsevier.com/xml/common/table/dtd" xmlns:sb="http://www.elsevier.com/xml/common/struct-bib/dtd" xmlns:ce="http://www.elsevier.com/xml/common/dtd" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:cals="http://www.elsevier.com/xml/common/cals/dtd"><mml:msub><mml:mi>k</mml:mi><mml:mi>t</mml:mi></mml:msub></mml:math> jet-finder
journal, September 2006

Cacciari, Matteo; Salam, Gavin P.
Physics Letters B, Vol. 641, Issue 1
DOI: 10.1016/j.physletb.2006.08.037

Learning to classify from impure samples with high-dimensional data
journal, July 2018

Komiske, Patrick T.; Metodiev, Eric M.; Nachman, Benjamin
Physical Review D, Vol. 98, Issue 1
DOI: 10.1103/PhysRevD.98.011502

Novelty detection meets collider physics
journal, April 2020

Hajer, Jan; Li, Ying-Ying; Liu, Tao
Physical Review D, Vol. 101, Issue 7
DOI: 10.1103/PhysRevD.101.076015

Search for Higgs Boson Decays into a $Z$ Boson and a Light Hadronically Decaying Resonance Using 13 TeV $p p$ Collision Data from the ATLAS Detector
journal, November 2020

Aad, G.; Abbott, B.; Abbott, D. C.
Physical Review Letters, Vol. 125, Issue 22
DOI: 10.1103/PhysRevLett.125.221802

Boosting H → b b ¯ $$ H\to b\overline{b} $$ with machine learning
journal, October 2018

Lin, Joshua; Freytsis, Marat; Moult, Ian
Journal of High Energy Physics, Vol. 2018, Issue 10
DOI: 10.1007/JHEP10(2018)101

Parameterized neural networks for high-energy physics
journal, April 2016

Baldi, Pierre; Cranmer, Kyle; Faucett, Taylor
The European Physical Journal C, Vol. 76, Issue 5
DOI: 10.1140/epjc/s10052-016-4099-4

Maximizing boosted top identification by minimizing N-subjettiness
journal, February 2012

Thaler, Jesse; Van Tilburg, Ken
Journal of High Energy Physics, Vol. 2012, Issue 2
DOI: 10.1007/JHEP02(2012)093

Top Tagging: A Method for Identifying Boosted Hadronically Decaying Top Quarks
journal, October 2008

Kaplan, David E.; Rehermann, Keith; Schwartz, Matthew D.
Physical Review Letters, Vol. 101, Issue 14
DOI: 10.1103/PhysRevLett.101.142001

Reducing the Dependence of the Neural Network Function to Systematic Uncertainties in the Input Space
journal, February 2020

Wunsch, Stefan; Jörger, Simon; Wolf, Roger
Computing and Software for Big Science, Vol. 4, Issue 1
DOI: 10.1007/s41781-020-00037-9

Jet tagging via particle clouds
journal, March 2020

Qu, Huilin; Gouskos, Loukas
Physical Review D, Vol. 101, Issue 5
DOI: 10.1103/PhysRevD.101.056019

Jet-images — deep learning edition
journal, July 2016

de Oliveira, Luke; Kagan, Michael; Mackey, Lester
Journal of High Energy Physics, Vol. 2016, Issue 7
DOI: 10.1007/JHEP07(2016)069

Quark-gluon tagging: Machine learning vs detector
journal, January 2019

Kasieczka, Gregor; Kiefer, Nicholas; Plehn, Tilman
SciPost Physics, Vol. 6, Issue 6
DOI: 10.21468/SciPostPhys.6.6.069

Machine learning uncertainties with adversarial neural networks
journal, January 2019

Englert, Christoph; Galler, Peter; Harris, Philip
The European Physical Journal C, Vol. 79, Issue 1
DOI: 10.1140/epjc/s10052-018-6511-8

INFERNO: Inference-Aware Neural Optimisation
journal, November 2019

de Castro, Pablo; Dorigo, Tommaso
Computer Physics Communications, Vol. 244
DOI: 10.1016/j.cpc.2019.06.007

Guiding new physics searches with unsupervised learning
journal, March 2019

De Simone, Andrea; Jacques, Thomas
The European Physical Journal C, Vol. 79, Issue 4
DOI: 10.1140/epjc/s10052-019-6787-3

Reports of my demise are greatly exaggerated: $N$-subjettiness taggers take on jet images
journal, January 2019

Moore, Liam; Nordström, Karl; Varma, Sreedevi
SciPost Physics, Vol. 7, Issue 3
DOI: 10.21468/SciPostPhys.7.3.036

Searching for new physics with deep autoencoders
journal, April 2020

Farina, Marco; Nakai, Yuichiro; Shih, David
Physical Review D, Vol. 101, Issue 7
DOI: 10.1103/PhysRevD.101.075021

Interpretable deep learning for two-prong jet classification with jet spectra
journal, July 2019

Chakraborty, Amit; Lim, Sung Hak; Nojiri, Mihoko M.
Journal of High Energy Physics, Vol. 2019, Issue 7
DOI: 10.1007/JHEP07(2019)135

Searching for exotic particles in high-energy physics with deep learning
journal, July 2014

Baldi, P.; Sadowski, P.; Whiteson, D.
Nature Communications, Vol. 5, Issue 1
DOI: 10.1038/ncomms5308

Neural network-based top tagger with two-point energy correlations and geometry of soft emissions
journal, July 2020

Chakraborty, Amit; Lim, Sung Hak; Nojiri, Mihoko M.
Journal of High Energy Physics, Vol. 2020, Issue 7
DOI: 10.1007/JHEP07(2020)111

Pileup mitigation at the Large Hadron Collider with graph neural networks
journal, July 2019

Arjona Martínez, J.; Cerri, O.; Spiropulu, M.
The European Physical Journal Plus, Vol. 134, Issue 7
DOI: 10.1140/epjp/i2019-12710-3

QCD-aware recursive neural networks for jet physics
journal, January 2019

Louppe, Gilles; Cho, Kyunghyun; Becot, Cyril
Journal of High Energy Physics, Vol. 2019, Issue 1
DOI: 10.1007/JHEP01(2019)057

A deep neural network to search for new long-lived particles decaying to jets
journal, August 2020

,
Machine Learning: Science and Technology, Vol. 1, Issue 3, 035012
DOI: 10.1088/2632-2153/ab9023

Transferability of deep learning models in searches for new physics at colliders
journal, February 2020

Romão, M. Crispim; Castro, N. F.; Pedro, R.
Physical Review D, Vol. 101, Issue 3
DOI: 10.1103/PhysRevD.101.035042

Parton shower uncertainties in jet substructure analyses with deep neural networks
journal, January 2017

Barnard, James; Dawe, Edmund Noel; Dolan, Matthew J.
Physical Review D, Vol. 95, Issue 1
DOI: 10.1103/PhysRevD.95.014018

uBoost: a boosting method for producing uniform selection efficiencies from multivariate classifiers
journal, December 2013

Stevens, J.; Williams, M.
Journal of Instrumentation, Vol. 8, Issue 12
DOI: 10.1088/1748-0221/8/12/P12013

QCD or what?
journal, January 2019

Heimel, Theo; Kasieczka, Gregor; Plehn, Tilman
SciPost Physics, Vol. 6, Issue 3
DOI: 10.21468/SciPostPhys.6.3.030

The anti- k _t jet clustering algorithm
journal, April 2008

Cacciari, Matteo; Salam, Gavin P.; Soyez, Gregory
Journal of High Energy Physics, Vol. 2008, Issue 04
DOI: 10.1088/1126-6708/2008/04/063

SUSY Les Houches Accord: Interfacing SUSY Spectrum Calculators, Decay Packages, and Event Generators
journal, July 2004

Skands, P.; Allanach, B. C.; Baer, H.
Journal of High Energy Physics, Vol. 2004, Issue 07
DOI: 10.1088/1126-6708/2004/07/036

Pulling out all the tops with computer vision and deep learning
journal, October 2018

Macaluso, Sebastian; Shih, David
Journal of High Energy Physics, Vol. 2018, Issue 10
DOI: 10.1007/JHEP10(2018)121

A search for top squarks with R-parity-violating decays to all-hadronic final states with the ATLAS detector in s = 8 $$ \sqrt{s}=8 $$ TeV proton-proton collisions
journal, June 2016

Aad, G.; Abbott, B.; Abdallah, J.
Journal of High Energy Physics, Vol. 2016, Issue 6
DOI: 10.1007/JHEP06(2016)067

Decorrelated jet substructure tagging using adversarial neural networks
journal, October 2017

Shimmin, Chase; Sadowski, Peter; Baldi, Pierre
Physical Review D, Vol. 96, Issue 7
DOI: 10.1103/PhysRevD.96.074034

Simulation assisted likelihood-free anomaly detection
journal, May 2020

Andreassen, Anders; Nachman, Benjamin; Shih, David
Physical Review D, Vol. 101, Issue 9
DOI: 10.1103/PhysRevD.101.095004

Measurement of $σ B (W \to e ν)$ and $σ B (Z^{0} \to e^{+} e^{-})$ in $\bar{p} p$ collisions at $\sqrt{s} = 1800$ GeV
journal, July 1991

Abe, F.; Amidei, D.; Apollinari, G.
Physical Review D, Vol. 44, Issue 1
DOI: 10.1103/PhysRevD.44.29

Search for Higgs boson decays into pairs of light (pseudo)scalar particles in the γγjj final state in pp collisions at $\sqrt{s} = 13 TeV$ with the ATLAS detector
journal, July 2018

Aaboud, M.; Aad, G.; Abbott, B.
Physics Letters B, Vol. 782
DOI: 10.1016/j.physletb.2018.06.011

JEDI-net: a jet identification algorithm based on interaction networks
journal, January 2020

Moreno, Eric A.; Cerri, Olmo; Duarte, Javier M.
The European Physical Journal C, Vol. 80, Issue 1
DOI: 10.1140/epjc/s10052-020-7608-4

PYTHIA 6.4 physics and manual
journal, May 2006

Sjöstrand, Torbjörn; Mrenna, Stephen; Skands, Peter
Journal of High Energy Physics, Vol. 2006, Issue 05
DOI: 10.1088/1126-6708/2006/05/026

SUSY Les Houches Accord 2
journal, January 2009

Allanach, B. C.; Balázs, C.; Bélanger, G.
Computer Physics Communications, Vol. 180, Issue 1
DOI: 10.1016/j.cpc.2008.08.004

Adversarially-trained autoencoders for robust unsupervised new physics searches
journal, October 2019

Blance, Andrew; Spannowsky, Michael; Waite, Philip
Journal of High Energy Physics, Vol. 2019, Issue 10
DOI: 10.1007/JHEP10(2019)047

Deep-learning top taggers or the end of QCD?
journal, May 2017

Kasieczka, Gregor; Plehn, Tilman; Russell, Michael
Journal of High Energy Physics, Vol. 2017, Issue 5
DOI: 10.1007/JHEP05(2017)006

Jet charge and machine learning
journal, October 2018

Fraser, Katherine; Schwartz, Matthew D.
Journal of High Energy Physics, Vol. 2018, Issue 10
DOI: 10.1007/JHEP10(2018)093

Search for new phenomena in final states with large jet multiplicities and missing transverse momentum using $$ \sqrt{s} $$ = 13 TeV proton-proton collisions recorded by ATLAS in Run 2 of the LHC
journal, October 2020

Aad, G.; Abbott, B.; Abbott, D. C.
Journal of High Energy Physics, Vol. 2020, Issue 10
DOI: 10.1007/JHEP10(2020)062

A generic anti-QCD jet tagger
journal, November 2017

Aguilar-Saavedra, J. A.; Collins, Jack; Mishra, Rashmish K.
Journal of High Energy Physics, Vol. 2017, Issue 11
DOI: 10.1007/JHEP11(2017)163

Jet flavor classification in high-energy physics with deep neural networks
journal, December 2016

Guest, Daniel; Collado, Julian; Baldi, Pierre
Physical Review D, Vol. 94, Issue 11
DOI: 10.1103/PhysRevD.94.112002

Recursive Neural Networks in Quark/Gluon Tagging
journal, June 2018

Cheng, Taoli
Computing and Software for Big Science, Vol. 2, Issue 1
DOI: 10.1007/s41781-018-0007-y

A search for pair-produced resonances in four-jet final states at $$\sqrt{s}=13$$ s = 13 $$\text {TeV}$$ TeV with the ATLAS detector
journal, March 2018

Aaboud, M.; Aad, G.; Abbott, B.
The European Physical Journal C, Vol. 78, Issue 3
DOI: 10.1140/epjc/s10052-018-5693-4

Automating the construction of jet observables with machine learning
journal, November 2019

Datta, Kaustuv; Larkoski, Andrew; Nachman, Benjamin
Physical Review D, Vol. 100, Issue 9
DOI: 10.1103/PhysRevD.100.095016

Towards machine learning analytics for jet substructure
journal, September 2020

Kasieczka, Gregor; Marzani, Simone; Soyez, Gregory
Journal of High Energy Physics, Vol. 2020, Issue 9
DOI: 10.1007/JHEP09(2020)195

Brownian distance covariance
journal, December 2009

Székely, Gábor J.; Rizzo, Maria L.
The Annals of Applied Statistics, Vol. 3, Issue 4
DOI: 10.1214/09-AOAS312

Deep Learning and Its Application to LHC Physics
journal, October 2018

Guest, Dan; Cranmer, Kyle; Whiteson, Daniel
Annual Review of Nuclear and Particle Science, Vol. 68, Issue 1
DOI: 10.1146/annurev-nucl-101917-021019

Robust Jet Classifiers through Distance Correlation
journal, September 2020

Kasieczka, Gregor; Shih, David
Physical Review Letters, Vol. 125, Issue 12
DOI: 10.1103/PhysRevLett.125.122001

Finding new physics without learning about it: anomaly detection as a tool for searches at colliders
journal, January 2021

Crispim Romão, M.; Castro, N. F.; Pedro, R.
The European Physical Journal C, Vol. 81, Issue 1
DOI: 10.1140/epjc/s10052-020-08807-w

Thinking outside the ROCs: Designing Decorrelated Taggers (DDT) for jet substructure
journal, May 2016

Dolen, James; Harris, Philip; Marzani, Simone
Journal of High Energy Physics, Vol. 2016, Issue 5
DOI: 10.1007/JHEP05(2016)156

DELPHES 3: a modular framework for fast simulation of a generic collider experiment
journal, February 2014

de Favereau, J.; Delaere, C.; Demin, P.
Journal of High Energy Physics, Vol. 2014, Issue 2
DOI: 10.1007/JHEP02(2014)057

Energy dependence of mean multiplicities in gluon and quark jets at the next-to-next-to-next-to-leading order
journal, July 1999

Dremin, I. M.; Gary, J. W.
Physics Letters B, Vol. 459, Issue 1-3
DOI: 10.1016/S0370-2693(99)00713-3

Variational autoencoders for new physics mining at the Large Hadron Collider
journal, May 2019

Cerri, Olmo; Nguyen, Thong Q.; Pierini, Maurizio
Journal of High Energy Physics, Vol. 2019, Issue 5
DOI: 10.1007/JHEP05(2019)036

The Machine Learning landscape of top taggers
journal, January 2019

Kasieczka, Gregor; Plehn, Tilman; Butter, Anja
SciPost Physics, Vol. 7, Issue 1
DOI: 10.21468/SciPostPhys.7.1.014

Anomaly Detection for Resonant New Physics with Machine Learning
journal, December 2018

Collins, Jack; Howe, Kiel; Nachman, Benjamin
Physical Review Letters, Vol. 121, Issue 24
DOI: 10.1103/PhysRevLett.121.241803

QBDT, a new boosting decision tree method with systematical uncertainties into training for High Energy Physics
journal, June 2019

Xia, Li-Gang
Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, Vol. 930
DOI: 10.1016/j.nima.2019.03.088

Jet-images: computer vision inspired techniques for jet tagging
journal, February 2015

Cogan, Josh; Kagan, Michael; Strauss, Emanuel
Journal of High Energy Physics, Vol. 2015, Issue 2
DOI: 10.1007/JHEP02(2015)118

Learning multivariate new physics
journal, January 2021

D’Agnolo, Raffaele Tito; Grosso, Gaia; Pierini, Maurizio
The European Physical Journal C, Vol. 81, Issue 1
DOI: 10.1140/epjc/s10052-021-08853-y

Learning the latent structure of collider events
journal, October 2020

Dillon, B. M.; Faroughy, D. A.; Kamenik, J. F.
Journal of High Energy Physics, Vol. 2020, Issue 10
DOI: 10.1007/JHEP10(2020)206

New approaches for boosting to uniformity
journal, March 2015

Rogozhnikov, A.; Bukva, A.; Gligorov, V.
Journal of Instrumentation, Vol. 10, Issue 03
DOI: 10.1088/1748-0221/10/03/T03002

Tag N’ Train: a technique to train improved classifiers on unlabeled data
journal, January 2021

Amram, Oz; Suarez, Cristina Mantilla
Journal of High Energy Physics, Vol. 2021, Issue 1
DOI: 10.1007/JHEP01(2021)153

Identification of heavy-flavour jets with the CMS detector in pp collisions at 13 TeV
journal, May 2018

Sirunyan, A. M.; Tumasyan, A.; Adam, W.
Journal of Instrumentation, Vol. 13, Issue 05
DOI: 10.1088/1748-0221/13/05/P05011

Novel jet observables from machine learning
journal, March 2018

Datta, Kaustuv; Larkoski, Andrew J.
Journal of High Energy Physics, Vol. 2018, Issue 3
DOI: 10.1007/JHEP03(2018)086

An introduction to PYTHIA 8.2
journal, June 2015

Sjöstrand, Torbjörn; Ask, Stefan; Christiansen, Jesper R.
Computer Physics Communications, Vol. 191
DOI: 10.1016/j.cpc.2015.01.024

Convolved substructure: analytically decorrelating jet substructure observables
journal, May 2018

Moult, Ian; Nachman, Benjamin; Neill, Duff
Journal of High Energy Physics, Vol. 2018, Issue 5
DOI: 10.1007/JHEP05(2018)002

Measuring and testing dependence by correlation of distances
journal, December 2007

Székely, Gábor J.; Rizzo, Maria L.; Bakirov, Nail K.
The Annals of Statistics, Vol. 35, Issue 6
DOI: 10.1214/009053607000000505

Partial distance correlation with methods for dissimilarities
journal, December 2014

Székely, Gábor J.; Rizzo, Maria L.
The Annals of Statistics, Vol. 42, Issue 6
DOI: 10.1214/14-AOS1255

Playing tag with ANN: boosted top identification with pattern recognition
journal, July 2015

Almeida, Leandro G.; Backović, Mihailo; Cliche, Mathieu
Journal of High Energy Physics, Vol. 2015, Issue 7
DOI: 10.1007/JHEP07(2015)086

CapsNets continuing the convolutional quest
journal, January 2020

Diefenbacher, Sascha; Frost, Hermann; Kasieczka, Gregor
SciPost Physics, Vol. 8, Issue 2
DOI: 10.21468/SciPostPhys.8.2.023

The distance correlation $t$ -test of independence in high dimension
journal, May 2013

Székely, Gábor J.; Rizzo, Maria L.
Journal of Multivariate Analysis, Vol. 117
DOI: 10.1016/j.jmva.2013.02.012

Dijet Resonance Search with Weak Supervision Using $\sqrt{s} = 13 TeV$ $p p$ Collisions in the ATLAS Detector
journal, September 2020

Aad, G.; Abbott, B.; Abbott, D. C.
Physical Review Letters, Vol. 125, Issue 13
DOI: 10.1103/PhysRevLett.125.131801

Similar Records in DOE PAGES and OSTI.GOV collections:

Simulation assisted likelihood-free anomaly detection

Journal Article Andreassen, Anders ; Nachman, Benjamin ; Shih, David - Physical Review. D.

Given the lack of evidence for new particle discoveries at the Large Hadron Collider (LHC), it is critical to broaden the search program. A variety of model-independent searches have been proposed, adding sensitivity to unexpected signals. There are generally two types of such searches: those that rely heavily on simulations and those that are entirely based on (unlabeled) data. This paper introduces a hybrid method that makes the best of both approaches. For potential signals that are resonant in one known feature, this new method first learns a parametrized reweighting function to morph a given simulation to match the datamore »« less
Cited by 61
https://doi.org/10.1103/physrevd.101.095004
Robust Jet Classifiers through Distance Correlation

Journal Article Kasieczka, Gregor ; Shih, David - Physical Review Letters

While deep learning has proven to be extremely successful at supervised classification tasks at the LHC and beyond, for practical applications, raw classification accuracy is often not the only consideration. One crucial issue is the stability of network predictions, either versus changes of individual features of the input data or against systematic perturbations. We present a new method based on a novel application of "distance correlation," a measure quantifying nonlinear correlations, that achieves equal performance to state-of-the-art adversarial decorrelation networks but is much simpler and more stable to train. To demonstrate the effectiveness of our method, we carefully recast amore »« less
https://doi.org/10.1103/PhysRevLett.125.122001
Anomaly detection with density estimation

Journal Article Nachman, Benjamin ; Shih, David - Physical Review. D.

We leverage recent breakthroughs in neural density estimation to propose a new unsupervised ANOmaly detection with Density Estimation (ANODE) technique. By estimating the conditional probability density of the data in a signal region and in sidebands, and interpolating the latter into the signal region, a fully data-driven likelihood ratio of data versus background can be constructed. This likelihood ratio is broadly sensitive to overdensities in the data that could be due to localized anomalies. In addition, a unique potential benefit of the ANODE method is that the background can be directly estimated using the learned densities. Finally, ANODE is robustmore »« less
Cited by 104
https://doi.org/10.1103/physrevd.101.075042
Simulation-assisted decorrelation for resonant anomaly detection

Journal Article Benkendorfer, Kees ; Le Pottier, Luc ; Nachman, Benjamin - Physical Review. D.

A growing number of weak and unsupervised machine learning approaches to anomaly detection are being proposed to significantly extend the search program at the Large Hadron Collider (LHC) and elsewhere. One of the prototypical examples for these methods is the search for resonant new physics, where a bump hunt can be performed in an invariant mass spectrum after applying a classifier to enhance the presence of a potential signal. A significant challenge to methods that rely entirely on data is that they are susceptible to sculpting artificial bumps from the dependence of the machine learning classifier on the invariant mass.more »« less
https://doi.org/10.1103/PhysRevD.104.035003
ThickBrick: optimal event selection and categorization in high energy physics. Part I. Signal discovery

Journal Article Matchev, Konstantin T. ; Shyamsundar, Prasanth - Journal of High Energy Physics (Online)

We provide a prescription called ThickBrick to train optimal machine-learning-based event selectors and categorizers that maximize the statistical significance of a potential signal excess in high energy physics (HEP) experiments, as quantified by any of six different performance measures. For analyses where the signal search is performed in the distribution of some event variables, our prescription ensures that only the information complementary to those event variables is used in event selection and categorization. This eliminates a major misalignment with the physics goals of the analysis (maximizing the significance of an excess) that exists in the training of typical ML-based eventmore »« less
https://doi.org/10.1007/jhep03(2021)291

Full Text Available

Similar Records

Title: Automating the ABCD method with machine learning

Abstract

Citation Formats

Evolution of average multiplicities of quark and gluon jets journal, February 2000

Optimal Statistical Inference in the Presence of Systematic Uncertainties Using Neural Network Optimization Based on Binned Poisson Likelihoods with Nuisance Parameters journal, January 2021

Learning representations of irregular particle-detector geometry with distance-weighted graph networks journal, July 2019

Search for pair-produced resonances decaying to jet pairs in proton–proton collisions at s = 8 TeV journal, July 2015

Deep-learned Top Tagging with a Lorentz Layer journal, January 2018

Electromagnetic showers beyond shower shapes journal, January 2020

Energy flow networks: deep sets for particle jets journal, January 2019

How much information is in a jet? journal, June 2017

End-to-End Physics Event Classification with CMS Open Data: Applying Image-Based Deep Learning to Detector Data for the Direct Classification of Collision Events at the LHC journal, March 2020

Learning new physics from a machine journal, January 2019

Deep learning in color: towards automated quark/gluon jet discrimination journal, January 2017

Boosted W and Z tagging with jet charge and deep learning journal, March 2020

Interaction networks for the identification of boosted H → b b ¯ decays journal, July 2020

FastJet user manual: (for version 3.0.2) journal, March 2012

Topology Classification with Deep Learning to Improve Real-Time Event Selection at the LHC journal, August 2019

Energy flow polynomials: a complete linear basis for jet substructure journal, April 2018

Identification of heavy, energetic, hadronically decaying particles using machine-learning techniques journal, June 2020

Uncovering latent jet substructure journal, September 2019

Calorimetry with deep learning: particle simulation and reconstruction for collider physics journal, July 2020

Anomaly detection with density estimation journal, April 2020

Identifying boosted objects with N-subjettiness journal, March 2011

Jet substructure at the Large Hadron Collider: A review of recent advances in theory and machine learning journal, November 2019

Fast convolutional neural networks for identifying long-lived particles in a high-granularity calorimeter journal, December 2020

Mass agnostic jet taggers journal, January 2020

Extending the search for new resonances with machine learning journal, January 2019

Learning to classify from impure samples with high-dimensional data journal, July 2018

Novelty detection meets collider physics journal, April 2020

Search for Higgs Boson Decays into a Z Boson and a Light Hadronically Decaying Resonance Using 13 TeV p p Collision Data from the ATLAS Detector journal, November 2020

Boosting H → b b ¯ $$ H\to b\overline{b} $$ with machine learning journal, October 2018

Parameterized neural networks for high-energy physics journal, April 2016

Maximizing boosted top identification by minimizing N-subjettiness journal, February 2012

Top Tagging: A Method for Identifying Boosted Hadronically Decaying Top Quarks journal, October 2008

Reducing the Dependence of the Neural Network Function to Systematic Uncertainties in the Input Space journal, February 2020

Jet tagging via particle clouds journal, March 2020

Jet-images — deep learning edition journal, July 2016

Quark-gluon tagging: Machine learning vs detector journal, January 2019

Machine learning uncertainties with adversarial neural networks journal, January 2019

INFERNO: Inference-Aware Neural Optimisation journal, November 2019

Guiding new physics searches with unsupervised learning journal, March 2019

Reports of my demise are greatly exaggerated: $N$-subjettiness taggers take on jet images journal, January 2019

Searching for new physics with deep autoencoders journal, April 2020

Interpretable deep learning for two-prong jet classification with jet spectra journal, July 2019

Searching for exotic particles in high-energy physics with deep learning journal, July 2014

Neural network-based top tagger with two-point energy correlations and geometry of soft emissions journal, July 2020

Pileup mitigation at the Large Hadron Collider with graph neural networks journal, July 2019

QCD-aware recursive neural networks for jet physics journal, January 2019

A deep neural network to search for new long-lived particles decaying to jets journal, August 2020

Transferability of deep learning models in searches for new physics at colliders journal, February 2020

Parton shower uncertainties in jet substructure analyses with deep neural networks journal, January 2017

uBoost: a boosting method for producing uniform selection efficiencies from multivariate classifiers journal, December 2013

QCD or what? journal, January 2019

The anti- k t jet clustering algorithm journal, April 2008

SUSY Les Houches Accord: Interfacing SUSY Spectrum Calculators, Decay Packages, and Event Generators journal, July 2004

Pulling out all the tops with computer vision and deep learning journal, October 2018

A search for top squarks with R-parity-violating decays to all-hadronic final states with the ATLAS detector in s = 8 $$ \sqrt{s}=8 $$ TeV proton-proton collisions journal, June 2016

Decorrelated jet substructure tagging using adversarial neural networks journal, October 2017

Simulation assisted likelihood-free anomaly detection journal, May 2020

Measurement of σ B ( W → e ν ) and σ B ( Z 0 → e + e − ) in p ¯ p collisions at s = 1800 GeV journal, July 1991

Search for Higgs boson decays into pairs of light (pseudo)scalar particles in the γγjj final state in pp collisions at s = 13 TeV with the ATLAS detector journal, July 2018

JEDI-net: a jet identification algorithm based on interaction networks journal, January 2020

PYTHIA 6.4 physics and manual journal, May 2006

SUSY Les Houches Accord 2 journal, January 2009

Adversarially-trained autoencoders for robust unsupervised new physics searches journal, October 2019

Deep-learning top taggers or the end of QCD? journal, May 2017

Jet charge and machine learning journal, October 2018

Search for new phenomena in final states with large jet multiplicities and missing transverse momentum using $$ \sqrt{s} $$ = 13 TeV proton-proton collisions recorded by ATLAS in Run 2 of the LHC journal, October 2020

A generic anti-QCD jet tagger journal, November 2017

Jet flavor classification in high-energy physics with deep neural networks journal, December 2016

Recursive Neural Networks in Quark/Gluon Tagging journal, June 2018

A search for pair-produced resonances in four-jet final states at $$\sqrt{s}=13$$ s = 13 $$\text {TeV}$$ TeV with the ATLAS detector journal, March 2018

Automating the construction of jet observables with machine learning journal, November 2019

Towards machine learning analytics for jet substructure journal, September 2020

Brownian distance covariance journal, December 2009

Deep Learning and Its Application to LHC Physics journal, October 2018

Robust Jet Classifiers through Distance Correlation journal, September 2020

Finding new physics without learning about it: anomaly detection as a tool for searches at colliders journal, January 2021

Thinking outside the ROCs: Designing Decorrelated Taggers (DDT) for jet substructure journal, May 2016

Evolution of average multiplicities of quark and gluon jets
journal, February 2000

Optimal Statistical Inference in the Presence of Systematic Uncertainties Using Neural Network Optimization Based on Binned Poisson Likelihoods with Nuisance Parameters
journal, January 2021

Learning representations of irregular particle-detector geometry with distance-weighted graph networks
journal, July 2019

Search for pair-produced resonances decaying to jet pairs in proton–proton collisions at $\sqrt{s} = 8 TeV$
journal, July 2015

Deep-learned Top Tagging with a Lorentz Layer
journal, January 2018

Electromagnetic showers beyond shower shapes
journal, January 2020

Energy flow networks: deep sets for particle jets
journal, January 2019

How much information is in a jet?
journal, June 2017

End-to-End Physics Event Classification with CMS Open Data: Applying Image-Based Deep Learning to Detector Data for the Direct Classification of Collision Events at the LHC
journal, March 2020

Learning new physics from a machine
journal, January 2019

Deep learning in color: towards automated quark/gluon jet discrimination
journal, January 2017

Boosted $W$ and $Z$ tagging with jet charge and deep learning
journal, March 2020

Interaction networks for the identification of boosted $H \to b \bar{b}$ decays
journal, July 2020

FastJet user manual: (for version 3.0.2)
journal, March 2012

Topology Classification with Deep Learning to Improve Real-Time Event Selection at the LHC
journal, August 2019

Energy flow polynomials: a complete linear basis for jet substructure
journal, April 2018

Identification of heavy, energetic, hadronically decaying particles using machine-learning techniques
journal, June 2020

Uncovering latent jet substructure
journal, September 2019

Calorimetry with deep learning: particle simulation and reconstruction for collider physics
journal, July 2020

Anomaly detection with density estimation
journal, April 2020

Identifying boosted objects with N-subjettiness
journal, March 2011

Jet substructure at the Large Hadron Collider: A review of recent advances in theory and machine learning
journal, November 2019

Fast convolutional neural networks for identifying long-lived particles in a high-granularity calorimeter
journal, December 2020

Mass agnostic jet taggers
journal, January 2020

Extending the search for new resonances with machine learning
journal, January 2019

Learning to classify from impure samples with high-dimensional data
journal, July 2018

Novelty detection meets collider physics
journal, April 2020

Search for Higgs Boson Decays into a $Z$ Boson and a Light Hadronically Decaying Resonance Using 13 TeV $p p$ Collision Data from the ATLAS Detector
journal, November 2020

Boosting H → b b ¯ $$ H\to b\overline{b} $$ with machine learning
journal, October 2018

Parameterized neural networks for high-energy physics
journal, April 2016

Maximizing boosted top identification by minimizing N-subjettiness
journal, February 2012

Top Tagging: A Method for Identifying Boosted Hadronically Decaying Top Quarks
journal, October 2008

Reducing the Dependence of the Neural Network Function to Systematic Uncertainties in the Input Space
journal, February 2020

Jet tagging via particle clouds
journal, March 2020

Jet-images — deep learning edition
journal, July 2016

Quark-gluon tagging: Machine learning vs detector
journal, January 2019

Machine learning uncertainties with adversarial neural networks
journal, January 2019

INFERNO: Inference-Aware Neural Optimisation
journal, November 2019

Guiding new physics searches with unsupervised learning
journal, March 2019

Reports of my demise are greatly exaggerated: $N$-subjettiness taggers take on jet images
journal, January 2019

Searching for new physics with deep autoencoders
journal, April 2020

Interpretable deep learning for two-prong jet classification with jet spectra
journal, July 2019

Searching for exotic particles in high-energy physics with deep learning
journal, July 2014

Neural network-based top tagger with two-point energy correlations and geometry of soft emissions
journal, July 2020

Pileup mitigation at the Large Hadron Collider with graph neural networks
journal, July 2019

QCD-aware recursive neural networks for jet physics
journal, January 2019

A deep neural network to search for new long-lived particles decaying to jets
journal, August 2020

Transferability of deep learning models in searches for new physics at colliders
journal, February 2020

Parton shower uncertainties in jet substructure analyses with deep neural networks
journal, January 2017

uBoost: a boosting method for producing uniform selection efficiencies from multivariate classifiers
journal, December 2013

QCD or what?
journal, January 2019

The anti- k _t jet clustering algorithm
journal, April 2008

SUSY Les Houches Accord: Interfacing SUSY Spectrum Calculators, Decay Packages, and Event Generators
journal, July 2004

Pulling out all the tops with computer vision and deep learning
journal, October 2018

A search for top squarks with R-parity-violating decays to all-hadronic final states with the ATLAS detector in s = 8 $$ \sqrt{s}=8 $$ TeV proton-proton collisions
journal, June 2016

Decorrelated jet substructure tagging using adversarial neural networks
journal, October 2017

Simulation assisted likelihood-free anomaly detection
journal, May 2020

Measurement of $σ B (W \to e ν)$ and $σ B (Z^{0} \to e^{+} e^{-})$ in $\bar{p} p$ collisions at $\sqrt{s} = 1800$ GeV
journal, July 1991

Search for Higgs boson decays into pairs of light (pseudo)scalar particles in the γγjj final state in pp collisions at $\sqrt{s} = 13 TeV$ with the ATLAS detector
journal, July 2018

JEDI-net: a jet identification algorithm based on interaction networks
journal, January 2020

PYTHIA 6.4 physics and manual
journal, May 2006

SUSY Les Houches Accord 2
journal, January 2009

Adversarially-trained autoencoders for robust unsupervised new physics searches
journal, October 2019

Deep-learning top taggers or the end of QCD?
journal, May 2017

Jet charge and machine learning
journal, October 2018

Search for new phenomena in final states with large jet multiplicities and missing transverse momentum using $$ \sqrt{s} $$ = 13 TeV proton-proton collisions recorded by ATLAS in Run 2 of the LHC
journal, October 2020

A generic anti-QCD jet tagger
journal, November 2017

Jet flavor classification in high-energy physics with deep neural networks
journal, December 2016

Recursive Neural Networks in Quark/Gluon Tagging
journal, June 2018

A search for pair-produced resonances in four-jet final states at $$\sqrt{s}=13$$ s = 13 $$\text {TeV}$$ TeV with the ATLAS detector
journal, March 2018

Automating the construction of jet observables with machine learning
journal, November 2019

Towards machine learning analytics for jet substructure
journal, September 2020

Brownian distance covariance
journal, December 2009

Deep Learning and Its Application to LHC Physics
journal, October 2018

Robust Jet Classifiers through Distance Correlation
journal, September 2020

Finding new physics without learning about it: anomaly detection as a tool for searches at colliders
journal, January 2021

Thinking outside the ROCs: Designing Decorrelated Taggers (DDT) for jet substructure
journal, May 2016

DELPHES 3: a modular framework for fast simulation of a generic collider experiment
journal, February 2014

Energy dependence of mean multiplicities in gluon and quark jets at the next-to-next-to-next-to-leading order
journal, July 1999

Variational autoencoders for new physics mining at the Large Hadron Collider
journal, May 2019