(Machine) learning to do more with less
Abstract
Determining the best method for training a machine learning algorithm is critical to maximizing its ability to classify data. In this paper, we compare the standard “fully supervised” approach (which relies on knowledge of event-by-event truth-level labels) with a recent proposal that instead utilizes class ratios as the only discriminating information provided during training. This so-called “weakly supervised” technique has access to less information than the fully supervised method and yet is still able to yield impressive discriminating power. In addition, weak supervision seems particularly well suited to particle physics since quantum mechanics is incompatible with the notion of mapping an individual event onto any single Feynman diagram. We examine the technique in detail — both analytically and numerically — with a focus on the robustness to issues of mischaracterizing the training samples. Weakly supervised networks turn out to be remarkably insensitive to a class of systematic mismodeling. Furthermore, we demonstrate that the event level outputs for weakly versus fully supervised networks are probing different kinematics, even though the numerical quality metrics are essentially identical. This implies that it should be possible to improve the overall classification ability by combining the output from the two types of networks. For concreteness,more »
- Publication Date:
- Research Org.:
- Univ. of Oregon, Eugene, OR (United States)
- Sponsoring Org.:
- USDOE Office of Science (SC), High Energy Physics (HEP)
- OSTI Identifier:
- 1512295
- Alternate Identifier(s):
- OSTI ID: 1736031
- Grant/Contract Number:
- SC0018191; SC0011640
- Resource Type:
- Accepted Manuscript
- Journal Name:
- Journal of High Energy Physics (Online)
- Additional Journal Information:
- Journal Name: Journal of High Energy Physics (Online); Journal Volume: 2018; Journal Issue: 2; Journal ID: ISSN 1029-8479
- Publisher:
- Springer Berlin
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS; Beyond Standard Model; Hadron-Hadron scattering (experiments); Particle correlations and fluctuations; Supersymmetry
Citation Formats
Cohen, Timothy, Freytsis, Marat, and Ostdiek, Bryan. (Machine) learning to do more with less. United States: N. p., 2018.
Web. doi:10.1007/jhep02(2018)034.
Cohen, Timothy, Freytsis, Marat, & Ostdiek, Bryan. (Machine) learning to do more with less. United States. https://doi.org/10.1007/jhep02(2018)034
Cohen, Timothy, Freytsis, Marat, and Ostdiek, Bryan. Tue .
"(Machine) learning to do more with less". United States. https://doi.org/10.1007/jhep02(2018)034. https://www.osti.gov/servlets/purl/1512295.
@article{osti_1512295,
title = {(Machine) learning to do more with less},
author = {Cohen, Timothy and Freytsis, Marat and Ostdiek, Bryan},
abstractNote = {Determining the best method for training a machine learning algorithm is critical to maximizing its ability to classify data. In this paper, we compare the standard “fully supervised” approach (which relies on knowledge of event-by-event truth-level labels) with a recent proposal that instead utilizes class ratios as the only discriminating information provided during training. This so-called “weakly supervised” technique has access to less information than the fully supervised method and yet is still able to yield impressive discriminating power. In addition, weak supervision seems particularly well suited to particle physics since quantum mechanics is incompatible with the notion of mapping an individual event onto any single Feynman diagram. We examine the technique in detail — both analytically and numerically — with a focus on the robustness to issues of mischaracterizing the training samples. Weakly supervised networks turn out to be remarkably insensitive to a class of systematic mismodeling. Furthermore, we demonstrate that the event level outputs for weakly versus fully supervised networks are probing different kinematics, even though the numerical quality metrics are essentially identical. This implies that it should be possible to improve the overall classification ability by combining the output from the two types of networks. For concreteness, we apply this technology to a signature of beyond the Standard Model physics to demonstrate that all these impressive features continue to hold in a scenario of relevance to the LHC. Example code is provided on GitHub.},
doi = {10.1007/jhep02(2018)034},
journal = {Journal of High Energy Physics (Online)},
number = 2,
volume = 2018,
place = {United States},
year = {Tue Feb 06 00:00:00 EST 2018},
month = {Tue Feb 06 00:00:00 EST 2018}
}
Free Publicly Available Full Text
Publisher's Version of Record
Other availability
Search WorldCat to find libraries that may hold this journal
Cited by: 56 works
Citation information provided by
Web of Science
Web of Science
Figures / Tables:
Figure 1: A schematic representation of a network with three layers (one hidden), such as the networks used in this paper. Each of the layers is marked by a dashed green box. All of the yellow circles represent a real number (node). The green lines are the connections between nodesmore »
All figures and tables
(16 total)
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.
Works referenced in this record:
The anti- k t jet clustering algorithm
journal, April 2008
- Cacciari, Matteo; Salam, Gavin P.; Soyez, Gregory
- Journal of High Energy Physics, Vol. 2008, Issue 04
Least squares quantization in PCM
journal, March 1982
- Lloyd, S.
- IEEE Transactions on Information Theory, Vol. 28, Issue 2
Ensemble-based classifiers
journal, November 2009
- Rokach, Lior
- Artificial Intelligence Review, Vol. 33, Issue 1-2, p. 1-39
Macroscopic strings and ``quirks'' at colliders
journal, November 2009
- Kang, Junhai; Luty, Markus A.
- Journal of High Energy Physics, Vol. 2009, Issue 11
How much information is in a jet?
journal, June 2017
- Datta, Kaustuv; Larkoski, Andrew
- Journal of High Energy Physics, Vol. 2017, Issue 6
Popular Ensemble Methods: An Empirical Study
journal, July 1999
- Opitz, D.; Maclin, R.
- Journal of Artificial Intelligence Research, Vol. 11
Deep learning in color: towards automated quark/gluon jet discrimination
journal, January 2017
- Komiske, Patrick T.; Metodiev, Eric M.; Schwartz, Matthew D.
- Journal of High Energy Physics, Vol. 2017, Issue 1
Deep-learning top taggers or the end of QCD?
journal, May 2017
- Kasieczka, Gregor; Plehn, Tilman; Russell, Michael
- Journal of High Energy Physics, Vol. 2017, Issue 5
The automated computation of tree-level and next-to-leading order differential cross sections, and their matching to parton shower simulations
journal, July 2014
- Alwall, J.; Frederix, R.; Frixione, S.
- Journal of High Energy Physics, Vol. 2014, Issue 7
FastJet user manual: (for version 3.0.2)
journal, March 2012
- Cacciari, Matteo; Salam, Gavin P.; Soyez, Gregory
- The European Physical Journal C, Vol. 72, Issue 3
Multiple instance classification: Review, taxonomy and comparative study
journal, August 2013
- Amores, Jaume
- Artificial Intelligence, Vol. 201
Ensemble based systems in decision making
journal, January 2006
- Polikar, R.
- IEEE Circuits and Systems Magazine, Vol. 6, Issue 3
Jets and Photons
journal, March 2013
- Ellis, Stephen D.; Roy, Tuhin S.; Scholtz, Jakub
- Physical Review Letters, Vol. 110, Issue 12
SCYNet: testing supersymmetric models at the LHC with neural networks
journal, October 2017
- Bechtle, Philip; Belkner, Sebastian; Dercks, Daniel
- The European Physical Journal C, Vol. 77, Issue 10
Triggering soft bombs at the LHC
journal, August 2017
- Knapen, Simon; Griso, Simone Pagan; Papucci, Michele
- Journal of High Energy Physics, Vol. 2017, Issue 8
Solving the multiple instance problem with axis-parallel rectangles
journal, January 1997
- Dietterich, Thomas G.; Lathrop, Richard H.; Lozano-Pérez, Tomás
- Artificial Intelligence, Vol. 89, Issue 1-2
Parameterized neural networks for high-energy physics
journal, April 2016
- Baldi, Pierre; Cranmer, Kyle; Faucett, Taylor
- The European Physical Journal C, Vol. 76, Issue 5
Machine learning for event selection in high energy physics
journal, December 2009
- Whiteson, Shimon; Whiteson, Daniel
- Engineering Applications of Artificial Intelligence, Vol. 22, Issue 8
Dissecting jets and missing energy searches using n-body extended simplified models
journal, August 2016
- Cohen, Timothy; Dolan, Matthew J.; El Hedri, Sonia
- Journal of High Energy Physics, Vol. 2016, Issue 8
PhysicsGP: A Genetic Programming approach to event selection
journal, May 2005
- Cranmer, Kyle; Bowman, R. Sean
- Computer Physics Communications, Vol. 167, Issue 3
The BSM-AI project: SUSY-AI–generalizing LHC limits on supersymmetry with machine learning
journal, April 2017
- Caron, Sascha; Kim, Jong Soo; Rolbiecki, Krzysztof
- The European Physical Journal C, Vol. 77, Issue 4
Classification without labels: learning from mixed samples in high energy physics
journal, October 2017
- Metodiev, Eric M.; Nachman, Benjamin; Thaler, Jesse
- Journal of High Energy Physics, Vol. 2017, Issue 10
Fast supersymmetry phenomenology at the Large Hadron Collider using machine learning techniques
journal, April 2012
- Buckley, A.; Shilton, A.; White, M. J.
- Computer Physics Communications, Vol. 183, Issue 4
Jet-images — deep learning edition
journal, July 2016
- de Oliveira, Luke; Kagan, Michael; Mackey, Lester
- Journal of High Energy Physics, Vol. 2016, Issue 7
Echoes of a hidden valley at hadron colliders
journal, August 2007
- Strassler, Matthew J.; Zurek, Kathryn M.
- Physics Letters B, Vol. 651, Issue 5-6
Weakly supervised classification in high energy physics
journal, May 2017
- Dery, Lucio Mwinmaarong; Nachman, Benjamin; Rubbo, Francesco
- Journal of High Energy Physics, Vol. 2017, Issue 5
Jet-images: computer vision inspired techniques for jet tagging
journal, February 2015
- Cogan, Josh; Kagan, Michael; Strauss, Emanuel
- Journal of High Energy Physics, Vol. 2015, Issue 2
Learning Particle Physics by Example: Location-Aware Generative Adversarial Networks for Physics Synthesis
journal, September 2017
- de Oliveira, Luke; Paganini, Michela; Nachman, Benjamin
- Computing and Software for Big Science, Vol. 1, Issue 1
Playing tag with ANN: boosted top identification with pattern recognition
journal, July 2015
- Almeida, Leandro G.; Backović, Mihailo; Cliche, Mathieu
- Journal of High Energy Physics, Vol. 2015, Issue 7
Searching for exotic particles in high-energy physics with deep learning
journal, July 2014
- Baldi, P.; Sadowski, P.; Whiteson, D.
- Nature Communications, Vol. 5, Issue 1
Performance of b -jet identification in the ATLAS experiment
journal, January 2016
- Collaboration, Atlas
- Journal of Instrumentation, Vol. 11, Issue 04, p. P04008-P04008
SCYNet: Testing supersymmetric models at the LHC with neural networks
text, January 2017
- Bechtle, Philip; Belkner, Sebastian; Dercks, Daniel
- RWTH Aachen University
The automated computation of tree-level and next-to-leading order differential cross sections, and their matching to parton shower simulations
text, January 2014
- Alwall, J.; Frederix, R.; Frixione, S.
- Springer
Deep-learning top taggers or the end of QCD
text, January 2017
- Kasieczka, Gregor; Plehn, Tilman; Russell, Michael
- ETH Zurich
The anti-k_t jet clustering algorithm
text, January 2008
- Cacciari, Matteo; Salam, Gavin P.; Soyez, Gregory
- arXiv
Searching for Exotic Particles in High-Energy Physics with Deep Learning
text, January 2014
- Baldi, Pierre; Sadowski, Peter; Whiteson, Daniel
- arXiv
Jet-Images: Computer Vision Inspired Techniques for Jet Tagging
text, January 2014
- Cogan, Josh; Kagan, Michael; Strauss, Emanuel
- arXiv
Dissecting Jets and Missing Energy Searches Using $n$-body Extended Simplified Models
text, January 2016
- Cohen, Timothy; Dolan, Matthew J.; Hedri, Sonia El
- arXiv
Deep learning in color: towards automated quark/gluon jet discrimination
text, January 2016
- Komiske, Patrick T.; Metodiev, Eric M.; Schwartz, Matthew D.
- arXiv
Learning Particle Physics by Example: Location-Aware Generative Adversarial Networks for Physics Synthesis
text, January 2017
- de Oliveira, Luke; Paganini, Michela; Nachman, Benjamin
- arXiv
Deep-learning Top Taggers or The End of QCD?
text, January 2017
- Kasieczka, Gregor; Plehn, Tilman; Russell, Michael
- arXiv
Weakly Supervised Classification in High Energy Physics
text, January 2017
- Dery, Lucio Mwinmaarong; Nachman, Benjamin; Rubbo, Francesco
- arXiv
Machine learning for event selection in high energy physics
journal, December 2009
- Whiteson, Shimon; Whiteson, Daniel
- Engineering Applications of Artificial Intelligence, Vol. 22, Issue 8
Determination of the CMSSM parameters using neural networks
journal, October 2013
- Bornhauser, Nicki; Drees, Manuel
- Physical Review D, Vol. 88, Issue 7
Parameterized neural networks for high-energy physics
journal, April 2016
- Baldi, Pierre; Cranmer, Kyle; Faucett, Taylor
- The European Physical Journal C, Vol. 76, Issue 5
Signals of New Physics in the Underlying Event
text, January 2008
- Harnik, Roni; Wizansky, Tommer
- arXiv
Works referencing / citing this record:
Quark jet versus gluon jet: fully-connected neural networks with high-level features
journal, June 2019
- Luo, Hui; Luo, Ming-Xing; Wang, Kai
- Science China Physics, Mechanics & Astronomy, Vol. 62, Issue 9
Identifying the Relevant Dependencies of the Neural Network Response on Characteristics of the Input Space
journal, September 2018
- Wunsch, Stefan; Friese, Raphael; Wolf, Roger
- Computing and Software for Big Science, Vol. 2, Issue 1
Deep learning for -parity violating supersymmetry searches at the LHC
journal, October 2018
- Guo, Jun; Li, Jinmian; Li, Tianjun
- Physical Review D, Vol. 98, Issue 7
Machine learning uncertainties with adversarial neural networks
journal, January 2019
- Englert, Christoph; Galler, Peter; Harris, Philip
- The European Physical Journal C, Vol. 79, Issue 1
JUNIPR: a framework for unsupervised machine learning in particle physics
journal, February 2019
- Andreassen, Anders; Feige, Ilya; Frye, Christopher
- The European Physical Journal C, Vol. 79, Issue 2
Guiding new physics searches with unsupervised learning
journal, March 2019
- De Simone, Andrea; Jacques, Thomas
- The European Physical Journal C, Vol. 79, Issue 4
Automating the construction of jet observables with machine learning
text, January 2019
- Datta, Kaustuv; Larkoski, Andrew; Nachman, Benjamin
- ETH Zurich
The Machine Learning landscape of top taggers
text, January 2019
- Kasieczka, Gregor; Plehn, Tilman; Butter, Anja
- RWTH Aachen University
Learning to Classify from Impure Samples with High-Dimensional Data
text, January 2018
- Komiske, Patrick T.; Metodiev, Eric M.; Nachman, Benjamin
- arXiv
Identifying the relevant dependencies of the neural network response on characteristics of the input space
text, January 2018
- Wunsch, Stefan; Friese, Raphael; Wolf, Roger
- arXiv
Machine Learning Uncertainties with Adversarial Neural Networks
text, January 2018
- Englert, Christoph; Galler, Peter; Harris, Philip
- arXiv
Automating the Construction of Jet Observables with Machine Learning
text, January 2019
- Datta, Kaustuv; Larkoski, Andrew; Nachman, Benjamin
- arXiv
CapsNets Continuing the Convolutional Quest
text, January 2019
- Diefenbacher, Sascha; Frost, Hermann; Kasieczka, Gregor
- arXiv
Unsupervised Post-Tuning of Deep Neural Networks
conference, July 2021
- Cerisara, Christophe; Caillon, Paul; Le Berre, Guillaume
- 2021 International Joint Conference on Neural Networks (IJCNN)
Dijet Resonance Search with Weak Supervision Using sqrt[s]=13 TeV pp Collisions in the ATLAS Detector.
text, January 2020
- Aad, G.; Abbott, B.; Abbott, Dc
- Apollo - University of Cambridge Repository
The Calabi-Yau Landscape: from Geometry, to Physics, to Machine-Learning
preprint, January 2018
- He, Yang-Hui
- arXiv
Uncovering latent jet substructure
text, January 2019
- Dillon, Barry M.; Faroughy, Darius A.; Kamenik, Jernej F.
- arXiv
Searching for periodic signals in kinematic distributions using continuous wavelet transforms
text, January 2019
- Beauchesne, Hugues; Kats, Yevgeny
- arXiv
Dealing with Nuisance Parameters using Machine Learning in High Energy Physics: a Review
preprint, January 2020
- Dorigo, Tommaso; de Castro, Pablo
- arXiv
Anomaly Detection for Physics Analysis and Less than Supervised Learning
preprint, January 2020
- Nachman, Benjamin
- arXiv
Figures / Tables found in this record:
Figures/Tables have been extracted from DOE-funded journal article accepted manuscripts.