Classification without labels: learning from mixed samples in high energy physics
Abstract
Modern machine learning techniques can be used to construct powerful models for difficult collider physics problems. In many applications, however, these models are trained on imperfect simulations due to a lack of truthlevel information in the data, which risks the model learning artifacts of the simulation. In this paper, we introduce the paradigm of classification without labels (CWoLa) in which a classifier is trained to distinguish statistical mixtures of classes, which are common in collider physics. Crucially, neither individual labels nor class proportions are required, yet we prove that the optimal classifier in the CWoLa paradigm is also the optimal classifier in the traditional fullysupervised case where all label information is available. After demonstrating the power of this method in an analytical toy example, we consider a realistic benchmark for collider physics: distinguishing quark versus gluoninitiated jets using mixed quark/gluon training samples. More generally, CWoLa can be applied to any classification problem where labels or class proportions are unknown or simulations are unreliable, but statistical mixtures of the classes are available.
 Authors:

 Massachusetts Inst. of Technology (MIT), Cambridge, MA (United States)
 Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
 Publication Date:
 Research Org.:
 Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
 Sponsoring Org.:
 USDOE Office of Science (SC)
 OSTI Identifier:
 1421837
 Grant/Contract Number:
 AC0205CH11231
 Resource Type:
 Accepted Manuscript
 Journal Name:
 Journal of High Energy Physics (Online)
 Additional Journal Information:
 Journal Name: Journal of High Energy Physics (Online); Journal Volume: 2017; Journal Issue: 10; Journal ID: ISSN 10298479
 Publisher:
 Springer Berlin
 Country of Publication:
 United States
 Language:
 English
 Subject:
 72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS; Jets
Citation Formats
Metodiev, Eric M., Nachman, Benjamin, and Thaler, Jesse. Classification without labels: learning from mixed samples in high energy physics. United States: N. p., 2017.
Web. doi:10.1007/JHEP10(2017)174.
Metodiev, Eric M., Nachman, Benjamin, & Thaler, Jesse. Classification without labels: learning from mixed samples in high energy physics. United States. doi:10.1007/JHEP10(2017)174.
Metodiev, Eric M., Nachman, Benjamin, and Thaler, Jesse. Wed .
"Classification without labels: learning from mixed samples in high energy physics". United States. doi:10.1007/JHEP10(2017)174. https://www.osti.gov/servlets/purl/1421837.
@article{osti_1421837,
title = {Classification without labels: learning from mixed samples in high energy physics},
author = {Metodiev, Eric M. and Nachman, Benjamin and Thaler, Jesse},
abstractNote = {Modern machine learning techniques can be used to construct powerful models for difficult collider physics problems. In many applications, however, these models are trained on imperfect simulations due to a lack of truthlevel information in the data, which risks the model learning artifacts of the simulation. In this paper, we introduce the paradigm of classification without labels (CWoLa) in which a classifier is trained to distinguish statistical mixtures of classes, which are common in collider physics. Crucially, neither individual labels nor class proportions are required, yet we prove that the optimal classifier in the CWoLa paradigm is also the optimal classifier in the traditional fullysupervised case where all label information is available. After demonstrating the power of this method in an analytical toy example, we consider a realistic benchmark for collider physics: distinguishing quark versus gluoninitiated jets using mixed quark/gluon training samples. More generally, CWoLa can be applied to any classification problem where labels or class proportions are unknown or simulations are unreliable, but statistical mixtures of the classes are available.},
doi = {10.1007/JHEP10(2017)174},
journal = {Journal of High Energy Physics (Online)},
number = 10,
volume = 2017,
place = {United States},
year = {2017},
month = {10}
}
Web of Science
Works referenced in this record:
Weak supervision and other nonstandard classification problems: A taxonomy
journal, January 2016
 HernándezGonzález, Jerónimo; Inza, Iñaki; Lozano, Jose A.
 Pattern Recognition Letters, Vol. 69
Jet Substructure as a New HiggsSearch Channel at the Large Hadron Collider
journal, June 2008
 Butterworth, Jonathan M.; Davison, Adam R.; Rubin, Mathieu
 Physical Review Letters, Vol. 100, Issue 24
Jet shapes and jet algorithms in SCET
journal, November 2010
 Ellis, Stephen D.; Vermilion, Christopher K.; Walsh, Jonathan R.
 Journal of High Energy Physics, Vol. 2010, Issue 11
The anti k _{ t } jet clustering algorithm
journal, April 2008
 Cacciari, Matteo; Salam, Gavin P.; Soyez, Gregory
 Journal of High Energy Physics, Vol. 2008, Issue 04
Soft drop
journal, May 2014
 Larkoski, Andrew J.; Marzani, Simone; Soyez, Gregory
 Journal of High Energy Physics, Vol. 2014, Issue 5
Substructure of high ${p}_{T}$ jets at the LHC
journal, April 2009
 Almeida, Leandro G.; Lee, Seung J.; Perez, Gilad
 Physical Review D, Vol. 79, Issue 7
A brief introduction to PYTHIA 8.1
journal, June 2008
 Sjöstrand, Torbjörn; Mrenna, Stephen; Skands, Peter
 Computer Physics Communications, Vol. 178, Issue 11
How much information is in a jet?
journal, June 2017
 Datta, Kaustuv; Larkoski, Andrew
 Journal of High Energy Physics, Vol. 2017, Issue 6
Identification of boosted, hadronically decaying W bosons and comparisons with ATLAS data taken at $$\sqrt{s} = 8$$ s = 8 TeV
journal, March 2016
 Aad, G.; Abbott, B.; Abdallah, J.
 The European Physical Journal C, Vol. 76, Issue 3
Jet observables without jet algorithms
journal, April 2014
 Bertolini, Daniele; Chan, Tucker; Thaler, Jesse
 Journal of High Energy Physics, Vol. 2014, Issue 4
Identification of bquark jets with the CMS experiment
journal, April 2013
 collaboration, The CMS
 Journal of Instrumentation, Vol. 8, Issue 04
Event shape–energy flow correlations
journal, July 2003
 Berger, Carola F.; Kúcs, Tibor; Sterman, George
 Physical Review D, Vol. 68, Issue 1
Jetimages — deep learning edition
journal, July 2016
 de Oliveira, Luke; Kagan, Michael; Mackey, Lester
 Journal of High Energy Physics, Vol. 2016, Issue 7
Deeplearning top taggers or the end of QCD?
journal, May 2017
 Kasieczka, Gregor; Plehn, Tilman; Russell, Michael
 Journal of High Energy Physics, Vol. 2017, Issue 5
Deep learning in color: towards automated quark/gluon jet discrimination
journal, January 2017
 Komiske, Patrick T.; Metodiev, Eric M.; Schwartz, Matthew D.
 Journal of High Energy Physics, Vol. 2017, Issue 1
Quarkgluon separation in threejet events
journal, May 1981
 Nilles, H. P.; Streng, K. H.
 Physical Review D, Vol. 23, Issue 9
Factorization for groomed jet substructure beyond the nexttoleading logarithm
journal, July 2016
 Frye, Christopher; Larkoski, Andrew J.; Schwartz, Matthew D.
 Journal of High Energy Physics, Vol. 2016, Issue 7
FastJet user manual: (for version 3.0.2)
journal, March 2012
 Cacciari, Matteo; Salam, Gavin P.; Soyez, Gregory
 The European Physical Journal C, Vol. 72, Issue 3
Weakly supervised classification in high energy physics
journal, May 2017
 Dery, Lucio Mwinmaarong; Nachman, Benjamin; Rubbo, Francesco
 Journal of High Energy Physics, Vol. 2017, Issue 5
Jetimages: computer vision inspired techniques for jet tagging
journal, February 2015
 Cogan, Josh; Kagan, Michael; Strauss, Emanuel
 Journal of High Energy Physics, Vol. 2015, Issue 2
Pure samples of quark and gluon jets at the LHC
journal, October 2011
 Gallicchio, Jason; Schwartz, Matthew D.
 Journal of High Energy Physics, Vol. 2011, Issue 10
Measurement of the chargedparticle multiplicity inside jets from $$\sqrt{s}=8$$ s = 8 $${\mathrm{TeV}}$$ TeV pp collisions with the ATLAS detector
journal, June 2016
 Aad, G.; Abbott, B.; Abdallah, J.
 The European Physical Journal C, Vol. 76, Issue 6
Systematics of quark/gluon tagging
journal, July 2017
 Gras, Philippe; Höche, Stefan; Kar, Deepak
 Journal of High Energy Physics, Vol. 2017, Issue 7
Jet shapes with the broadening axis
journal, April 2014
 Larkoski, Andrew J.; Neill, Duff; Thaler, Jesse
 Journal of High Energy Physics, Vol. 2014, Issue 4
Playing tag with ANN: boosted top identification with pattern recognition
journal, July 2015
 Almeida, Leandro G.; Backović, Mihailo; Cliche, Mathieu
 Journal of High Energy Physics, Vol. 2015, Issue 7
Classification with asymmetric label noise: Consistency and maximal denoising
journal, January 2016
 Blanchard, Gilles; Flaska, Marek; Handy, Gregory
 Electronic Journal of Statistics, Vol. 10, Issue 2
Quark and gluon jet substructure
journal, April 2013
 Gallicchio, Jason; Schwartz, Matthew D.
 Journal of High Energy Physics, Vol. 2013, Issue 4
Towards an understanding of jet substructure
journal, September 2013
 Dasgupta, Mrinal; Fregoso, Alessandro; Marzani, Simone
 Journal of High Energy Physics, Vol. 2013, Issue 9
Using neural networks to identify jets
journal, February 1991
 Lönnblad, Leif; Peterson, Carsten; Rögnvaldsson, Thorsteinn
 Nuclear Physics B, Vol. 349, Issue 3
Lightquark and gluon jet discrimination in $$pp$$ p p collisions at $$\sqrt{s}=7\mathrm {\ TeV}$$ s = 7 TeV with the ATLAS detector
journal, August 2014
 Aad, G.; Abbott, B.; Abdallah, J.
 The European Physical Journal C, Vol. 74, Issue 8
Jet trimming
journal, February 2010
 Krohn, David; Thaler, Jesse; Wang, LianTao
 Journal of High Energy Physics, Vol. 2010, Issue 2
Quark and Gluon Tagging at the LHC
journal, October 2011
 Gallicchio, Jason; Schwartz, Matthew D.
 Physical Review Letters, Vol. 107, Issue 17
Gaining (mutual) information about quark/gluon discrimination
journal, November 2014
 Larkoski, Andrew J.; Thaler, Jesse; Waalewijn, Wouter J.
 Journal of High Energy Physics, Vol. 2014, Issue 11
Works referencing / citing this record:
The Machine Learning landscape of top taggers
journal, January 2019
 Kasieczka, Gregor; Plehn, Tilman; Butter, Anja
 SciPost Physics, Vol. 7, Issue 1
A theory of quark vs. gluon discrimination
journal, October 2019
 Larkoski, Andrew J.; Metodiev, Eric M.
 Journal of High Energy Physics, Vol. 2019, Issue 10
An operational definition of quark and gluon jets
journal, November 2018
 Komiske, Patrick T.; Metodiev, Eric M.; Thaler, Jesse
 Journal of High Energy Physics, Vol. 2018, Issue 11
Quark jet versus gluon jet: fullyconnected neural networks with highlevel features
journal, June 2019
 Luo, Hui; Luo, MingXing; Wang, Kai
 Science China Physics, Mechanics & Astronomy, Vol. 62, Issue 9
A theory of quark vs. gluon discrimination
journal, October 2019
 Larkoski, Andrew J.; Metodiev, Eric M.
 Journal of High Energy Physics, Vol. 2019, Issue 10
An operational definition of quark and gluon jets
journal, November 2018
 Komiske, Patrick T.; Metodiev, Eric M.; Thaler, Jesse
 Journal of High Energy Physics, Vol. 2018, Issue 11
Quark jet versus gluon jet: fullyconnected neural networks with highlevel features
journal, June 2019
 Luo, Hui; Luo, MingXing; Wang, Kai
 Science China Physics, Mechanics & Astronomy, Vol. 62, Issue 9
Deep learning for $R$ parity violating supersymmetry searches at the LHC
journal, October 2018
 Guo, Jun; Li, Jinmian; Li, Tianjun
 Physical Review D, Vol. 98, Issue 7
Jet Topics: Disentangling Quarks and Gluons at Colliders
journal, June 2018
 Metodiev, Eric M.; Thaler, Jesse
 Physical Review Letters, Vol. 120, Issue 24
The Machine Learning landscape of top taggers
journal, January 2019
 Kasieczka, Gregor; Plehn, Tilman; Butter, Anja
 SciPost Physics, Vol. 7, Issue 1
Reweighting a parton shower using a neural network: the finalstate case
journal, January 2019
 Bothmann, Enrico; Del Debbio, Luigi
 Journal of High Energy Physics, Vol. 2019, Issue 1
QCDaware recursive neural networks for jet physics
journal, January 2019
 Louppe, Gilles; Cho, Kyunghyun; Becot, Cyril
 Journal of High Energy Physics, Vol. 2019, Issue 1
Energy flow networks: deep sets for particle jets
journal, January 2019
 Komiske, Patrick T.; Metodiev, Eric M.; Thaler, Jesse
 Journal of High Energy Physics, Vol. 2019, Issue 1
(Machine) learning to do more with less
journal, February 2018
 Cohen, Timothy; Freytsis, Marat; Ostdiek, Bryan
 Journal of High Energy Physics, Vol. 2018, Issue 2
Infrared safety of a neuralnet top tagging algorithm
journal, February 2019
 Choi, Suyong; Lee, Seung J.; Perelstein, Maxim
 Journal of High Energy Physics, Vol. 2019, Issue 2
Novel jet observables from machine learning
journal, March 2018
 Datta, Kaustuv; Larkoski, Andrew J.
 Journal of High Energy Physics, Vol. 2018, Issue 3
Investigating the topology dependence of quark and gluon jets
journal, March 2019
 BrightThonney, Samuel; Nachman, Benjamin
 Journal of High Energy Physics, Vol. 2019, Issue 3
Energy flow polynomials: a complete linear basis for jet substructure
journal, April 2018
 Komiske, Patrick T.; Metodiev, Eric M.; Thaler, Jesse
 Journal of High Energy Physics, Vol. 2018, Issue 4
Jet angularity measurements for single inclusive jet production
journal, April 2018
 Kang, ZhongBo; Lee, Kyle; Ringer, Felix
 Journal of High Energy Physics, Vol. 2018, Issue 4
Interpretable deep learning for twoprong jet classification with jet spectra
journal, July 2019
 Chakraborty, Amit; Lim, Sung Hak; Nojiri, Mihoko M.
 Journal of High Energy Physics, Vol. 2019, Issue 7
Jet charge and machine learning
journal, October 2018
 Fraser, Katherine; Schwartz, Matthew D.
 Journal of High Energy Physics, Vol. 2018, Issue 10
Boosting H → b b ¯ $$ H\to b\overline{b} $$ with machine learning
journal, October 2018
 Lin, Joshua; Freytsis, Marat; Moult, Ian
 Journal of High Energy Physics, Vol. 2018, Issue 10
Pulling out all the tops with computer vision and deep learning
journal, October 2018
 Macaluso, Sebastian; Shih, David
 Journal of High Energy Physics, Vol. 2018, Issue 10
Adversariallytrained autoencoders for robust unsupervised new physics searches
journal, October 2019
 Blance, Andrew; Spannowsky, Michael; Waite, Philip
 Journal of High Energy Physics, Vol. 2019, Issue 10
The Lund jet plane
journal, December 2018
 Dreyer, Frédéric A.; Salam, Gavin P.; Soyez, Grégory
 Journal of High Energy Physics, Vol. 2018, Issue 12
Identifying the Relevant Dependencies of the Neural Network Response on Characteristics of the Input Space
journal, September 2018
 Wunsch, Stefan; Friese, Raphael; Wolf, Roger
 Computing and Software for Big Science, Vol. 2, Issue 1
Solving differential equations with neural networks: Applications to the calculation of cosmological phase transitions
journal, July 2019
 Piscopo, Maria Laura; Spannowsky, Michael; Waite, Philip
 Physical Review D, Vol. 100, Issue 1
Uncovering latent jet substructure
journal, September 2019
 Dillon, Barry M.; Faroughy, Darius A.; Kamenik, Jernej F.
 Physical Review D, Vol. 100, Issue 5
Automating the construction of jet observables with machine learning
journal, November 2019
 Datta, Kaustuv; Larkoski, Andrew; Nachman, Benjamin
 Physical Review D, Vol. 100, Issue 9
Learning to classify from impure samples with highdimensional data
journal, July 2018
 Komiske, Patrick T.; Metodiev, Eric M.; Nachman, Benjamin
 Physical Review D, Vol. 98, Issue 1
Extending the search for new resonances with machine learning
journal, January 2019
 Collins, Jack H.; Howe, Kiel; Nachman, Benjamin
 Physical Review D, Vol. 99, Issue 1
Anomaly Detection for Resonant New Physics with Machine Learning
journal, December 2018
 Collins, Jack; Howe, Kiel; Nachman, Benjamin
 Physical Review Letters, Vol. 121, Issue 24
binary junipr: An Interpretable Probabilistic Model for Discrimination
journal, October 2019
 Andreassen, Anders; Feige, Ilya; Frye, Christopher
 Physical Review Letters, Vol. 123, Issue 18
Machine learning and the physical sciences
journal, December 2019
 Carleo, Giuseppe; Cirac, Ignacio; Cranmer, Kyle
 Reviews of Modern Physics, Vol. 91, Issue 4
Jet substructure at the Large Hadron Collider
journal, December 2019
 Kogler, Roman; Nachman, Benjamin; Schmidt, Alexander
 Reviews of Modern Physics, Vol. 91, Issue 4
Production of $$\tau \tau jj$$ττjj final states at the LHC and the TauSpinner algorithm: the spin2 case
journal, January 2018
 Bahmani, M.; Kalinowski, J.; Kotlarski, W.
 The European Physical Journal C, Vol. 78, Issue 1
Machine learning uncertainties with adversarial neural networks
journal, January 2019
 Englert, Christoph; Galler, Peter; Harris, Philip
 The European Physical Journal C, Vol. 79, Issue 1
JUNIPR: a framework for unsupervised machine learning in particle physics
journal, February 2019
 Andreassen, Anders; Feige, Ilya; Frye, Christopher
 The European Physical Journal C, Vol. 79, Issue 2
Guiding new physics searches with unsupervised learning
journal, March 2019
 De Simone, Andrea; Jacques, Thomas
 The European Physical Journal C, Vol. 79, Issue 4
QCD or what?
journal, January 2019
 Heimel, Theo; Kasieczka, Gregor; Plehn, Tilman
 SciPost Physics, Vol. 6, Issue 3
Quarkgluon tagging: Machine learning vs detector
journal, January 2019
 Kasieczka, Gregor; Kiefer, Nicholas; Plehn, Tilman
 SciPost Physics, Vol. 6, Issue 6
Deeplearning jets with uncertainties and more
journal, January 2020
 Bollweg, Sven; Haussmann, Manuel; Kasieczka, Gregor
 SciPost Physics, Vol. 8, Issue 1
CapsNets continuing the convolutional quest
journal, January 2020
 Diefenbacher, Sascha; Frost, Hermann; Kasieczka, Gregor
 SciPost Physics, Vol. 8, Issue 2