An ensemble model of QSAR tools for regulatory risk assessment
Abstract
Quantitative structure activity relationships (QSARs) are theoretical models that relate a quantitative measure of chemical structure to a physical property or a biological effect. QSAR predictions can be used for chemical risk assessment for protection of human and environmental health, which makes them interesting to regulators, especially in the absence of experimental data. For compatibility with regulatory use, QSAR models should be transparent, reproducible and optimized to minimize the number of false negatives. In silico QSAR tools are gaining wide acceptance as a faster alternative to otherwise time-consuming clinical and animal testing methods. However, different QSAR tools often make conflicting predictions for a given chemical and may also vary in their predictive performance across different chemical datasets. In a regulatory context, conflicting predictions raise interpretation, validation and adequacy concerns. To address these concerns, ensemble learning techniques in the machine learning paradigm can be used to integrate predictions from multiple tools. By leveraging various underlying QSAR algorithms and training datasets, the resulting consensus prediction should yield better overall predictive ability. We present a novel ensemble QSAR model using Bayesian classification. The model allows for varying a cut-off parameter that allows for a selection in the desirable trade-off between model sensitivity andmore »
- Authors:
-
[1];
- National Center for Computational Toxicology (ORISE Fellow), Research Triangle Park, NC (United States)
- Marquette Univ., Milwaukee, WI (United States)
- Georgetown Univ. Medical Center, Washington, D.C. (United States)
- Publication Date:
- Research Org.:
- Oak Ridge Institute for Science and Education (ORISE), Oak Ridge, TN (United States)
- Sponsoring Org.:
- USDOE
- OSTI Identifier:
- 1375955
- Resource Type:
- Accepted Manuscript
- Journal Name:
- Journal of Cheminformatics
- Additional Journal Information:
- Journal Volume: 8; Journal Issue: 1; Journal ID: ISSN 1758-2946
- Publisher:
- Chemistry Central Ltd.
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CHEMISTRY; 97 MATHEMATICS AND COMPUTING; Computational toxicology; In silico QSAR tools; Hybrid QSAR models; Ensemble models; Risk assessment
Citation Formats
Pradeep, Prachi, Povinelli, Richard J., White, Shannon, and Merrill, Stephen J. An ensemble model of QSAR tools for regulatory risk assessment. United States: N. p., 2016.
Web. doi:10.1186/s13321-016-0164-0.
Pradeep, Prachi, Povinelli, Richard J., White, Shannon, & Merrill, Stephen J. An ensemble model of QSAR tools for regulatory risk assessment. United States. https://doi.org/10.1186/s13321-016-0164-0
Pradeep, Prachi, Povinelli, Richard J., White, Shannon, and Merrill, Stephen J. Thu .
"An ensemble model of QSAR tools for regulatory risk assessment". United States. https://doi.org/10.1186/s13321-016-0164-0. https://www.osti.gov/servlets/purl/1375955.
@article{osti_1375955,
title = {An ensemble model of QSAR tools for regulatory risk assessment},
author = {Pradeep, Prachi and Povinelli, Richard J. and White, Shannon and Merrill, Stephen J.},
abstractNote = {Quantitative structure activity relationships (QSARs) are theoretical models that relate a quantitative measure of chemical structure to a physical property or a biological effect. QSAR predictions can be used for chemical risk assessment for protection of human and environmental health, which makes them interesting to regulators, especially in the absence of experimental data. For compatibility with regulatory use, QSAR models should be transparent, reproducible and optimized to minimize the number of false negatives. In silico QSAR tools are gaining wide acceptance as a faster alternative to otherwise time-consuming clinical and animal testing methods. However, different QSAR tools often make conflicting predictions for a given chemical and may also vary in their predictive performance across different chemical datasets. In a regulatory context, conflicting predictions raise interpretation, validation and adequacy concerns. To address these concerns, ensemble learning techniques in the machine learning paradigm can be used to integrate predictions from multiple tools. By leveraging various underlying QSAR algorithms and training datasets, the resulting consensus prediction should yield better overall predictive ability. We present a novel ensemble QSAR model using Bayesian classification. The model allows for varying a cut-off parameter that allows for a selection in the desirable trade-off between model sensitivity and specificity. The predictive performance of the ensemble model is compared with four in silico tools (Toxtree, Lazar, OECD Toolbox, and Danish QSAR) to predict carcinogenicity for a dataset of air toxins (332 chemicals) and a subset of the gold carcinogenic potency database (480 chemicals). Leave-one-out cross validation results show that the ensemble model achieves the best trade-off between sensitivity and specificity (accuracy: 83.8 % and 80.4 %, and balanced accuracy: 80.6 % and 80.8 %) and highest inter-rater agreement [kappa (κ): 0.63 and 0.62] for both the datasets. The ROC curves demonstrate the utility of the cut-off feature in the predictive ability of the ensemble model. In conclusion, this feature provides an additional control to the regulators in grading a chemical based on the severity of the toxic endpoint under study.},
doi = {10.1186/s13321-016-0164-0},
journal = {Journal of Cheminformatics},
number = 1,
volume = 8,
place = {United States},
year = {Thu Sep 22 00:00:00 EDT 2016},
month = {Thu Sep 22 00:00:00 EDT 2016}
}
Free Publicly Available Full Text
Publisher's Version of Record
Other availability
Search WorldCat to find libraries that may hold this journal
Cited by: 29 works
Citation information provided by
Web of Science
Web of Science
Figures / Tables:
Fig. 1: Bayesian classifier ensemble for predicting carcinogenicity. The posterior probability, Pk, as determined from Table 1 is compared with a variable cut-off between 0 and 1.
All figures and tables
(5 total)
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.
Works referenced in this record:
Computer‐assisted analysis of interlaboratory Ames test variability
journal, January 1988
- Benigni, R.; Giuliani, A.
- Journal of Toxicology and Environmental Health, Vol. 25, Issue 1
The Challenges Involved in Modeling Toxicity Data In Silico: A Review
journal, March 2012
- Gleeson, M. Paul; Modi, Sandeep; Bender, Andreas
- Current Pharmaceutical Design, Vol. 18, Issue 9
Classifier ensembles: Select real-world applications
journal, January 2008
- Oza, Nikunj C.; Tumer, Kagan
- Information Fusion, Vol. 9, Issue 1
Combined Use of MC4PC, MDL-QSAR, BioEpisteme, Leadscope PDM, and Derek for Windows Software to Achieve High-Performance, High-Confidence, Mode of Action–Based Predictions of Chemical Carcinogenesis in Rodents
journal, January 2008
- Matthews, Edwin J.; Kruhlak, Naomi L.; Benz, R. Daniel
- Toxicology Mechanisms and Methods, Vol. 18, Issue 2-3
Interpretable, Probability-Based Confidence Metric for Continuous Quantitative Structure–Activity Relationship Models
journal, February 2013
- Keefer, Christopher E.; Kauffman, Gregory W.; Gupta, Rishi Raj
- Journal of Chemical Information and Modeling, Vol. 53, Issue 2
Animal testing and alternative approaches for the human health risk assessment under the proposed new European chemicals regulation
journal, May 2004
- H�fer, Thomas; Gerner, Ingrid; Gundert-Remy, Ursula
- Archives of Toxicology, Vol. 78, Issue 10
Use of QSARs in international decision-making frameworks to predict health effects of chemical substances.
journal, August 2003
- Cronin, Mark T. D.; Jaworska, Joanna S.; Walker, John D.
- Environmental Health Perspectives, Vol. 111, Issue 10
Evaluation of model predictive ability by external validation techniques
journal, February 2010
- Consonni, Viviana; Ballabio, Davide; Todeschini, Roberto
- Journal of Chemometrics, Vol. 24, Issue 3-4
Boosting: An Ensemble Learning Tool for Compound Classification and QSAR Modeling
journal, May 2005
- Svetnik, Vladimir; Wang, Ting; Tong, Christopher
- Journal of Chemical Information and Modeling, Vol. 45, Issue 3
A Coefficient of Agreement for Nominal Scales
journal, April 1960
- Cohen, Jacob
- Educational and Psychological Measurement, Vol. 20, Issue 1
Receiver-Operating Characteristic Analysis for Evaluating Diagnostic Tests and Predictive Models
journal, February 2007
- Zou, Kelly H.; O’Malley, A. James; Mauri, Laura
- Circulation, Vol. 115, Issue 5
Toxicokinetics as a key to the integrated toxicity risk assessment based primarily on non-animal approaches
journal, August 2013
- Coecke, Sandra; Pelkonen, Olavi; Leite, Sofia Batista
- Toxicology in Vitro, Vol. 27, Issue 5
U.S. EPA Regulatory Perspectives on the Use of QSAR for New and Existing Chemical Evaluations
journal, September 1995
- Zeeman, M.; Auer, C. M.; Clements, R. G.
- SAR and QSAR in Environmental Research, Vol. 3, Issue 3
Ensemble QSAR: A QSAR method based on conformational ensembles and metric descriptors
journal, April 2011
- Pissurlenkar, Raghuvir R. S.; Khedkar, Vijay M.; Iyer, Radhakrishnan P.
- Journal of Computational Chemistry, Vol. 32, Issue 10
The application of discovery toxicology and pathology towards the design of safer pharmaceutical lead candidates
journal, August 2007
- Kramer, Jeffrey A.; Sagartz, John E.; Morris, Dale L.
- Nature Reviews Drug Discovery, Vol. 6, Issue 8
In silico toxicology models and databases as FDA Critical Path Initiative toolkits
journal, January 2011
- Valerio, Luis G.
- Human Genomics, Vol. 5, Issue 3
Summary of a workshop on regulatory acceptance of (Q)SARs for human health and environmental endpoints.
journal, August 2003
- Jaworska, Joanna S.; Comber, M.; Auer, C.
- Environmental Health Perspectives, Vol. 111, Issue 10
Computer Prediction of Possible Toxic Action from Chemical Structure; The DEREK System
journal, July 1991
- Sanderson, D. M.; Earnshaw, C. G.
- Human & Experimental Toxicology, Vol. 10, Issue 4
Comparative Evaluation of in Silico Systems for Ames Test Mutagenicity Prediction: Scope and Limitations
journal, June 2011
- Hillebrecht, Alexander; Muster, Wolfgang; Brigo, Alessandro
- Chemical Research in Toxicology, Vol. 24, Issue 6
Comparison of MC4PC and MDL-QSAR rodent carcinogenicity predictions and the enhancement of predictive performance by combining QSAR models
journal, December 2007
- Contrera, Joseph F.; Kruhlak, Naomi L.; Matthews, Edwin J.
- Regulatory Toxicology and Pharmacology, Vol. 49, Issue 3
Integration of QSAR models for bioconcentration suitable for REACH
journal, July 2013
- Gissi, Andrea; Nicolotti, Orazio; Carotti, Angelo
- Science of The Total Environment, Vol. 456-457
Computational toxicology in drug development
journal, April 2008
- Muster, Wolfgang; Breidenbach, Alexander; Fischer, Holger
- Drug Discovery Today, Vol. 13, Issue 7-8
A new hybrid system of QSAR models for predicting bioconcentration factors (BCF)
journal, December 2008
- Zhao, Chunyan; Boriani, Elena; Chana, Antonio
- Chemosphere, Vol. 73, Issue 11
Real External Predictivity of QSAR Models: How To Evaluate It? Comparison of Different Validation Criteria and Proposal of Using the Concordance Correlation Coefficient
journal, August 2011
- Chirico, Nicola; Gramatica, Paola
- Journal of Chemical Information and Modeling, Vol. 51, Issue 9
A weighted voting framework for classifiers ensembles
journal, December 2012
- Kuncheva, Ludmila I.; Rodríguez, Juan J.
- Knowledge and Information Systems, Vol. 38, Issue 2
Methods for detecting carcinogens and mutagens with the salmonella/mammalian-microsome mutagenicity test
journal, December 1975
- Ames, Bruce N.; McCann, Joyce; Yamasaki, Edith
- Mutation Research/Environmental Mutagenesis and Related Subjects, Vol. 31, Issue 6
LeadScope † : Software for Exploring Large Sets of Screening Data
journal, November 2000
- Roberts, Gulsevin; Myatt, Glenn J.; Johnson, Wayne P.
- Journal of Chemical Information and Computer Sciences, Vol. 40, Issue 6
BioEpisteme®—An in silico approach for predicting and understanding the underlying molecular mechanisms contributing to toxicity responses
journal, July 2010
- Valencia, A.
- Toxicology Letters, Vol. 196
The Role of Qsar Methodology in the Regulatory Assessment of Chemicals
book, October 2009
- Worth, Andrew Paul
- Recent Advances in QSAR Studies. Challenges and Advances in Computational Chemistry and Physics
The Challenges Involved in Modeling Toxicity Data In Silico: A Review
journal, March 2012
- Gleeson, M. Paul; Modi, Sandeep; Bender, Andreas
- Current Drug Metabolism, Vol. 18, Issue 9
Evaluation of model predictive ability by external validation techniques
journal, February 2010
- Consonni, Viviana; Ballabio, Davide; Todeschini, Roberto
- Journal of Chemometrics, Vol. 24, Issue 3-4
Ensemble QSAR: A QSAR method based on conformational ensembles and metric descriptors
journal, April 2011
- Pissurlenkar, Raghuvir R. S.; Khedkar, Vijay M.; Iyer, Radhakrishnan P.
- Journal of Computational Chemistry, Vol. 32, Issue 10
QSAR analysis of metal ion toxicity data in sunflower callus cultures (Helianthus annuus ?Sunspot?)
journal, April 2003
- Enache, Monica; Dearden, John?C.; Walker, John?D.
- QSAR & Combinatorial Science, Vol. 22, Issue 2
Animal testing and alternative approaches for the human health risk assessment under the proposed new European chemicals regulation
journal, May 2004
- H�fer, Thomas; Gerner, Ingrid; Gundert-Remy, Ursula
- Archives of Toxicology, Vol. 78, Issue 10
A weighted voting framework for classifiers ensembles
journal, December 2012
- Kuncheva, Ludmila I.; Rodríguez, Juan J.
- Knowledge and Information Systems, Vol. 38, Issue 2
Methods for detecting carcinogens and mutagens with the salmonella/mammalian-microsome mutagenicity test
journal, December 1975
- Ames, Bruce N.; McCann, Joyce; Yamasaki, Edith
- Mutation Research/Environmental Mutagenesis and Related Subjects, Vol. 31, Issue 6
A new hybrid system of QSAR models for predicting bioconcentration factors (BCF)
journal, December 2008
- Zhao, Chunyan; Boriani, Elena; Chana, Antonio
- Chemosphere, Vol. 73, Issue 11
Computational toxicology in drug development
journal, April 2008
- Muster, Wolfgang; Breidenbach, Alexander; Fischer, Holger
- Drug Discovery Today, Vol. 13, Issue 7-8
Classifier ensembles: Select real-world applications
journal, January 2008
- Oza, Nikunj C.; Tumer, Kagan
- Information Fusion, Vol. 9, Issue 1
Integration of QSAR models for bioconcentration suitable for REACH
journal, July 2013
- Gissi, Andrea; Nicolotti, Orazio; Carotti, Angelo
- Science of The Total Environment, Vol. 456-457
Toxicokinetics as a key to the integrated toxicity risk assessment based primarily on non-animal approaches
journal, August 2013
- Coecke, Sandra; Pelkonen, Olavi; Leite, Sofia Batista
- Toxicology in Vitro, Vol. 27, Issue 5
Comparison of MC4PC and MDL-QSAR rodent carcinogenicity predictions and the enhancement of predictive performance by combining QSAR models
journal, December 2007
- Contrera, Joseph F.; Kruhlak, Naomi L.; Matthews, Edwin J.
- Regulatory Toxicology and Pharmacology, Vol. 49, Issue 3
Boosting: An Ensemble Learning Tool for Compound Classification and QSAR Modeling
journal, May 2005
- Svetnik, Vladimir; Wang, Ting; Tong, Christopher
- Journal of Chemical Information and Modeling, Vol. 45, Issue 3
Interpretable, Probability-Based Confidence Metric for Continuous Quantitative Structure–Activity Relationship Models
journal, February 2013
- Keefer, Christopher E.; Kauffman, Gregory W.; Gupta, Rishi Raj
- Journal of Chemical Information and Modeling, Vol. 53, Issue 2
Comparative Evaluation of in Silico Systems for Ames Test Mutagenicity Prediction: Scope and Limitations
journal, June 2011
- Hillebrecht, Alexander; Muster, Wolfgang; Brigo, Alessandro
- Chemical Research in Toxicology, Vol. 24, Issue 6
The application of discovery toxicology and pathology towards the design of safer pharmaceutical lead candidates
journal, August 2007
- Kramer, Jeffrey A.; Sagartz, John E.; Morris, Dale L.
- Nature Reviews Drug Discovery, Vol. 6, Issue 8
U.S. EPA Regulatory Perspectives on the Use of QSAR for New and Existing Chemical Evaluations
journal, September 1995
- Zeeman, M.; Auer, C. M.; Clements, R. G.
- SAR and QSAR in Environmental Research, Vol. 3, Issue 3
Computer‐assisted analysis of interlaboratory Ames test variability
journal, January 1988
- Benigni, R.; Giuliani, A.
- Journal of Toxicology and Environmental Health, Vol. 25, Issue 1
Combined Use of MC4PC, MDL-QSAR, BioEpisteme, Leadscope PDM, and Derek for Windows Software to Achieve High-Performance, High-Confidence, Mode of Action–Based Predictions of Chemical Carcinogenesis in Rodents
journal, January 2008
- Matthews, Edwin J.; Kruhlak, Naomi L.; Benz, R. Daniel
- Toxicology Mechanisms and Methods, Vol. 18, Issue 2-3
Medical Device Development: From Prototype to Regulatory Approval
journal, June 2004
- Kaplan, Aaron V.; Baim, Donald S.; Smith, John J.
- Circulation, Vol. 109, Issue 25
QSAR Applicability Domain Estimation by Projection of the Training Set in Descriptor Space: A Review
journal, October 2005
- Jaworska, Joanna; Nikolova-Jeliazkova, Nina; Aldenberg, Tom
- Alternatives to Laboratory Animals, Vol. 33, Issue 5
Computer Prediction of Possible Toxic Action from Chemical Structure; The DEREK System
journal, July 1991
- Sanderson, D. M.; Earnshaw, C. G.
- Human & Experimental Toxicology, Vol. 10, Issue 4
In silico toxicology models and databases as FDA Critical Path Initiative toolkits
journal, January 2011
- Valerio, Luis G.
- Human Genomics, Vol. 5, Issue 3
Summary of a workshop on regulatory acceptance of (Q)SARs for human health and environmental endpoints.
journal, August 2003
- Jaworska, Joanna S.; Comber, M.; Auer, C.
- Environmental Health Perspectives, Vol. 111, Issue 10
Use of QSARs in international decision-making frameworks to predict health effects of chemical substances.
journal, August 2003
- Cronin, Mark T. D.; Jaworska, Joanna S.; Walker, John D.
- Environmental Health Perspectives, Vol. 111, Issue 10
Proposal and multicentric validation of a laparoscopic Roux-en-Y gastric bypass surgery ontology.
text, January 2022
- Lavanchy, Joël Lukas; Gonzalez, Cristians; Kassem, Hasan
- Springer-Verlag
Works referencing / citing this record:
Mixtures of QSAR models: Learning application domains of p K predicto rs
journal, April 2020
- Dörgő, Gyula; Péter Hamadi, Omar; Varga, Tamás
- Journal of Chemometrics, Vol. 34, Issue 4
QSAR classification and regression models for β-secretase inhibitors using relative distance matrices
journal, March 2018
- Luque Ruiz, I.; Gómez-Nieto, M. Á.
- SAR and QSAR in Environmental Research, Vol. 29, Issue 5
QSAR/QSPR models based on quantum chemistry for risk assessment of pesticides according to current European legislation
journal, November 2019
- Villaverde, J. J.; Sevilla-Morán, B.; López-Goti, C.
- SAR and QSAR in Environmental Research, Vol. 31, Issue 1
Decision tree models to classify nanomaterials according to the DF4nanoGrouping scheme
journal, December 2017
- Gajewicz, Agnieszka; Puzyn, Tomasz; Odziomek, Katarzyna
- Nanotoxicology, Vol. 12, Issue 1
Comprehensive ensemble in QSAR prediction for drug discovery
journal, October 2019
- Kwon, Sunyoung; Bae, Ho; Jo, Jeonghee
- BMC Bioinformatics, Vol. 20, Issue 1
Decision tree models to classify nanomaterials according to the DF4nanoGrouping scheme
journal, December 2017
- Gajewicz, Agnieszka; Puzyn, Tomasz; Odziomek, Katarzyna
- Nanotoxicology, Vol. 12, Issue 1
Figures / Tables found in this record:
Figures/Tables have been extracted from DOE-funded journal article accepted manuscripts.