A cross-study analysis of drug response prediction in cancer cell lines

Xia, Fangfang; Allen, Jonathan; Balaprakash, Prasanna; Brettin, Thomas; Garcia-Cardona, Cristina; Clyde, Austin; Cohn, Judith; Doroshow, James; Duan, Xiaotian; Dubinkina, Veronika; Evrard, Yvonne; Fan, Ya Ju; Gans, Jason; He, Stewart; Lu, Pinyi; Maslov, Sergei; Partin, Alexander; Shukla, Maulik; Stahlberg, Eric; Wozniak, Justin M.; Yoo, Hyunseung; Zaki, George; Zhu, Yitan; Stevens, Rick

doi:10.1093/bib/bbab356

A cross-study analysis of drug response prediction in cancer cell lines

Journal Article · Tue Sep 14 00:00:00 EDT 2021 · Briefings in Bioinformatics

DOI:https://doi.org/10.1093/bib/bbab356· OSTI ID:1820194

Xia, Fangfang ^[1]; Allen, Jonathan ^[2]; Balaprakash, Prasanna ^[1]; Brettin, Thomas ^[1]; Garcia-Cardona, Cristina ^[3]; Clyde, Austin ^[4]; Cohn, Judith ^[3]; Doroshow, James ^[5]; Duan, Xiaotian ^[6]; Dubinkina, Veronika ^[7]; Evrard, Yvonne ^[8]; Fan, Ya Ju ^[2]; Gans, Jason ^[3]; He, Stewart ^[2]; Lu, Pinyi ^[8]; Maslov, Sergei ^[7]; Partin, Alexander ^[1]; Shukla, Maulik ^[1]; Stahlberg, Eric ^[8]; Wozniak, Justin M. ^[1] more »

Argonne National Laboratory
Lawrence Livermore National Laboratory
Los Alamos National Laboratory
Argonne National Laboratory, University of Chicago
National Cancer Institute
University of Chicago
University of Illinois at Urbana-Champaign
Frederick National Laboratory for Cancer Research

Abstract

To enable personalized cancer treatment, machine learning models have been developed to predict drug response as a function of tumor and drug features. However, most algorithm development efforts have relied on cross-validation within a single study to assess model accuracy. While an essential first step, cross-validation within a biological data set typically provides an overly optimistic estimate of the prediction performance on independent test sets. To provide a more rigorous assessment of model generalizability between different studies, we use machine learning to analyze five publicly available cell line-based data sets: National Cancer Institute 60, ancer Therapeutics Response Portal (CTRP), Genomics of Drug Sensitivity in Cancer, Cancer Cell Line Encyclopedia and Genentech Cell Line Screening Initiative (gCSI). Based on observed experimental variability across studies, we explore estimates of prediction upper bounds. We report performance results of a variety of machine learning models, with a multitasking deep neural network achieving the best cross-study generalizability. By multiple measures, models trained on CTRP yield the most accurate predictions on the remaining testing data, and gCSI is the most predictable among the cell line data sets included in this study. With these experiments and further simulations on partial data, two lessons emerge: (1) differences in viability assays can limit model generalizability across studies and (2) drug diversity, more than tumor diversity, is crucial for raising model generalizability in preclinical screening.

View Journal Article

Sponsoring Organization:: USDOE

Grant/Contract Number:: AC02-06CH11357; AC52-07NA27344; AC52-06NA25396; AC05-00OR22725

OSTI ID:: 1820194

Alternate ID(s):: OSTI ID: 1869533
OSTI ID: 1880129

Journal Information:: Briefings in Bioinformatics, Journal Name: Briefings in Bioinformatics Journal Issue: 1 Vol. 23; ISSN 1467-5463

Publisher:: Oxford University PressCopyright Statement

Country of Publication:: United Kingdom

Language:: English

References (48)

The art and practice of structure-based drug design: A molecular modeling perspective Bohacek, Regine S.; McMartin, Colin; Guida, Wayne C. Medicinal Research Reviews, Vol. 16, Issue 1 https://doi.org/10.1002/(SICI)1098-1128(199601)16:1<3::AID-MED1>3.0.CO;2-6	journal	January 1996
Machine Learning for Cancer Drug Combination Wang, Ziyan; Li, Hongyang; Guan, Yuanfang Clinical Pharmacology & Therapeutics, Vol. 107, Issue 4 https://doi.org/10.1002/cpt.1773	journal	February 2020
Machine learning and feature selection for drug response prediction in precision oncology applications Ali, Mehreen; Aittokallio, Tero Biophysical Reviews, Vol. 11, Issue 1 https://doi.org/10.1007/s12551-018-0446-z	journal	August 2018
An Interactive Resource to Identify Cancer Genetic and Lineage Dependencies Targeted by Small Molecules Basu, Amrita; Bodycombe, Nicole E.; Cheah, Jaime H. Cell, Vol. 154, Issue 5 https://doi.org/10.1016/j.cell.2013.08.003	journal	August 2013
Deep-Resp-Forest: A deep forest model to predict anti-cancer drug response Su, Ran; Liu, Xinyi; Wei, Leyi Methods, Vol. 166 https://doi.org/10.1016/j.ymeth.2019.02.009	journal	August 2019
Random Forests Breiman, Leo Machine Learning, Vol. 45, Issue 1, p. 5-32 https://doi.org/10.1023/A:1010933404324	journal	January 2001
The Cancer Cell Line Encyclopedia enables predictive modelling of anticancer drug sensitivity Barretina, Jordi; Caponigro, Giordano; Stransky, Nicolas Nature, Vol. 483, Issue 7391 https://doi.org/10.1038/nature11003	journal	March 2012
Inconsistency in large pharmacogenomic studies Haibe-Kains, Benjamin; El-Hachem, Nehme; Birkbak, Nicolai Juul Nature, Vol. 504, Issue 7480 https://doi.org/10.1038/nature12831	journal	November 2013
Reproducible pharmacogenomic profiling of cancer cell line panels Haverty, Peter M.; Lin, Eva; Tan, Jenille Nature, Vol. 533, Issue 7603 https://doi.org/10.1038/nature17987	journal	May 2016
Consistency in drug response profiling Mpindi, John Patrick; Yadav, Bhagwan; Östling, Päivi Nature, Vol. 540, Issue 7631 https://doi.org/10.1038/nature20171	journal	November 2016
Quantifying the chemical beauty of drugs Bickerton, G. Richard; Paolini, Gaia V.; Besnard, Jérémy Nature Chemistry, Vol. 4, Issue 2 https://doi.org/10.1038/nchem.1243	journal	January 2012
Large-scale gene function analysis with the PANTHER classification system Mi, Huaiyu; Muruganujan, Anushya; Casagrande, John T. Nature Protocols, Vol. 8, Issue 8 https://doi.org/10.1038/nprot.2013.092	journal	July 2013
The NCI60 human tumour cell line anticancer drug screen Shoemaker, Robert H. Nature Reviews Cancer, Vol. 6, Issue 10 https://doi.org/10.1038/nrc1951	journal	October 2006
Community assessment to advance computational prediction of cancer drug combinations in a pharmacogenomic screen Menden, Michael P.; Wang, Dennis; Mason, Mike J. Nature Communications, Vol. 10, Issue 1 https://doi.org/10.1038/s41467-019-09799-2	journal	June 2019
Representation of features as images with neighborhood dependencies for compatibility with convolutional neural networks Bazgir, Omid; Zhang, Ruibo; Dhruba, Saugato Rahman Nature Communications, Vol. 11, Issue 1 https://doi.org/10.1038/s41467-020-18197-y	journal	September 2020
Drug ranking using machine learning systematically predicts the efficacy of anti-cancer drugs Gerdes, Henry; Casado, Pedro; Dokal, Arran Nature Communications, Vol. 12, Issue 1 https://doi.org/10.1038/s41467-021-22170-8	journal	March 2021
Next-generation characterization of the Cancer Cell Line Encyclopedia Ghandi, Mahmoud; Huang, Franklin W.; Jané-Valbuena, Judit Nature, Vol. 569, Issue 7757 https://doi.org/10.1038/s41586-019-1186-3	journal	May 2019
New insight for pharmacogenomics studies from the transcriptional analysis of two large-scale cancer cell line panels Sadacca, Benjamin; Hamy, Anne-Sophie; Laurent, Cécile Scientific Reports, Vol. 7, Issue 1 https://doi.org/10.1038/s41598-017-14770-6	journal	November 2017
Ensemble transfer learning for the prediction of anti-cancer drug response Zhu, Yitan; Brettin, Thomas; Evrard, Yvonne A. Scientific Reports, Vol. 10, Issue 1 https://doi.org/10.1038/s41598-020-74921-0	journal	October 2020
A normalized drug response metric improves accuracy and consistency of anticancer drug sensitivity quantification in cell-based screening Gupta, Abhishekh; Gautam, Prson; Wennerberg, Krister Communications Biology, Vol. 3, Issue 1 https://doi.org/10.1038/s42003-020-0765-z	journal	January 2020
Quantitative scoring of differential drug sensitivity for individually optimized anticancer therapies Yadav, Bhagwan; Pemovska, Tea; Szwajda, Agnieszka Scientific Reports, Vol. 4, Issue 1 https://doi.org/10.1038/srep05193	journal	June 2014
Effect of normalization methods on the performance of supervised learning algorithms applied to HTSeq-FPKM-UQ data sets: 7SK RNA expression as a predictor of survival in patients with colon adenocarcinoma Shahriyari, Leili Briefings in Bioinformatics, Vol. 20, Issue 3 https://doi.org/10.1093/bib/bbx153	journal	November 2017
Comparison and evaluation of integrative methods for the analysis of multilevel omics data: a study based on simulated and experimental cancer data Pucher, Bettina M.; Zeleznik, Oana A.; Thallinger, Gerhard G. Briefings in Bioinformatics, Vol. 20, Issue 2 https://doi.org/10.1093/bib/bby027	journal	April 2018
Evaluating the consistency of large-scale pharmacogenomic studies Rahman, Raziur; Dhruba, Saugato Rahman; Matlock, Kevin Briefings in Bioinformatics, Vol. 20, Issue 5 https://doi.org/10.1093/bib/bby046	journal	May 2019
Meta-GDBP: a high-level stacked regression model to improve anticancer drug response prediction Su, Ran; Liu, Xinyi; Xiao, Guobao Briefings in Bioinformatics, Vol. 21, Issue 3 https://doi.org/10.1093/bib/bbz022	journal	March 2019
Improving drug response prediction by integrating multiple data sources: matrix factorization, kernel and network-based approaches Güvenç Paltun, Betül; Mamitsuka, Hiroshi; Kaski, Samuel Briefings in Bioinformatics, Vol. 22, Issue 1 https://doi.org/10.1093/bib/bbz153	journal	December 2019
A survey and systematic assessment of computational methods for drug response prediction Chen, Jinyu; Zhang, Louxin Briefings in Bioinformatics, Vol. 22, Issue 1 https://doi.org/10.1093/bib/bbz164	journal	January 2020
Deep learning for drug response prediction in cancer Baptista, Delora; Ferreira, Pedro G.; Rocha, Miguel Briefings in Bioinformatics https://doi.org/10.1093/bib/bbz171	journal	January 2020
Predicting Cancer Drug Response using a Recommender System Suphavilai, Chayaporn; Bertrand, Denis; Nagarajan, Niranjan Bioinformatics, Vol. 34, Issue 22 https://doi.org/10.1093/bioinformatics/bty452	journal	June 2018
Dr.VAE: improving drug response prediction via modeling of drug perturbation effects Rampášek, Ladislav; Hidru, Daniel; Smirnov, Petr Bioinformatics, Vol. 35, Issue 19 https://doi.org/10.1093/bioinformatics/btz158	journal	March 2019
deepDR: a network-based deep learning approach to in silico drug repositioning Zeng, Xiangxiang; Zhu, Siyi; Liu, Xiangrong Bioinformatics, Vol. 35, Issue 24 https://doi.org/10.1093/bioinformatics/btz418	journal	May 2019
ChEMBL: a large-scale bioactivity database for drug discovery Gaulton, A.; Bellis, L. J.; Bento, A. P. Nucleic Acids Research, Vol. 40, Issue D1 https://doi.org/10.1093/nar/gkr777	journal	September 2011
Genomics of Drug Sensitivity in Cancer (GDSC): a resource for therapeutic biomarker discovery in cancer cells Yang, Wanjuan; Soares, Jorge; Greninger, Patricia Nucleic Acids Research, Vol. 41, Issue D1 https://doi.org/10.1093/nar/gks1111	journal	November 2012
BindingDB in 2015: A public database for medicinal chemistry, computational chemistry and systems pharmacology Gilson, Michael K.; Liu, Tiqing; Baitaluk, Michael Nucleic Acids Research, Vol. 44, Issue D1 https://doi.org/10.1093/nar/gkv1072	journal	October 2015
Data Portal for the Library of Integrated Network-based Cellular Signatures (LINCS) program: integrated access to diverse large-scale cellular perturbation response data Koleti, Amar; Terryn, Raymond; Stathias, Vasileios Nucleic Acids Research, Vol. 46, Issue D1 https://doi.org/10.1093/nar/gkx1063	journal	November 2017
PharmacoDB: an integrative database for mining in vitro anticancer drug screening studies Smirnov, Petr; Kofia, Victor; Maru, Alexander Nucleic Acids Research, Vol. 46, Issue D1 https://doi.org/10.1093/nar/gkx911	journal	October 2017
PubChem 2019 update: improved access to chemical data Kim, Sunghwan; Chen, Jie; Cheng, Tiejun Nucleic Acids Research, Vol. 47, Issue D1 https://doi.org/10.1093/nar/gky1033	journal	October 2018
Q-Rank: Reinforcement Learning for Recommending Algorithms to Predict Drug Sensitivity to Cancer Therapy Daoud, Salma; Mdhaffar, Afef; Jmaiel, Mohamed IEEE Journal of Biomedical and Health Informatics, Vol. 24, Issue 11 https://doi.org/10.1109/JBHI.2020.3004663	journal	November 2020
Precision Oncology beyond Targeted Therapy: Combining Omics Data with Machine Learning Matches the Majority of Cancer Cells to Effective Therapeutics Ding, Michael Q.; Chen, Lujia; Cooper, Gregory F. Molecular Cancer Research, Vol. 16, Issue 2 https://doi.org/10.1158/1541-7786.MCR-17-0378	journal	November 2017
Harnessing Connectivity in a Large-Scale Small-Molecule Sensitivity Dataset Seashore-Ludlow, Brinton; Rees, Matthew G.; Cheah, Jaime H. Cancer Discovery, Vol. 5, Issue 11 https://doi.org/10.1158/2159-8290.CD-15-0235	journal	October 2015
The NCI Genomic Data Commons as an engine for precision medicine Jensen, Mark A.; Ferretti, Vincent; Grossman, Robert L. Blood, Vol. 130, Issue 4 https://doi.org/10.1182/blood-2017-03-735654	journal	July 2017
Predicting tumor cell line response to drug pairs with deep learning Xia, Fangfang; Shukla, Maulik; Brettin, Thomas BMC Bioinformatics, Vol. 19, Issue S18 https://doi.org/10.1186/s12859-018-2509-3	journal	December 2018
Improving prediction of phenotypic drug response on cancer cell lines using deep convolutional network Liu, Pengfei; Li, Hongjian; Li, Shuai BMC Bioinformatics, Vol. 20, Issue 1 https://doi.org/10.1186/s12859-019-2910-6	journal	July 2019
Revisiting inconsistency in large pharmacogenomic studies Safikhani, Zhaleh; Smirnov, Petr; Freeman, Mark F1000Research, Vol. 5 https://doi.org/10.12688/f1000research.9611.1	journal	January 2016
Open source machine-learning algorithms for the prediction of optimal cancer drug therapies Huang, Cai; Mezencev, Roman; McDonald, John F. PLOS ONE, Vol. 12, Issue 10 https://doi.org/10.1371/journal.pone.0186906	journal	October 2017
The cornucopia of meaningful leads: Applying deep adversarial autoencoders for new molecule development in oncology Kadurin, Artur; Aliper, Alexander; Kazennov, Andrey Oncotarget, Vol. 8, Issue 7 https://doi.org/10.18632/oncotarget.14073	journal	December 2016
Predicting Synergism of Cancer Drug Combinations Using NCI-ALMANAC Data Sidorov, Pavel; Naulaerts, Stefan; Ariey-Bonnet, Jérémy Frontiers in Chemistry, Vol. 7 https://doi.org/10.3389/fchem.2019.00509	journal	July 2019
Paclitaxel Response Can Be Predicted With Interpretable Multi-Variate Classifiers Exploiting DNA-Methylation and miRNA Data Bomane, Alexandra; Gonçalves, Anthony; Ballester, Pedro J. Frontiers in Genetics, Vol. 10 https://doi.org/10.3389/fgene.2019.01041	journal	October 2019

Similar Records

RNA-seq latent featurizer using center loss cost function

Software · Tue Mar 19 20:00:00 EDT 2019 · OSTI ID:code-63718

A cross-study analysis of drug response prediction in cancer cell lines

Citation Formats

References (48)

Similar Records

Related Subjects