DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Attention

Certain functionality on DOE PAGES may be limited from July 10th, 8 PM EDT through July 13th, 6 AM EDT. We apologize for any inconvenience.

Title: Comparing Individualized Survival Predictions From Random Survival Forests and Multistate Models in the Presence of Missing Data: A Case Study of Patients With Oropharyngeal Cancer

Journal Article · · Cancer Informatics
ORCiD logo [1];  [2];  [1];  [3];  [4];  [1]
  1. Department of Biostatistics, University of Michigan, Ann Arbor, MI, USA
  2. Department of Biostatistics, University of Michigan, Ann Arbor, MI, USA, Information Systems & Modeling, Los Alamos National Laboratory, Los Alamos, NM, USA
  3. Department of Otolaryngology, University of Michigan, Ann Arbor, MI, USA
  4. Department of Oncology, Georgetown University School of Medicine, Washington, DC, USA

Background: In recent years, interest in prognostic calculators for predicting patient health outcomes has grown with the popularity of personalized medicine. These calculators, which can inform treatment decisions, employ many different methods, each of which has advantages and disadvantages. Methods: We present a comparison of a multistate model (MSM) and a random survival forest (RSF) through a case study of prognostic predictions for patients with oropharyngeal squamous cell carcinoma. The MSM is highly structured and takes into account some aspects of the clinical context and knowledge about oropharyngeal cancer, while the RSF can be thought of as a black-box non-parametric approach. Key in this comparison are the high rate of missing values within these data and the different approaches used by the MSM and RSF to handle missingness. Results: We compare the accuracy (discrimination and calibration) of survival probabilities predicted by both approaches and use simulation studies to better understand how predictive accuracy is influenced by the approach to (1) handling missing data and (2) modeling structural/disease progression information present in the data. We conclude that both approaches have similar predictive accuracy, with a slight advantage going to the MSM. Conclusions: Although the MSM shows slightly better predictive ability than the RSF, consideration of other differences are key when selecting the best approach for addressing a specific research question. These key differences include the methods’ ability to incorporate domain knowledge, and their ability to handle missing data as well as their interpretability, and ease of implementation. Ultimately, selecting the statistical method that has the most potential to aid in clinical decisions requires thoughtful consideration of the specific goals.

Research Organization:
Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
Sponsoring Organization:
National Institutes of Health (NIH); USDOE; USDOE Laboratory Directed Research and Development (LDRD) Program
Grant/Contract Number:
89233218CNA000001
OSTI ID:
1987897
Report Number(s):
LA-UR--22-24202; 11769351231183847
Journal Information:
Cancer Informatics, Journal Name: Cancer Informatics Vol. 22; ISSN 1176-9351
Publisher:
SAGE PublicationsCopyright Statement
Country of Publication:
United States
Language:
English

References (32)

Individual and Population Comparisons of Surgery and Radiotherapy Outcomes in Prostate Cancer Using Bayesian Multistate Models journal February 2019
Development and Assessment of a Model for Predicting Individualized Outcomes in Patients With Oropharyngeal Cancer journal August 2021
Interleukin-6 predicts recurrence and survival among head and neck cancer patients journal August 2008
Individualized survival prediction for patients with oropharyngeal cancer in the human papillomavirus era journal October 2018
Cigarette use, comorbidities, and prognosis in a prospective head and neck squamous cell carcinoma population journal July 2016
Impact of American Joint Committee on Cancer Eighth Edition clinical stage and smoking history on oncologic outcomes in human papillomavirus‐associated oropharyngeal squamous cell carcinoma journal February 2019
Tutorial in biostatistics: competing risks and multi-state models journal January 2007
Imputing missing covariate values for the Cox model journal July 2009
Multiple imputation of missing covariates for the Cox proportional hazards cure model journal July 2016
Modeling Survival Data: Extending the Cox Model book January 2000
Missing data imputation using statistical and machine learning methods in a real breast cancer problem journal October 2010
The mstate package for estimation and prediction in non- and semi-parametric multi-state and competing risks models journal September 2010
Prognostication of metastatic death in uveal melanoma patients: A Markov multi-state model journal November 2018
Evaluation of statistical and machine learning models for time series prediction: Identifying the state-of-the-art and the best conditions for the use of each model journal May 2019
Missing covariate data in medical research: To impute is better than to ignore journal July 2010
Comparison of multistate model, survival regression, and matched case–control methods for estimating excess length of stay due to healthcare-associated infections journal August 2022
Machine learning methods and predictive ability metrics for genome-wide prediction of complex traits journal August 2014
Prediction and behavioral analysis of travel mode choice: A comparison of machine learning and logit models journal July 2020
Personalized Risk Prediction in Clinical Oncology Research: Applications and Practical Issues Using Survival Trees and Random Forests journal October 2017
Benchmarking missing-values approaches for predictive models on health databases journal April 2022
A Multistate Model Predicting Mortality, Length of Stay, and Readmission for Surgical Patients journal October 2015
A Review of Hot Deck Imputation for Survey Non-response journal April 2010
Multiple imputation of covariates by fully conditional specification: Accommodating the substantive model journal February 2014
Multiple imputation for cause-specific Cox models: Assessing methods for estimation and prediction journal June 2022
A comparison of two frameworks for multi-state modelling, applied to outcomes after hospital admissions with COVID-19 journal July 2022
Interpreting the concordance statistic of a logistic regression model: relation to the variance and odds ratio of a continuous explanatory variable journal June 2012
Nonparametric prediction of event times in randomized clinical trials journal August 2004
Random survival forests journal September 2008
mstate: AnRPackage for the Analysis of Competing Risks and Multi-State Models journal January 2011
mice : Multivariate Imputation by Chained Equations in R journal January 2011
Predicting 10-Year Risk of End-Organ Complications of Type 2 Diabetes With and Without Metabolic Surgery: A Machine Learning Approach journal February 2020
A Grey-Box Ensemble Model Exploiting Black-Box Accuracy and White-Box Intrinsic Interpretability journal January 2020