Overestimated prediction using polygenic prediction derived from summary statistics

Park, David Keetae; Chen, Mingshen; Kim, Seungsoo; Joo, Yoonjung Yoonie; Loving, Rebekah K.; Kim, Hyoung Seop; Cha, Jiook; Yoo, Shinjae; Kim, Jong Hun

doi:10.1186/s12863-023-01151-4

Title: Overestimated prediction using polygenic prediction derived from summary statistics

Journal Article · 14 September 2023 · BMC Genomic Data (Online)

DOI: https://doi.org/10.1186/s12863-023-01151-4 · OSTI ID:2006815

Park, David Keetae ^[1]; Chen, Mingshen ^[2]; Kim, Seungsoo ^[1]; Joo, Yoonjung Yoonie ^[3]; Loving, Rebekah K. ^[4]; Kim, Hyoung Seop ^[5]; Cha, Jiook ^[6]; Yoo, Shinjae ^[7]; Kim, Jong Hun ^[5]

Columbia Univ., New York, NY (United States)
Stony Brook Univ., NY (United States)
Sungkyunkwan University, Seoul (South Korea). Samsung Medical Center, Samsung Advanced Institute for Health Sciences & Technology (SAHIST)
California Institute of Technology (CalTech), Pasadena, CA (United States)
National Health Insurance Service Ilsan Hospital, Goyang (South Korea). Dementia Center, Department of Physical Medicine and Rehabilitation
Seoul National Univ. (Korea, Republic of). Brain and Cognitive Sciences, AI Institute
Brookhaven National Laboratory (BNL), Upton, NY (United States)

When polygenic risk score (PRS) is derived from summary statistics, independence between discovery and test sets cannot be monitored. We compared two types of PRS studies derived from raw genetic data (denoted as rPRS) and the summary statistics for IGAP (sPRS). Two variables with the high heritability in UK Biobank, hypertension, and height, are used to derive an exemplary scale effect of PRS. sPRS without APOE is derived from International Genomics of Alzheimer’s Project (IGAP), which records ΔAUC and ΔR² of 0.051 ± 0.013 and 0.063 ± 0.015 for Alzheimer’s Disease Sequencing Project (ADSP) and 0.060 and 0.086 for Accelerating Medicine Partnership - Alzheimer’s Disease (AMP-AD). On UK Biobank, rPRS performances for hypertension assuming a similar size of discovery and test sets are 0.0036 ± 0.0027 (ΔAUC) and 0.0032 ± 0.0028 (ΔR²). For height, ΔR² is 0.029 ± 0.0037. Considering the high heritability of hypertension and height of UK Biobank and sample size of UK Biobank, sPRS results from AD databases are inflated. Independence between discovery and test sets is a well-known basic requirement for PRS studies. However, a lot of PRS studies cannot follow such requirements because of impossible direct comparisons when using summary statistics. Thus, for sPRS, potential duplications should be carefully considered within the same ethnic group.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Brookhaven National Laboratory (BNL), Upton, NY (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Advanced Scientific Computing Research

Grant/Contract Number:: SC0012704

OSTI ID:: 2006815

Report Number(s):: BNL-224815-2023-JAAM

Journal Information:: BMC Genomic Data (Online), Vol. 24, Issue 1; ISSN 2730-6844

Publisher:: BioMed CentralCopyright Statement

Country of Publication:: United States

Language:: English

References (46)

GCTA: A Tool for Genome-wide Complex Trait Analysis Yang, Jian; Lee, S. Hong; Goddard, Michael E. The American Journal of Human Genetics, Vol. 88, Issue 1 https://doi.org/10.1016/j.ajhg.2010.11.011	journal	January 2011
A multi-omic atlas of the human frontal cortex for aging and Alzheimer’s disease research De Jager, Philip L.; Ma, Yiyi; McCabe, Cristin Scientific Data, Vol. 5, Issue 1 https://doi.org/10.1038/sdata.2018.142	journal	August 2018
Second-generation PLINK: rising to the challenge of larger and richer datasets Chang, Christopher C.; Chow, Carson C.; Tellier, Laurent CAM GigaScience, Vol. 4, Issue 1 https://doi.org/10.1186/s13742-015-0047-8	journal	February 2015
Improving reporting standards for polygenic scores in risk prediction studies Wand, Hannah; Lambert, Samuel A.; Tamburro, Cecelia Nature, Vol. 591, Issue 7849 https://doi.org/10.1038/s41586-021-03243-6	journal	March 2021
The Mount Sinai cohort of large-scale genomic, transcriptomic and proteomic data in Alzheimer's disease Wang, Minghui; Beckmann, Noam D.; Roussos, Panos Scientific Data, Vol. 5, Issue 1 https://doi.org/10.1038/sdata.2018.185	journal	September 2018
Improving the trans-ancestry portability of polygenic risk scores by prioritizing variants in predicted cell-type-specific regulatory elements Amariuta, Tiffany; Ishigaki, Kazuyoshi; Sugishita, Hiroki Nature Genetics, Vol. 52, Issue 12 https://doi.org/10.1038/s41588-020-00740-8	journal	November 2020
A note on a general definition of the coefficient of determination Nagelkerke, N. J. D. Biometrika, Vol. 78, Issue 3 https://doi.org/10.1093/biomet/78.3.691	journal	January 1991
Common polygenic variation enhances risk prediction for Alzheimer’s disease Escott-Price, Valentina; Sims, Rebecca; Bannister, Christian Brain, Vol. 138, Issue 12 https://doi.org/10.1093/brain/awv268	journal	October 2015
Cross-cancer evaluation of polygenic risk scores for 16 cancer types in two large cohorts Graff, Rebecca E.; Cavazos, Taylor B.; Thai, Khanh K. Nature Communications, Vol. 12, Issue 1 https://doi.org/10.1038/s41467-021-21288-z	journal	February 2021
Fast Principal-Component Analysis Reveals Convergent Evolution of ADH1B in Europe and East Asia Galinsky, Kevin J.; Bhatia, Gaurav; Loh, Po-Ru The American Journal of Human Genetics, Vol. 98, Issue 3 https://doi.org/10.1016/j.ajhg.2015.12.022	journal	March 2016
From Polygenic Scores to Precision Medicine in Alzheimer’s Disease: A Systematic Review Harrison, Judith R.; Mistry, Sumit; Muskett, Natalie Journal of Alzheimer's Disease, Vol. 74, Issue 4 https://doi.org/10.3233/JAD-191233	journal	April 2020
Predictive Accuracy of a Polygenic Risk Score–Enhanced Prediction Model vs a Clinical Risk Score for Coronary Artery Disease Elliott, Joshua; Bodinier, Barbara; Bond, Tom A. JAMA, Vol. 323, Issue 7 https://doi.org/10.1001/jama.2019.22241	journal	February 2020
The Polygenic Score Catalog as an open database for reproducibility and systematic evaluation Lambert, Samuel A.; Gil, Laurent; Jupp, Simon Nature Genetics, Vol. 53, Issue 4 https://doi.org/10.1038/s41588-021-00783-5	journal	March 2021
EraSOR: a software tool to eliminate inflation caused by sample overlap in polygenic score analyses Choi, Shing Wan; Mak, Timothy Shin Heng; Hoggart, Clive J. GigaScience, Vol. 12 https://doi.org/10.1093/gigascience/giad043	journal	December 2022
The UK Biobank resource with deep phenotyping and genomic data Bycroft, Clare; Freeman, Colin; Petkova, Desislava Nature, Vol. 562, Issue 7726 https://doi.org/10.1038/s41586-018-0579-z	journal	October 2018
Polygenic and clinical risk scores and their impact on age at onset and prediction of cardiometabolic diseases and common cancers Mars, Nina; Koskela, Jukka T. Nature Medicine, Vol. 26, Issue 4 https://doi.org/10.1038/s41591-020-0800-0	journal	April 2020
Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach DeLong, Elizabeth R.; DeLong, David M.; Clarke-Pearson, Daniel L. Biometrics, Vol. 44, Issue 3 https://doi.org/10.2307/2531595	journal	September 1988
Validation of a polygenic risk score for dementia in black and white individuals Marden, Jessica R.; Walter, Stefan; Tchetgen Tchetgen, Eric J. Brain and Behavior, Vol. 4, Issue 5 https://doi.org/10.1002/brb3.248	journal	July 2014
Principles for the post-GWAS functional characterization of cancer risk loci Freedman, Matthew L.; Monteiro, Alvaro N. A.; Gayther, Simon A. Nature Genetics, Vol. 43, Issue 6 https://doi.org/10.1038/ng.840	journal	May 2011
Genome-wide polygenic scores for common diseases identify individuals with risk equivalent to monogenic mutations Khera, Amit V.; Chaffin, Mark; Aragam, Krishna G. Nature Genetics, Vol. 50, Issue 9 https://doi.org/10.1038/s41588-018-0183-z	journal	August 2018
Development and Evaluation of a Genetic Risk Score for Obesity Belsky, Daniel W.; Moffitt, Terrie E.; Sugden, Karen Biodemography and Social Biology, Vol. 59, Issue 1 https://doi.org/10.1080/19485565.2013.774628	journal	January 2013
The AD Knowledge Portal: A Repository for Multi‐Omic Data on Alzheimer's Disease and Aging Greenwood, Anna K.; Montgomery, Kelsey S.; Kauer, Nicole Current Protocols in Human Genetics, Vol. 108, Issue 1 https://doi.org/10.1002/cphg.105	journal	October 2020
PRSice-2: Polygenic Risk Score software for biobank-scale data Choi, Shing Wan; O'Reilly, Paul F. GigaScience, Vol. 8, Issue 7 https://doi.org/10.1093/gigascience/giz082	journal	July 2019
Population-specific and trans-ancestry genome-wide analyses identify distinct and shared genetic risk loci for coronary artery disease Koyama, Satoshi; Ito, Kaoru; Terao, Chikashi Nature Genetics, Vol. 52, Issue 11 https://doi.org/10.1038/s41588-020-0705-3	journal	October 2020
Alzheimer's Disease Sequencing Project discovery and replication criteria for cases and controls: Data from a community‐based prospective cohort study with autopsy follow‐up Crane, Paul K.; Foroud, Tatiana; Montine, Thomas J. Alzheimer's & Dementia, Vol. 13, Issue 12 https://doi.org/10.1016/j.jalz.2017.09.010	journal	October 2017
Beyond SNP heritability: Polygenicity and discoverability of phenotypes estimated with a univariate Gaussian mixture model Holland, Dominic; Frei, Oleksandr; Desikan, Rahul PLOS Genetics, Vol. 16, Issue 5 https://doi.org/10.1371/journal.pgen.1008612	journal	May 2020
Polygenic scores via penalized regression on summary statistics Mak, Timothy Shin Heng; Porsch, Robert Milan; Choi, Shing Wan Genetic Epidemiology, Vol. 41, Issue 6 https://doi.org/10.1002/gepi.22050	journal	May 2017
Polygenic Scores for Height in Admixed Populations Bitarello, Bárbara D.; Mathieson, Iain G3 Genes\|Genomes\|Genetics, Vol. 10, Issue 11 https://doi.org/10.1534/g3.120.401658	journal	November 2020
Assessment of Claims of Improved Prediction Beyond the Framingham Risk Score Tzoulaki, Ioanna JAMA, Vol. 302, Issue 21 https://doi.org/10.1001/jama.2009.1757	journal	December 2009
Clinical use of current polygenic risk scores may exacerbate health disparities Martin, Alicia R.; Kanai, Masahiro; Kamatani, Yoichiro Nature Genetics, Vol. 51, Issue 4 https://doi.org/10.1038/s41588-019-0379-x	journal	March 2019
Analysis of polygenic risk score usage and performance in diverse human populations Duncan, L.; Shen, H.; Gelaye, B. Nature Communications, Vol. 10, Issue 1 https://doi.org/10.1038/s41467-019-11112-0	journal	July 2019
Polygenic Risk Predicts Obesity in Both White and Black Young Adults Domingue, Benjamin W.; Belsky, Daniel W.; Harris, Kathleen Mullan PLoS ONE, Vol. 9, Issue 7 https://doi.org/10.1371/journal.pone.0101596	journal	July 2014
PRSice: Polygenic Risk Score software Euesden, Jack; Lewis, Cathryn M.; O’Reilly, Paul F. Bioinformatics, Vol. 31, Issue 9 https://doi.org/10.1093/bioinformatics/btu848	journal	December 2014
The Alzheimer's Disease Sequencing Project: Study design and sample selection Beecham, Gary W.; Bis, J. C.; Martin, E. R. Neurology Genetics, Vol. 3, Issue 5 https://doi.org/10.1212/NXG.0000000000000194	journal	October 2017
Genetic Determinants of “Cognitive Impairment, No Dementia” Dubé, Joseph B.; Johansen, Christopher T.; Robinson, John F. Journal of Alzheimer's Disease, Vol. 33, Issue 3 https://doi.org/10.3233/JAD-2012-121477	journal	January 2013
pROC: an open-source package for R and S+ to analyze and compare ROC curves Robin, Xavier; Turck, Natacha; Hainard, Alexandre BMC Bioinformatics, Vol. 12, Issue 1 https://doi.org/10.1186/1471-2105-12-77	journal	March 2011
The multiplex model of the genetics of Alzheimer’s disease Sims, Rebecca; Hill, Matthew; Williams, Julie Nature Neuroscience, Vol. 23, Issue 3 https://doi.org/10.1038/s41593-020-0599-5	journal	February 2020
Human whole genome genotype and transcriptome data for Alzheimer’s and other neurodegenerative diseases Allen, Mariet; Carrasquillo, Minerva M.; Funk, Cory Scientific Data, Vol. 3, Issue 1 https://doi.org/10.1038/sdata.2016.89	journal	October 2016
Meta-analysis of 74,046 individuals identifies 11 new susceptibility loci for Alzheimer's disease Lambert, Jean-Charles; Ibrahim-Verbaas, Carla A.; Harold, Denise Nature Genetics, Vol. 45, Issue 12 https://doi.org/10.1038/ng.2802	journal	October 2013
Heterogeneity in polygenic scores for common human traits Ware, Erin B.; Schmitz, Lauren L.; Faul, Jessica https://doi.org/10.1101/106062	posted_content	February 2017
An atlas of genetic associations in UK Biobank Canela-Xandri, Oriol; Rawlik, Konrad; Tenesa, Albert Nature Genetics, Vol. 50, Issue 11 https://doi.org/10.1038/s41588-018-0248-z	journal	October 2018
An Alzheimer’s Disease Genetic Risk Score Predicts Longitudinal Thinning of Hippocampal Complex Subregions in Healthy Older Adults Harrison, Theresa M.; Mahmood, Zanjbeel; Lau, Edward P. eneuro, Vol. 3, Issue 3 https://doi.org/10.1523/ENEURO.0098-16.2016	journal	May 2016
Power and Predictive Accuracy of Polygenic Risk Scores Dudbridge, Frank PLoS Genetics, Vol. 9, Issue 3 https://doi.org/10.1371/journal.pgen.1003348	journal	March 2013
A Type 1 Diabetes Genetic Risk Score Can Aid Discrimination Between Type 1 and Type 2 Diabetes in Young Adults Oram, Richard A.; Patel, Kashyap; Hill, Anita Diabetes Care, Vol. 39, Issue 3 https://doi.org/10.2337/dc15-1111	journal	November 2015
Predictive Utility of Polygenic Risk Scores for Coronary Heart Disease in Three Major Racial and Ethnic Groups Dikilitas, Ozan; Schaid, Daniel J.; Kosel, Matthew L. The American Journal of Human Genetics, Vol. 106, Issue 5 https://doi.org/10.1016/j.ajhg.2020.04.002	journal	May 2020
Tutorial: a guide to performing polygenic risk score analyses Choi, Shing Wan; Mak, Timothy Shin-Heng; O’Reilly, Paul F. Nature Protocols, Vol. 15, Issue 9 https://doi.org/10.1038/s41596-020-0353-1	journal	July 2020

Similar Records

Genetics of varicose veins reveals polygenic architecture and genetic overlap with arterial and venous disease

Journal Article · 2023 · Nature Cardiovascular Research · OSTI ID:1909145

Levin, Michael G.; Huffman, Jennifer E.; Verma, Anurag; +19 more

Learning epistatic polygenic phenotypes with Boolean interactions

Journal Article · 2024 · PLoS ONE · OSTI ID:2470817

Behr, Merle; Kumbier, Karl; Cordova-Palomera, Aldo; +10 more

Quantile-Specific Heritability of Inflammatory and Oxidative Stress Biomarkers Linked to Cardiovascular Disease

Journal Article · 2022 · Journal of Inflammation Research · OSTI ID:2470789

Williams, Paul T.

Related Subjects

97 MATHEMATICS AND COMPUTING
Polygenic risk score
Complex genetic disease
Alzheimer’s disease
Overestimation bias

Title: Overestimated prediction using polygenic prediction derived from summary statistics

Citation Formats

References (46)

Similar Records

Related Subjects