Node-degree aware edge sampling mitigates inflated classification performance in biomedical random walk-based graph representation learning

Journal Article · Sun Mar 03 23:00:00 EST 2024 · Bioinformatics Advances

DOI:https://doi.org/10.1093/bioadv/vbae036· OSTI ID:2375469

Cappelletti, Luca ^[1]; Rekerle, Lauren ^[2]; Fontana, Tommaso ^[1]; Hansen, Peter ^[2]; Casiraghi, Elena ^[3]; Ravanmehr, Vida ^[2]; Mungall, Christopher J. ^[4]; Yang, Jeremy J. ^[5]; Spranger, Leonard ^[6]; Karlebach, Guy ^[2]; Caufield, J. Harry ^[4]; Carmody, Leigh ^[2]; Coleman, Ben ^[7]; Oprea, Tudor I. ^[8]; Reese, Justin ^[4]; Valentini, Giorgio ^[9]; Robinson, Peter N. ^[10]

---

1. Univ. degli Studi di Milano (Italy)
2. Jackson Laboratory for Genomic Medicine, Farmington, CT (United States)
3. Univ. degli Studi di Milano (Italy); Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)
4. Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)
5. Univ. of New Mexico, Albuquerque, NM (United States).
6. Freie Univ., Berlin (Germany)
7. Jackson Laboratory for Genomic Medicine, Farmington, CT (United States); Univ. of Connecticut, Farmington, CT (United States)
8. Univ. of New Mexico, Albuquerque, NM (United States)
9. Univ. degli Studi di Milano (Italy); European Laboratory for Learning and Intelligent Systems (ELLIS) (Europe)
10. Jackson Laboratory for Genomic Medicine, Farmington, CT (United States); Univ. of Connecticut, Farmington, CT (United States); European Laboratory for Learning and Intelligent Systems (ELLIS) (Europe); Universitätsmedizin Berlin (Germany)

Motivation: Graph representation learning is a family of related approaches that learn low-dimensional vector representations of nodes and other graph elements called embeddings. Embeddings approximate characteristics of the graph and can be used for a variety of machine-learning tasks such as novel edge prediction. For many biomedical applications, partial knowledge exists about positive edges that represent relationships between pairs of entities, but little to no knowledge is available about negative edges that represent the explicit lack of a relationship between two nodes. For this reason, classification procedures are forced to assume that the vast majority of unlabeled edges are negative. Existing approaches to sampling negative edges for training and evaluating classifiers do so by uniformly sampling pairs of nodes. Results: We show here that this sampling strategy typically leads to sets of positive and negative examples with imbalanced node degree distributions. Using representative heterogeneous biomedical knowledge graph and random walk-based graph machine learning, we show that this strategy substantially impacts classification performance. If users of graph machine-learning models apply the models to prioritize examples that are drawn from approximately the same distribution as the positive examples are, then performance of models as estimated in the validation phase may be artificially inflated. We present a degree-aware node sampling approach that mitigates this effect and is simple to implement. Availability and implementation: Our code and data are publicly available at https://github.com/monarch-initiative/negativeExampleSelection.

• Research Organization: Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)
• Sponsoring Organization: USDOE Office of Science (SC), Biological and Environmental Research (BER); National Institutes of Health (NIH); National Cancer Institute (NCI)
• Grant/Contract Number: AC02-05CH11231
• OSTI ID: 2375469
• Journal Information: Bioinformatics Advances, Journal Name: Bioinformatics Advances Journal Issue: 1 Vol. 4; ISSN 2635-0041
• Publisher: Oxford University Press
• Country of Publication: United States
• Language: English

Related Subjects

59 BASIC BIOLOGICAL SCIENCES