Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

Adversarial Training for Privacy-Preserving Deep Learning Model Distribution

Conference ·
 [1];  [1];  [2];  [3];  [4];  [4];  [1]
  1. ORNL
  2. LSUHSC-Louisiana Tumor Registry
  3. University of Kentucky
  4. National Cancer Institute, Bethesda, MD
+ Show Author Affiliations

Collaboration among cancer registries is essential to develop accurate, robust, and generalizable deep learning models for automated information extraction from cancer pathology reports. Sharing data presents a serious privacy issue, especially in biomedical research and healthcare delivery domains. Distributing pretrained deep learning (DL) models has been proposed to avoid critical data sharing. However, there is growing recognition that collaboration among clinical institutes through DL model distribution exposes new security and privacy vulnerabilities. These vulnerabilities increase in natural language processing (NLP) applications, in which the dataset vocabulary with word vector representations needs to be associated with the other model parameters. In this paper, we propose a novel privacy-preserving DL model distribution across cancer registries for information extraction from cancer pathology reports with privacy and confidentiality considerations. The proposed approach exploits the adversarial training framework to distinguish private features from shared features among different datasets. It only shares registry-invariant model parameters, without sharing raw data nor registry-specific model parameters among cancer registries. Thus, it protects both the data and the trained model simultaneously. We compare our proposed approach to single-registry models, and a model trained on centrally hosted data from different cancer registries. The results show that the proposed approach significantly outperforms the single-registry models and achieves statistically indistinguishable micro and macro F1-score as compared to the centralized model.

Research Organization:
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
Sponsoring Organization:
USDOE
DOE Contract Number:
AC05-00OR22725
OSTI ID:
1606810
Country of Publication:
United States
Language:
English

References (14)

Distributed deep learning networks among institutions for medical imaging journal March 2018
De-identification of psychiatric intake records: Overview of 2016 CEGS N-GRID shared tasks Track 1 journal November 2017
Deep Learning for Automated Extraction of Primary Sites From Cancer Pathology Reports journal January 2018
Deep Learning with Differential Privacy
  • Abadi, Martin; Chu, Andy; Goodfellow, Ian
  • CCS'16: 2016 ACM SIGSAC Conference on Computer and Communications Security, Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security https://doi.org/10.1145/2976749.2978318
conference October 2016
Deep Transfer Learning Across Cancer Registries for Information Extraction from Pathology Reports conference May 2019
Classifying cancer pathology reports with hierarchical self-attention networks journal November 2019
Privacy-Preserving Deep Learning via Additively Homomorphic Encryption journal May 2018
A privacy-preserving distributed filtering framework for NLP artifacts journal September 2019
Coarse-to-fine multi-task training of convolutional neural networks for automated information extraction from cancer pathology reports conference March 2018
Deep Models Under the GAN conference October 2017
Convolutional Neural Networks for Sentence Classification conference January 2014
De-identification of patient notes with recurrent neural networks journal December 2016
Classifying medical relations in clinical text via convolutional neural networks journal January 2019
Adversarial Multi-task Learning for Text Classification
  • Liu, Pengfei; Qiu, Xipeng; Huang, Xuanjing
  • Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) https://doi.org/10.18653/v1/P17-1001
conference January 2017

Similar Records

Optimal vocabulary selection approaches for privacy-preserving deep NLP model training for information extraction and cancer epidemiology
Journal Article · Sun Feb 13 23:00:00 EST 2022 · Cancer Biomarkers · OSTI ID:1855683
Privacy-Preserving Knowledge Transfer with Bootstrap Aggregation of Teacher Ensembles
Conference · Sun Feb 28 23:00:00 EST 2021 · OSTI ID:1771902

Related Subjects