Secure Federated Learning Across Heterogeneous Cloud and High-Performance Computing Resources: A Case Study on Federated Fine-Tuning of LLaMA 2
- Univ. of Illinois at Urbana-Champaign, IL (United States)
- Argonne National Laboratory (ANL), Argonne, IL (United States)
Federated learning enables multiple data owners to collaboratively train robust machine learning models without transferring large or sensitive local datasets by only sharing the parameters of the locally trained models. Here, in this article, we elaborate on the design of our Advanced Privacy-Preserving Federated Learning (APPFL) framework, which streamlines end-to-end secure and reliable federated learning experiments across cloud computing facilities and high-performance computing resources by leveraging Globus Compute, a distributed function as a service platform, and Amazon Web Services. We further demonstrate the use case of APPFL in fine-tuning an LLaMA 2 7B model using several cloud resources and supercomputers.
- Research Organization:
- Argonne National Laboratory (ANL), Argonne, IL (United States)
- Sponsoring Organization:
- USDOE Office of Science (SC); National Science Foundation (NSF)
- Grant/Contract Number:
- AC02-06CH11357
- OSTI ID:
- 2569854
- Journal Information:
- Computing in Science and Engineering, Journal Name: Computing in Science and Engineering Journal Issue: 3 Vol. 26; ISSN 1521-9615; ISSN 1558-366X
- Publisher:
- IEEE Computer SocietyCopyright Statement
- Country of Publication:
- United States
- Language:
- English
Globus auth: A research identity and access management platform
|
conference | October 2016 |
APPFL: Open-Source Software Framework for Privacy-Preserving Federated Learning
|
conference | May 2022 |
funcX: A Federated Function Serving Fabric for Science
|
conference | June 2020 |
Advances and Open Problems in Federated Learning
|
journal | January 2021 |
Similar Records
Enabling end-to-end secure federated learning in biomedical research on heterogeneous computing environments with APPFLx