Fed-DeepONet: Stochastic Gradient-Based Federated Training of Deep Operator Networks

Moya, Christian; Lin, Guang

doi:10.3390/a15090325

Fed-DeepONet: Stochastic Gradient-Based Federated Training of Deep Operator Networks

Journal Article · Mon Sep 12 00:00:00 EDT 2022 · Algorithms

DOI:https://doi.org/10.3390/a15090325· OSTI ID:1886960

Moya, Christian;

The Deep Operator Network (DeepONet) framework is a different class of neural network architecture that one trains to learn nonlinear operators, i.e., mappings between infinite-dimensional spaces. Traditionally, DeepONets are trained using a centralized strategy that requires transferring the training data to a centralized location. Such a strategy, however, limits our ability to secure data privacy or use high-performance distributed/parallel computing platforms. To alleviate such limitations, in this paper, we study the federated training of DeepONets for the first time. That is, we develop a framework, which we refer to as Fed-DeepONet, that allows multiple clients to train DeepONets collaboratively under the coordination of a centralized server. To achieve Fed-DeepONets, we propose an efficient stochastic gradient-based algorithm that enables the distributed optimization of the DeepONet parameters by averaging first-order estimates of the DeepONet loss gradient. Then, to accelerate the training convergence of Fed-DeepONets, we propose a moment-enhanced (i.e., adaptive) stochastic gradient-based strategy. Finally, we verify the performance of Fed-DeepONet by learning, for different configurations of the number of clients and fractions of available clients, (i) the solution operator of a gravity pendulum and (ii) the dynamic response of a parametric library of pendulums.

Sponsoring Organization:: USDOE

Grant/Contract Number:: SC0021142

OSTI ID:: 1886960

Journal Information:: Algorithms, Journal Name: Algorithms Journal Issue: 9 Vol. 15; ISSN 1999-4893; ISSN ALGOCH

Publisher:: MDPI AGCopyright Statement

Country of Publication:: Switzerland

Language:: English

References (11)

Sparse Identification of Nonlinear Dynamics with Control (SINDYc)**SLB acknowledges support from the U.S. Air Force Center of Excellence on Nature Inspired Flight Technologies and Ideas (FA9550-14-1-0398). JLP thanks Bill and Melinda Gates for their active support of the Institute of Disease Modeling and their sponsorship through the Global Good Fund. JNK acknowledges support from the U.S. Air Force Office of Scientific Research (FA9550-09-0174). Brunton, Steven L.; Proctor, Joshua L.; Kutz, J. Nathan IFAC-PapersOnLine, Vol. 49, Issue 18 https://doi.org/10.1016/j.ifacol.2016.10.249	journal	January 2016
Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations Raissi, M.; Perdikaris, P.; Karniadakis, G. E. Journal of Computational Physics, Vol. 378 https://doi.org/10.1016/j.jcp.2018.10.045	journal	February 2019
Data driven governing equations approximation using deep neural networks Qin, Tong; Wu, Kailiang; Xiu, Dongbin Journal of Computational Physics, Vol. 395 https://doi.org/10.1016/j.jcp.2019.06.042	journal	October 2019
DeepM&Mnet: Inferring the electroconvection multiphysics fields based on operator approximation by neural networks Cai, Shengze; Wang, Zhicheng; Lu, Lu Journal of Computational Physics, Vol. 436 https://doi.org/10.1016/j.jcp.2021.110296	journal	July 2021
DeepONet-grid-UQ: A trustworthy deep operator framework for predicting the power grid’s post-fault trajectories Moya, Christian; Zhang, Shiqi; Lin, Guang Neurocomputing, Vol. 535 https://doi.org/10.1016/j.neucom.2023.03.015	journal	May 2023
Deep learning LeCun, Yann; Bengio, Yoshua; Hinton, Geoffrey Nature, Vol. 521, Issue 7553 https://doi.org/10.1038/nature14539	journal	May 2015
Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators Lu, Lu; Jin, Pengzhan; Pang, Guofei Nature Machine Intelligence, Vol. 3, Issue 3 https://doi.org/10.1038/s42256-021-00302-5	journal	March 2021
Discovering governing equations from data by sparse identification of nonlinear dynamical systems Brunton, Steven L.; Proctor, Joshua L.; Kutz, J. Nathan Proceedings of the National Academy of Sciences, Vol. 113, Issue 15 https://doi.org/10.1073/pnas.1517384113	journal	March 2016
Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems No authors listed IEEE Transactions on Neural Networks, Vol. 6, Issue 4 https://doi.org/10.1109/72.392253	journal	July 1995
Digital Twin in the IoT Context: A Survey on Technical Features, Scenarios, and Architectural Models Minerva, Roberto; Lee, Gyu Myoung; Crespi, Noel Proceedings of the IEEE, Vol. 108, Issue 10 https://doi.org/10.1109/JPROC.2020.2998530	journal	October 2020
Data-Driven Learning of Nonautonomous Systems Qin, Tong; Chen, Zhen; Jakeman, John D. SIAM Journal on Scientific Computing, Vol. 43, Issue 3 https://doi.org/10.1137/20M1342859	journal	January 2021

Similar Records

B-DeepONet: An enhanced Bayesian DeepONet for solving noisy parametric PDEs using accelerated replica exchange SGLD

Journal Article · Thu Oct 13 20:00:00 EDT 2022 · Journal of Computational Physics · OSTI ID:2421766

DeepONet-grid-UQ: A trustworthy deep operator framework for predicting the power grid’s post-fault trajectories

Journal Article · Wed Mar 15 20:00:00 EDT 2023 · Neurocomputing · OSTI ID:2382633

FedADMP: A Joint Anomaly Detection and Mobility Prediction Framework via Federated Learning

Journal Article · Wed Oct 20 20:00:00 EDT 2021 · ICST Transaction on Security and Safety · OSTI ID:1835237

Fed-DeepONet: Stochastic Gradient-Based Federated Training of Deep Operator Networks

Citation Formats

References (11)

Similar Records

Related Subjects