Securely Aggregated Coded Matrix Inversion

Charalambides, Neophytos; Pilanci, Mert; Hero, Alfred O.

doi:10.1109/jsait.2023.3312233

Title: Securely Aggregated Coded Matrix Inversion

Journal Article · 08 September 2023 · IEEE Journal on Selected Areas in Information Theory

DOI: https://doi.org/10.1109/jsait.2023.3312233 · OSTI ID:2283180

^[1];

^[2];

^[3]

University of Michigan, Ann Arbor, MI (United States); University of Michigan
Stanford University, CA (United States)
University of Michigan, Ann Arbor, MI (United States)

Coded computing is a method for mitigating straggling workers in a centralized computing network, by using erasure-coding techniques. Federated learning is a decentralized model for training data distributed across client devices. In this work we propose approximating the inverse of an aggregated data matrix, where the data is generated by clients; similar to the federated learning paradigm, while also being resilient to stragglers. To do so, we propose a coded computing method based on gradient coding. We modify this method so that the coordinator does not access the local data at any point; while the clients access the aggregated matrix in order to complete their tasks. Here, the network we consider is not centrally administrated, and the communications which take place are secure against potential eavesdroppers.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Georgia Institute of Technology, Atlanta, GA (United States)

Sponsoring Organization:: USDOE

Grant/Contract Number:: NA0003921

OSTI ID:: 2283180

Journal Information:: IEEE Journal on Selected Areas in Information Theory, Journal Name: IEEE Journal on Selected Areas in Information Theory Vol. 4; ISSN 2641-8770

Publisher:: IEEECopyright Statement

Country of Publication:: United States

Language:: English

References (36)

Convex Optimization Boyd, Stephen; Vandenberghe, Lieven https://doi.org/10.1017/CBO9780511804441	book	January 2004
Sequential Lagrange multiplier condition forϵ-optimal solution in convex programming Bai, Fusheng; Wu, Zhiyou; Zhu, Daoli Optimization, Vol. 57, Issue 5 https://doi.org/10.1080/02331930802355234	journal	October 2008
Lagrange Coded Computing with Sparsity Constraints Fahim, Mohammad; Cadambe, Viveck R. 2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton) https://doi.org/10.1109/ALLERTON.2019.8919966	conference	September 2019
Secure Linear MDS Coded Matrix Inversion Charalambides, Neophytos; Pilanci, Mert; Hero, Alfred O. 2022 58th Annual Allerton Conference on Communication, Control, and Computing (Allerton) https://doi.org/10.1109/Allerton49937.2022.9929386	conference	September 2022
Approximate Gaussian Elimination for Laplacians - Fast, Sparse, and Simple Kyng, Rasmus; Sachdeva, Sushant 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS) https://doi.org/10.1109/FOCS.2016.68	conference	October 2016
Coded Federated Learning Dhakal, Sagar; Prakash, Saurav; Yona, Yair 2019 IEEE Globecom Workshops (GC Wkshps) https://doi.org/10.1109/GCWkshps45667.2019.9024521	conference	December 2019
Balanced Sparsest generator matrices for MDS codes Dau, Son Hoang; Song, Wentu; Dong, Zheng 2013 IEEE International Symposium on Information Theory https://doi.org/10.1109/ISIT.2013.6620554	conference	July 2013
Balanced Reed-Solomon codes Halbawi, Wael; Liu, Zihan; Hassibi, Babak 2016 IEEE International Symposium on Information Theory (ISIT) https://doi.org/10.1109/ISIT.2016.7541436	conference	July 2016
Encoded distributed optimization Karakus, Can; Sun, Yifan; Diggavi, Suhas 2017 IEEE International Symposium on Information Theory (ISIT) https://doi.org/10.1109/ISIT.2017.8007058	conference	June 2017
Improving Distributed Gradient Descent Using Reed-Solomon Codes Halbawi, Wael; Azizan, Navid; Salehi, Fariborz 2018 IEEE International Symposium on Information Theory (ISIT) https://doi.org/10.1109/ISIT.2018.8437467	conference	June 2018
Coded Federated Computing in Wireless Networks with Straggling Devices and Imperfect CSI Ha, Sukjong; Zhang, Jingjing; Simeone, Osvaldo 2019 IEEE International Symposium on Information Theory (ISIT) https://doi.org/10.1109/ISIT.2019.8849507	conference	July 2019
Product Lagrange Coded Computing Subramaniam, Adarsh M.; Heidarzadeh, Anoosheh; Pradhan, Asit Kumar 2020 IEEE International Symposium on Information Theory (ISIT) https://doi.org/10.1109/ISIT44484.2020.9174440	conference	June 2020
Numerically Stable Binary Gradient Coding Charalambides, Neophytos; Mahdavifar, Hessam; Hero, Alfred O. 2020 IEEE International Symposium on Information Theory (ISIT) https://doi.org/10.1109/ISIT44484.2020.9174512	conference	June 2020
Orthonormal Sketches for Secure Coded Regression Charalambides, Neophytos; Mahdavifar, Hessam; Pilanci, Mert 2022 IEEE International Symposium on Information Theory (ISIT) https://doi.org/10.1109/ISIT50566.2022.9834356	conference	June 2022
Generalized Lagrange Coded Computing: A Flexible Computation-Communication Tradeoff Zhu, Jinbao; Li, Songze 2022 IEEE International Symposium on Information Theory (ISIT) https://doi.org/10.1109/ISIT50566.2022.9834535	conference	June 2022
Computational Code-Based Privacy in Coded Federated Learning Xhemrishi, Marvin; Graell i. Amat, Alexandre; Rosnes, Eirik 2022 IEEE International Symposium on Information Theory (ISIT) https://doi.org/10.1109/ISIT50566.2022.9834802	conference	June 2022
Balanced Reed-Solomon codes for all parameters Halbawi, Wael; Liu, Zihan; Hassibi, Babak 2016 IEEE Information Theory Workshop (ITW) https://doi.org/10.1109/ITW.2016.7606866	conference	September 2016
Coded Computing for Low-Latency Federated Learning Over Wireless Edge Networks Prakash, Saurav; Dhakal, Sagar; Akdeniz, Mustafa Riza IEEE Journal on Selected Areas in Communications, Vol. 39, Issue 1 https://doi.org/10.1109/JSAC.2020.3036961	journal	January 2021
Analog Lagrange Coded Computing Soleymani, Mahdi; Mahdavifar, Hessam; Avestimehr, A. Salman IEEE Journal on Selected Areas in Information Theory, Vol. 2, Issue 1 https://doi.org/10.1109/JSAIT.2021.3056377	journal	March 2021
Optimal Communication-Computation Trade-Off in Heterogeneous Gradient Coding Jahani-Nezhad, Tayyebeh; Maddah-Ali, Mohammad Ali IEEE Journal on Selected Areas in Information Theory, Vol. 2, Issue 3 https://doi.org/10.1109/JSAIT.2021.3085676	journal	September 2021
List-Decodable Coded Computing: Breaking the Adversarial Toleration Barrier Soleymani, Mahdi; Ali, Ramy E.; Mahdavifar, Hessam IEEE Journal on Selected Areas in Information Theory, Vol. 2, Issue 3 https://doi.org/10.1109/JSAIT.2021.3102956	journal	September 2021
Successive Approximation Coding for Distributed Matrix Multiplication Kiani, Shahrzad; Draper, Stark C. IEEE Journal on Selected Areas in Information Theory, Vol. 3, Issue 2 https://doi.org/10.1109/JSAIT.2022.3190859	journal	June 2022
Maximum-Likelihood Decoding of Reed–Solomon Codes is NP-Hard Guruswami, V.; Vardy, A. IEEE Transactions on Information Theory, Vol. 51, Issue 7 https://doi.org/10.1109/TIT.2005.850102	journal	July 2005
Explicit Codes Achieving List Decoding Capacity: Error-Correction With Optimal Redundancy Guruswami, Venkatesan; Rudra, Atri IEEE Transactions on Information Theory, Vol. 54, Issue 1 https://doi.org/10.1109/TIT.2007.911222	journal	January 2008
Speeding Up Distributed Machine Learning Using Codes Lee, Kangwook; Lam, Maximilian; Pedarsani, Ramtin IEEE Transactions on Information Theory, Vol. 64, Issue 3 https://doi.org/10.1109/TIT.2017.2736066	journal	March 2018
On the Optimal Recovery Threshold of Coded Matrix Multiplication Dutta, Sanghamitra; Fahim, Mohammad; Haddadpour, Farzin IEEE Transactions on Information Theory, Vol. 66, Issue 1 https://doi.org/10.1109/TIT.2019.2929328	journal	January 2020
Straggler Mitigation in Distributed Matrix Multiplication: Fundamental Limits and Optimal Coding Yu, Qian; Maddah-Ali, Mohammad Ali; Avestimehr, A. Salman IEEE Transactions on Information Theory, Vol. 66, Issue 3 https://doi.org/10.1109/TIT.2019.2963864	journal	March 2020
Gradient Coding From Cyclic MDS Codes and Expander Graphs Raviv, Netanel; Tamo, Itzhak; Tandon, Rashish IEEE Transactions on Information Theory, Vol. 66, Issue 12 https://doi.org/10.1109/TIT.2020.3029396	journal	December 2020
Numerically Stable Coded Matrix Computations via Circulant and Rotation Matrix Embeddings Ramamoorthy, Aditya; Tang, Li IEEE Transactions on Information Theory, Vol. 68, Issue 4 https://doi.org/10.1109/TIT.2021.3137266	journal	April 2022
Accuracy and Stability of Numerical Algorithms Higham, Nicholas J. https://doi.org/10.1137/1.9780898718027	book	January 2002
Numerical Linear Algebra Trefethen, Lloyd N.; Bau, David https://doi.org/10.1137/1.9780898719574	book	January 1997
How to share a secret Shamir, Adi Communications of the ACM, Vol. 22, Issue 11 https://doi.org/10.1145/359168.359176	journal	November 1979
Coded Computing: Mitigating Fundamental Bottlenecks in Large-Scale Distributed Computing and Machine Learning Li, Songze; Avestimehr, Salman Foundations and Trends® in Communications and Information Theory, Vol. 17, Issue 1 https://doi.org/10.1561/0100000103	journal	January 2020
Convex Optimization: Algorithms and Complexity Bubeck, Sébastien Foundations and Trends® in Machine Learning, Vol. 8, Issue 3-4 https://doi.org/10.1561/2200000050	journal	January 2015
Solution of Vandermonde Systems of Equations Bjorck, Ake; Pereyra, Victor Mathematics of Computation, Vol. 24, Issue 112 https://doi.org/10.2307/2004623	journal	October 1970
A Simple Proof of the Gale-Ryser Theorem Krause, Manfred The American Mathematical Monthly, Vol. 103, Issue 4 https://doi.org/10.2307/2975191	journal	April 1996

Similar Records

Gradient Coding With Iterative Block Leverage Score Sampling

Journal Article · 2024 · IEEE Transactions on Information Theory · OSTI ID:2475404

Charalambides, Neophytos; Pilanci, Mert; Hero, Alfred O.

Prospects of federated machine learning in fluid dynamics

Journal Article · 2022 · AIP Advances · OSTI ID:1979146

San, Omer; Pawar, Suraj; Rasheed, Adil

Decentralized digital twins of complex dynamical systems

Journal Article · 2023 · Scientific Reports · OSTI ID:2212807

San, Omer; Pawar, Suraj; Rasheed, Adil

Related Subjects

97 MATHEMATICS AND COMPUTING
Coding theory
cryptography
lagrange interpolation
numerical analysis
optimization

Title: Securely Aggregated Coded Matrix Inversion

Citation Formats

References (36)

Similar Records

Related Subjects