Stable parallel training of Wasserstein conditional generative adversarial neural networks

Lupo Pasini, Massimiliano; Yin, Junqi

doi:10.1007/s11227-022-04721-y

Stable parallel training of Wasserstein conditional generative adversarial neural networks

Journal Article · Wed Aug 03 04:00:00 EDT 2022 · Journal of Supercomputing

DOI:https://doi.org/10.1007/s11227-022-04721-y· OSTI ID:1908079

^[1]; ^[1]

Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)

In this work, we propose a stable, parallel approach to train Wasserstein conditional generative adversarial neural networks (W-CGANs) under the constraint of a fixed computational budget. Differently from previous distributed GANs training techniques, our approach avoids inter-process communications, reduces the risk of mode collapse and enhances scalability by using multiple generators, each one of them concurrently trained on a single data label. The use of the Wasserstein metric also reduces the risk of cycling by stabilizing the training of each generator. We illustrate the approach on the CIFAR10, CIFAR100, and ImageNet1k datasets, three standard benchmark image datasets, maintaining the original resolution of the images for each dataset. Performance is assessed in terms of scalability and final accuracy within a limited fixed computational time and computational resources. To measure accuracy, we use the inception score, the Fréchet inception distance, and image quality. An improvement in inception score and Fréchet inception distance is shown in comparison to previous results obtained by performing the parallel approach on deep convolutional conditional generative adversarial neural networks as well as an improvement of image quality of the new images created by the GANs approach. Weak scaling is attained on both datasets using up to 2000 NVIDIA V100 GPUs on the OLCF supercomputer Summit.

Research Organization:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Sponsoring Organization:: USDOE Laboratory Directed Research and Development (LDRD) Program

Grant/Contract Number:: AC05-00OR22725

OSTI ID:: 1908079

Journal Information:: Journal of Supercomputing, Journal Name: Journal of Supercomputing Journal Issue: 2 Vol. 79; ISSN 0920-8542

Publisher:: SpringerCopyright Statement

Country of Publication:: United States

Language:: English

References (12)

Relation Between the Kantorovich–Wasserstein Metric and the Kullback–Leibler Divergence Belavkin, Roman V. Information Geometry and Its Applications https://doi.org/10.1007/978-3-319-97798-0_15	book	January 2018
Scalable balanced training of conditional generative adversarial neural networks on image data Lupo Pasini, Massimiliano; Gabbi, Vittorio; Yin, Junqi The Journal of Supercomputing, Vol. 77, Issue 11 https://doi.org/10.1007/s11227-021-03808-2	journal	April 2021
Large-Scale Distributed Training Applied to Generative Adversarial Networks for Calorimeter Simulation Vlimant, Jean-Roch; Pantaleo, Felice; Pierini, Maurizio EPJ Web of Conferences, Vol. 214 https://doi.org/10.1051/epjconf/201921406025	journal	January 2019
Divergence measures based on the Shannon entropy Lin, J. IEEE Transactions on Information Theory, Vol. 37, Issue 1 https://doi.org/10.1109/18.61115	journal	January 1991
Multiagent Reinforcement Learning: Rollout and Policy Iteration Bertsekas, Dimitri IEEE/CAA Journal of Automatica Sinica, Vol. 8, Issue 2 https://doi.org/10.1109/JAS.2021.1003814	journal	February 2021
Image De-Raining Using a Conditional Generative Adversarial Network Zhang, He; Sindagi, Vishwanath; Patel, Vishal M. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 30, Issue 11 https://doi.org/10.1109/TCSVT.2019.2920407	journal	November 2020
On Information and Sufficiency Kullback, S.; Leibler, R. A. The Annals of Mathematical Statistics, Vol. 22, Issue 1 https://doi.org/10.1214/aoms/1177729694	journal	March 1951
Conditional Generative Adversarial Nets Mirza, Mehdi; Osindero, Simon arXiv https://doi.org/10.48550/ARXIV.1411.1784	preprint	January 2014
Generative Adversarial Network based on Resnet for Conditional Image Restoration Wang, Meng; Li, Huafeng; Li, Fang arXiv https://doi.org/10.48550/ARXIV.1707.04881	preprint	January 2017
A Note on the Inception Score Barratt, Shane; Sharma, Rishi arXiv https://doi.org/10.48550/ARXIV.1801.01973	preprint	January 2018
cGANs with Multi-Hinge Loss Kavalerov, Ilya; Czaja, Wojciech; Chellappa, Rama arXiv https://doi.org/10.48550/ARXIV.1912.04216	preprint	January 2019
Searching towards Class-Aware Generators for Conditional Generative Adversarial Networks Zhou, Peng; Xie, Lingxi; Zhang, Xiaopeng arXiv https://doi.org/10.48550/ARXIV.2006.14208	preprint	January 2020

Similar Records

Stable Parallel Training of Wasserstein Conditional Generative Adversarial Neural Networks : *Full/Regular Research Paper submission for the symposium CSCI-ISAI: Artificial Intelligence

Conference · Tue Nov 30 23:00:00 EST 2021 · OSTI ID:1877492

Scalable balanced training of conditional generative adversarial neural networks on image data

Journal Article · Mon Apr 26 00:00:00 EDT 2021 · Journal of Supercomputing · OSTI ID:1783019

Evolutionary Architecture Search for Generative Adversarial Networks Based on Weight Sharing

Journal Article · Thu Nov 30 23:00:00 EST 2023 · IEEE Transactions on Evolutionary Computation · OSTI ID:2429864

Related Subjects

97 MATHEMATICS AND COMPUTING
deep learning
generative adversarial neural networks
high-performance computing
parallel computing
supercomputing

Stable parallel training of Wasserstein conditional generative adversarial neural networks

Citation Formats

References (12)

Similar Records

Related Subjects