Mimicking of corruption in images
Abstract
Methods and systems are provided to generate an uncorrupted version of an image given an observed image that is a corrupted version of the image. In some embodiments, a corruption mimicking (“CM”) system iteratively trains a corruption mimicking network (“CMN”) to generate corrupted images given modeled images, updates latent vectors based on differences between the corrupted images and observed images, and applies a generator to the latent vectors to generate modeled images. The training, updating, and applying are performed until modeled images that are input to the CMN result in corrupted images that approximate the observed images. Because the CMN is trained to mimic the corruption of the observed images, the final modeled images represented the uncorrupted version of the observed images.
- Inventors:
- Issue Date:
- Research Org.:
- Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States)
- Sponsoring Org.:
- USDOE
- OSTI Identifier:
- 1840475
- Patent Number(s):
- 11126895
- Application Number:
- 16/840,358
- Assignee:
- Lawrence Livermore National Security, LLC (Livermore, CA)
- Patent Classifications (CPCs):
-
G - PHYSICS G06 - COMPUTING G06N - COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
- DOE Contract Number:
- AC52-07NA27344
- Resource Type:
- Patent
- Resource Relation:
- Patent File Date: 04/04/2020
- Country of Publication:
- United States
- Language:
- English
Citation Formats
Anirudh, Rushil, Bremer, Peer-Timo, Jayaraman Thiagarajan, Jayaraman, and Kailkhura, Bhavya. Mimicking of corruption in images. United States: N. p., 2021.
Web.
Anirudh, Rushil, Bremer, Peer-Timo, Jayaraman Thiagarajan, Jayaraman, & Kailkhura, Bhavya. Mimicking of corruption in images. United States.
Anirudh, Rushil, Bremer, Peer-Timo, Jayaraman Thiagarajan, Jayaraman, and Kailkhura, Bhavya. Tue .
"Mimicking of corruption in images". United States. https://www.osti.gov/servlets/purl/1840475.
@article{osti_1840475,
title = {Mimicking of corruption in images},
author = {Anirudh, Rushil and Bremer, Peer-Timo and Jayaraman Thiagarajan, Jayaraman and Kailkhura, Bhavya},
abstractNote = {Methods and systems are provided to generate an uncorrupted version of an image given an observed image that is a corrupted version of the image. In some embodiments, a corruption mimicking (“CM”) system iteratively trains a corruption mimicking network (“CMN”) to generate corrupted images given modeled images, updates latent vectors based on differences between the corrupted images and observed images, and applies a generator to the latent vectors to generate modeled images. The training, updating, and applying are performed until modeled images that are input to the CMN result in corrupted images that approximate the observed images. Because the CMN is trained to mimic the corruption of the observed images, the final modeled images represented the uncorrupted version of the observed images.},
doi = {},
journal = {},
number = ,
volume = ,
place = {United States},
year = {2021},
month = {9}
}
Works referenced in this record:
Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks
conference, October 2017
- Zhu, Jun-Yan; Park, Taesung; Isola, Phillip
- 2017 IEEE International Conference on Computer Vision (ICCV)
Systems and Methods for Solving Inverse Problems Using a Coupled Autoencoder
patent-application, January 2019
- Gupta, Kavya; Bhowmick, Brojeshwar; Majumdar, Angshul
- US Patent Application 15/898144; 20190026869
Kernel-Predicting Convolutional Neural Networks for Denoising
patent-application, October 2018
- Vogels, Thijs; Novak, Jan; Rousselle, Fabrice
- US Patent Application 15/814190; 20180293711
Learning from Estimated High-Dynamic Range All Weather Lighting Parameters
patent-application, March 2021
- Zhang, Jinsong; Sunkavalli, Kalyan K.; Hold-Geoffroy, Yannick
- US Patent Application 16/564398; 20210073955
Solving Linear Inverse Problems Using Gan Priors: An Algorithm with Provable Guarantees
conference, April 2018
- Shah, Viraj; Hegde, Chinmay
- ICASSP 2018 - 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Techniques for Upscaling Images Generated with Undetermined Down Scaling Kernels
patent-application, February 2021
- Schroers, Christopher; Wang, Yifan; Cornillere, Victor
- US Patent Application 16/542227; 20210049739
Optimality and stability of the K-hyperline clustering algorithm
journal, July 2011
- Thiagarajan, Jayaraman J.; Ramamurthy, Karthikeyan N.; Spanias, Andreas
- Pattern Recognition Letters, Vol. 32, Issue 9
Image signal processor for processing images
patent, May 2020
- Hwang, Hau; Pankaj, Tushar Sinha; Gupta, Vishal
- US Patent Document 10,643,306
Method of Image Completion
patent-application, August 2019
- Lai, Shang-Hong; Tseng, Ching-Wei; Lin, Hung-Jin
- US Patent Application 15/987469; 20190236759
Method for Detecting an Anomalous Image Among a First Dataset of Images Using an Adversarial Autoencoder
patent-application, May 2019
- Beggel, Laura; Pfeiffer, Michael
- US Patent Application 16/171637; 20190130279
Evaluating Open-Universe Face Identification on the Web
conference, June 2013
- Becker, Brian C.; Ortiz, Enrique G.
- 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops
Systems and Methods for Reconstructing Frames
patent-application, March 2021
- Schroers, Christopher; Campos, Joaquim; Djelouah, Abdelaziz
- US Patent Application 16/556083; 20210067801
Deep Learning Face Attributes in the Wild
conference, December 2015
- Liu, Ziwei; Luo, Ping; Wang, Xiaogang
- 2015 IEEE International Conference on Computer Vision (ICCV)
Mixing matrix estimation using discriminative clustering for blind source separation
journal, January 2013
- J. Thiagarajan, Jayaraman; Natesan Ramamurthy, Karthikeyan; Spanias, Andreas
- Digital Signal Processing, Vol. 23, Issue 1
Image Processing Apparatus and Image Processing Method Thereof
patent-application, November 2020
- Lee, Cheon; Kim, Donghyun; Park, Yongsup
- US Patent Application 16/838650; 20200372608
Generating Multimodal Image Edits for a Digital Image
patent-application, August 2018
- DiVerdi, Stephen; Hoffman, Matthew Douglas; Saeedi, Ardavan
- US Patent Application 15/427598; 20180225812
Deep Residual Learning for Image Recognition
conference, June 2016
- He, Kaiming; Zhang, Xiangyu; Ren, Shaoqing
- 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Towards Evaluating the Robustness of Neural Networks
conference, May 2017
- Carlini, Nicholas; Wagner, David
- 2017 IEEE Symposium on Security and Privacy (SP)
Methods and Apparatus for Error Correction Coding with Triangular Factorization of Generator Matrix
patent-application, December 2020
- Arikan, Erdal
- US Patent Application 16/453887; 20200412385
Ultra-High Compression of Images Based on Deep Learning
patent-application, October 2016
- Taylor, Jr., Robert M.; Necioglu, Burhan
- US Patent Application 14/678532; 20160292589
Image Understanding Using Sparse Representations
journal, April 2014
- Thiagarajan, Jayaraman J.; Ramamurthy, Karthikeyan Natesan; Turaga, Pavan
- Synthesis Lectures on Image, Video, and Multimedia Processing, Vol. 7, Issue 1
Training a Class-Conditional Generative Adversarial Network
patent-application, March 2021
- Zhang, Dan; Khoreva, Anna
- US Patent Application 16/941699; 20210073630
Linear inverse problems in imaging
journal, July 2008
- Ribes, Alejandro; Schmitt, Francis
- IEEE Signal Processing Magazine, Vol. 25, Issue 4
ReconNet: Non-Iterative Reconstruction of Images from Compressively Sensed Measurements
conference, June 2016
- Kulkarni, Kuldeep; Lohit, Suhas; Turaga, Pavan
- 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
A Style-Based Generator Architecture for Generative Adversarial Networks
conference, June 2019
- Karras, Tero; Laine, Samuli; Aila, Timo
- 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?
conference, October 2019
- Abdal, Rameen; Qin, Yipeng; Wonka, Peter
- 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
Generating user-customized items using a visually-aware image generation network
patent, April 2021
- Fang, Chen; Wang, Zhaowen; Kang, Wangcheng
- US Patent Document 10,970,765
Context Encoders: Feature Learning by Inpainting
conference, June 2016
- Pathak, Deepak; Krahenbuhl, Philipp; Donahue, Jeff
- 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Semantic Image Inpainting with Deep Generative Models
conference, July 2017
- Yeh, Raymond A.; Chen, Chen; Lim, Teck Yian
- 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Generating User-Customized Items Using a Visually-Aware Image Generation Network
patent-application, August 2019
- Fang, Chen; Wang, Zhaowen; Kang, Wangcheng
- US Patent Application 15/897856; 20190251612
System and Method for MRI Image Synthesis for the Diagnosis of Parkinson's Disease Using Deep Learning
patent-application, July 2020
- Kopparapu, Neeyanth
- US Patent Application 16/750314; 20200237284
Semantic Segmentation Method and Apparatus for Three-Dimensional Image, Thermal, and Storage Medium
patent-application, February 2021
- Chen, Sihong
- US Patent Application 17/074629; 20210049397
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
conference, July 2017
- Ledig, Christian; Theis, Lucas; Huszar, Ferenc
- 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Scaling up biologically-inspired computer vision: A case study in unconstrained face recognition on facebook
conference, June 2011
- Pinto, Nicolas; Stone, Zak; Zickler, Todd
- CVPR 2011 WORKSHOPS
Image Processing Method and Device, Storage Medium and Electronic Device
patent-application, November 2020
- Tao, Xin; Gao, Hong Yun; Shen, Xiao Yong
- US Patent Application 16/934823; 20200349680
Video Enhancement Using a Generator with Filters of Generative Adversarial Network
patent-application, November 2020
- Mayol Cuevas, Walterio Wolfgang; Andrei, Silviu Stefan; Shapovalova, Nataliya
- US Patent Application 16/403386; 20200349682
ImageNet Large Scale Visual Recognition Challenge
journal, April 2015
- Russakovsky, Olga; Deng, Jia; Su, Hao
- International Journal of Computer Vision, Vol. 115, Issue 3
Generative Augmentation of Image Data
patent-application, September 2019
- Zuev, Konstantin; Sautins, Andrejs
- US Patent Application 15/938897; 20190294961
Neural Network System with Temporal Feedback for Denoising of Rendered Sequences
patent-application, April 2020
- Munkberg, Carl Jacob; Hasselgren, Jon Niklas Theodor; Patney, Anjul
- US Patent Application 16/718607; 20200126192
Information Processing System, Inference Method, Attack Detection Method, Inference Execution Program and Attack Detection Program
patent-application, December 2020
- Matsumura, Tadayuki
- US Patent Application 16/819532; 20200380338
Denoising Monte Carlo Renderings Using Progressive Neural Networks
patent-application, October 2018
- Vogels, Thijs; Rouselle, Fabrice; McWilliams, Brian
- US Patent Application 15/946652; 20180293496
Surface Defect Identification Method and Apparatus
patent-application, November 2020
- Chaton, Thomas
- US Patent Application 16/835908; 20200364842
DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks
conference, June 2016
- Moosavi-Dezfooli, Seyed-Mohsen; Fawzi, Alhussein; Frossard, Pascal
- 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Model Parameter Learning Device, Control Device, and Model Parameter Learning Method
patent-application, March 2020
- Nakanishi, Kosuke; Yasui, Yuji; Sasaki, Wataru
- US Patent Application 16/574370; 20200097772
Unsupervised Visual Domain Adaptation Using Subspace Alignment
conference, December 2013
- Fernando, Basura; Habrard, Amaury; Sebban, Marc
- 2013 IEEE International Conference on Computer Vision
Image-to-Image Translation with Conditional Adversarial Networks
conference, July 2017
- Isola, Phillip; Zhu, Jun-Yan; Zhou, Tinghui
- 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
One Network to Solve Them All — Solving Linear Inverse Problems Using Deep Projection Models
conference, October 2017
- Chang, J. H. Rick; Li, Chun-Liang; Poczos, Barnabas
- 2017 IEEE International Conference on Computer Vision (ICCV)
Image Distortion Correction Method and Image Distortion Correction Device Using the Same
patent-application, March 2015
- Chen, Ching-Han; Yao, Tun-Kai
- US Patent Application 14/175227; 20150063685
Method and Apparatus for Applying Deep Learning Techniques in Video Coding, Restoration and Video Quality Analysis (VQA)
patent-application, April 2021
- Topiwala, Pankaj N.; Krishnan, Madhu Peringassery; Dai, Wei
- US Patent Application 17/119981; 20210099715