Search Menu
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information
Search Scholarly Publications

Title: Detecting damaged buildings using real-time crowdsourced images and transfer learning

Abstract

After significant earthquakes, we can see images posted on social media platforms by individuals and media agencies owing to the mass usage of smartphones these days. These images can be utilized to provide information about the shaking damage in the earthquake region both to the public and research community, and potentially to guide rescue work. This paper presents an automated way to extract the damaged buildings images after earthquakes from social media platforms such as Twitter and thus identify the particular user posts containing such images. Using transfer learning and ~ 6500 manually labelled images, we trained a deep learning model to recognize images with damaged buildings in the scene. The trained model achieved good performance when tested on newly acquired images of earthquakes at different locations and when ran in near real-time on Twitter feed after the 2020 M7.0 earthquake in Turkey. Furthermore, to better understand how the model makes decisions, we also implemented the Grad-CAM method to visualize the important regions on the images that facilitate the decision.

Authors:
 [1];  [2];  [3];  [1];  [1];  [1];  [4]
+ Show Author Affiliations
  1. Univ. of California, Berkeley, CA (United States)
  2. Univ. of California, Berkeley, CA (United States). Berkeley Seismological Lab.; Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
  3. AT&T, Dallas, TX (United States)
  4. Univ. of California, Berkeley, CA (United States). Berkeley Seismological Lab.
Publication Date:
Research Org.:
Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
Sponsoring Org.:
USDOE National Nuclear Security Administration (NNSA); Gordon and Betty Moore Foundation
OSTI Identifier:
1870564
Report Number(s):
LLNL-JRNL-826989
Journal ID: ISSN 2045-2322; 1041828
Grant/Contract Number:  
AC52-07NA27344; GBMF5230
Resource Type:
Accepted Manuscript
Journal Name:
Scientific Reports
Additional Journal Information:
Journal Volume: 12; Journal Issue: 1; Journal ID: ISSN 2045-2322
Publisher:
Nature Publishing Group
Country of Publication:
United States
Language:
English
Subject:
58 GEOSCIENCES; natural hazards; seismology

Citation Formats

Chachra, Gaurav, Kong, Qingkai, Huang, Jim, Korlakunta, Srujay, Grannen, Jennifer, Robson, Alexander, and Allen, Richard M. Detecting damaged buildings using real-time crowdsourced images and transfer learning. United States: N. p., 2022. Web. doi:10.1038/s41598-022-12965-0.
Copy to clipboard
Chachra, Gaurav, Kong, Qingkai, Huang, Jim, Korlakunta, Srujay, Grannen, Jennifer, Robson, Alexander, & Allen, Richard M. Detecting damaged buildings using real-time crowdsourced images and transfer learning. United States. https://doi.org/10.1038/s41598-022-12965-0
Copy to clipboard
Chachra, Gaurav, Kong, Qingkai, Huang, Jim, Korlakunta, Srujay, Grannen, Jennifer, Robson, Alexander, and Allen, Richard M. Fri . "Detecting damaged buildings using real-time crowdsourced images and transfer learning". United States. https://doi.org/10.1038/s41598-022-12965-0. https://www.osti.gov/servlets/purl/1870564.
Copy to clipboard
@article{osti_1870564,
title = {Detecting damaged buildings using real-time crowdsourced images and transfer learning},
author = {Chachra, Gaurav and Kong, Qingkai and Huang, Jim and Korlakunta, Srujay and Grannen, Jennifer and Robson, Alexander and Allen, Richard M.},
abstractNote = {After significant earthquakes, we can see images posted on social media platforms by individuals and media agencies owing to the mass usage of smartphones these days. These images can be utilized to provide information about the shaking damage in the earthquake region both to the public and research community, and potentially to guide rescue work. This paper presents an automated way to extract the damaged buildings images after earthquakes from social media platforms such as Twitter and thus identify the particular user posts containing such images. Using transfer learning and ~ 6500 manually labelled images, we trained a deep learning model to recognize images with damaged buildings in the scene. The trained model achieved good performance when tested on newly acquired images of earthquakes at different locations and when ran in near real-time on Twitter feed after the 2020 M7.0 earthquake in Turkey. Furthermore, to better understand how the model makes decisions, we also implemented the Grad-CAM method to visualize the important regions on the images that facilitate the decision.},
doi = {10.1038/s41598-022-12965-0},
journal = {Scientific Reports},
number = 1,
volume = 12,
place = {United States},
year = {Fri May 27 00:00:00 EDT 2022},
month = {Fri May 27 00:00:00 EDT 2022}
}
Copy to clipboard
Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record
https://doi.org/10.1038/s41598-022-12965-0
Copyright Statement
Other availability
Search WorldCat Search WorldCat to find libraries that may hold this journal
Save / Share:
Export Metadata
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.

Works referenced in this record:

Low-Light Image Enhancement by Principal Component Analysis
journal, January 2019

MyShake: Using Human-Centered Design Methods to Promote Engagement in a Smartphone-Based Global Seismic Network
journal, December 2018

  • Rochford, Kaylin; Strauss, Jennifer A.; Kong, Qingkai
  • Frontiers in Earth Science, Vol. 6
  • DOI: 10.3389/feart.2018.00237

ImageNet: A large-scale hierarchical image database
conference, June 2009

  • Deng, Jia; Dong, Wei; Socher, Richard
  • 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPR Workshops), 2009 IEEE Conference on Computer Vision and Pattern Recognition
  • DOI: 10.1109/CVPR.2009.5206848

Image-Based Concrete Crack Detection Using Convolutional Neural Network and Exhaustive Search Technique
journal, April 2019

Image4Act: Online Social Media Image Processing for Disaster Response
conference, July 2017

  • Alam, Firoj; Imran, Muhammad; Ofli, Ferda
  • ASONAM '17: Advances in Social Networks Analysis and Mining 2017, Proceedings of the 2017 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2017
  • DOI: 10.1145/3110025.3110164

MyShake: Initial observations from a global smartphone seismic network: OBSERVATIONS FROM MYSHAKE
journal, September 2016

  • Kong, Qingkai; Allen, Richard M.; Schreier, Louis
  • Geophysical Research Letters, Vol. 43, Issue 18
  • DOI: 10.1002/2016GL070955

Processing Social Media Images by Combining Human and Machine Computing during Crises
journal, January 2018

  • Alam, Firoj; Ofli, Ferda; Imran, Muhammad
  • International Journal of Human–Computer Interaction, Vol. 34, Issue 4
  • DOI: 10.1080/10447318.2018.1427831

The MyShake Platform: A Global Vision for Earthquake Early Warning
journal, October 2019

  • Allen, Richard M.; Kong, Qingkai; Martin-Short, Robert
  • Pure and Applied Geophysics, Vol. 177, Issue 4
  • DOI: 10.1007/s00024-019-02337-7

Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
conference, October 2017

  • Selvaraju, Ramprasaath R.; Cogswell, Michael; Das, Abhishek
  • 2017 IEEE International Conference on Computer Vision (ICCV)
  • DOI: 10.1109/ICCV.2017.74

MyShake Citizen Seismologists Help Launch Dual-Use Seismic Network in California
journal, May 2020

Leveraging multimodal social media data for rapid disaster damage assessment
journal, December 2020

Toward Global Earthquake Early Warning with the MyShake Smartphone Seismic Network, Part 2: Understanding MyShake Performance around the World
journal, May 2020

  • Kong, Qingkai; Martin-Short, Robert; Allen, Richard M.
  • Seismological Research Letters, Vol. 91, Issue 4
  • DOI: 10.1785/0220190178

A survey of transfer learning
journal, May 2016

Contrast enhancement of night images
conference, July 2016

  • Priyanka, Steffi Agino; Tung, Hsiao-Jung; Wang, Yuan Kai
  • 2016 International Conference on Machine Learning and Cybernetics (ICMLC)
  • DOI: 10.1109/ICMLC.2016.7860931

Performance Comparison of Pretrained Convolutional Neural Networks on Crack Detection in Buildings
conference, July 2018

  • Özgenel, Ç. F.; Sorguç, Arzu Gönenç
  • 34th International Symposium on Automation and Robotics in Construction, Proceedings of the International Symposium on Automation and Robotics in Construction (IAARC)
  • DOI: 10.22260/ISARC2018/0094

Damage Assessment from Social Media Imagery Data During Disasters
conference, July 2017

  • Nguyen, Dat T.; Ofli, Ferda; Imran, Muhammad
  • ASONAM '17: Advances in Social Networks Analysis and Mining 2017, Proceedings of the 2017 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2017
  • DOI: 10.1145/3110025.3110109

MyShake: A smartphone seismic network for earthquake early warning and beyond
journal, February 2016

  • Kong, Qingkai; Allen, Richard M.; Schreier, Louis
  • Science Advances, Vol. 2, Issue 2
  • DOI: 10.1126/sciadv.1501055

Extreme Low-Light Image Enhancement for Surveillance Cameras Using Attention U-Net
journal, January 2020

Converting Night-Time Images to Day-Time Images through a Deep Learning Approach
conference, July 2017

  • Capece, Nicola; Erra, Ugo; Scolamiero, Raffaele
  • 2017 21st International Conference on Information Visualisation (IV), 2017 21st International Conference Information Visualisation (IV)
  • DOI: 10.1109/iV.2017.16
    Sort by:
    [ × clear filter / sort ]

    Similar Records in DOE PAGES and OSTI.GOV collections: