skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Double Q-Learning for Radiation Source Detection

Abstract

Anomalous radiation source detection in urban environments is challenging due to the complex nature of background radiation. When a suspicious area is determined, a radiation survey is usually carried out to search for anomalous radiation sources. To locate the source with high accuracy and in a short time, different survey approaches have been studied such as scanning the area with fixed survey paths and data-driven approaches that update the survey path on the fly with newly acquired measurements. In this work, we propose reinforcement learning as a data-driven approach to conduct radiation detection tasks with no human intervention. A simulated radiation environment is constructed, and a convolutional neural network-based double Q-learning algorithm is built and tested for radiation source detection tasks. Simulation results show that the double Q-learning algorithm can reliably navigate the detector and reduce the searching time by at least 44% compared with traditional uniform search methods and gradient search methods.

Authors:
ORCiD logo;
Publication Date:
Research Org.:
North Carolina State Univ., Raleigh, NC (United States)
Sponsoring Org.:
USDOE National Nuclear Security Administration (NNSA); Defense Threat Reduction Agency (DTRA)
OSTI Identifier:
1496491
Alternate Identifier(s):
OSTI ID: 1614545
Grant/Contract Number:  
NA0002576; HDTRA 1-14-1-0011
Resource Type:
Published Article
Journal Name:
Sensors
Additional Journal Information:
Journal Name: Sensors Journal Volume: 19 Journal Issue: 4; Journal ID: ISSN 1424-8220
Publisher:
MDPI AG
Country of Publication:
Switzerland
Language:
English
Subject:
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CHEMISTRY; 36 MATERIALS SCIENCE; 47 OTHER INSTRUMENTATION; chemistry; engineering; instruments & instrumentation; reinforcement learning; radiation detection; source searching

Citation Formats

Liu, Zheng, and Abbaszadeh, Shiva. Double Q-Learning for Radiation Source Detection. Switzerland: N. p., 2019. Web. https://doi.org/10.3390/s19040960.
Liu, Zheng, & Abbaszadeh, Shiva. Double Q-Learning for Radiation Source Detection. Switzerland. https://doi.org/10.3390/s19040960
Liu, Zheng, and Abbaszadeh, Shiva. Sun . "Double Q-Learning for Radiation Source Detection". Switzerland. https://doi.org/10.3390/s19040960.
@article{osti_1496491,
title = {Double Q-Learning for Radiation Source Detection},
author = {Liu, Zheng and Abbaszadeh, Shiva},
abstractNote = {Anomalous radiation source detection in urban environments is challenging due to the complex nature of background radiation. When a suspicious area is determined, a radiation survey is usually carried out to search for anomalous radiation sources. To locate the source with high accuracy and in a short time, different survey approaches have been studied such as scanning the area with fixed survey paths and data-driven approaches that update the survey path on the fly with newly acquired measurements. In this work, we propose reinforcement learning as a data-driven approach to conduct radiation detection tasks with no human intervention. A simulated radiation environment is constructed, and a convolutional neural network-based double Q-learning algorithm is built and tested for radiation source detection tasks. Simulation results show that the double Q-learning algorithm can reliably navigate the detector and reduce the searching time by at least 44% compared with traditional uniform search methods and gradient search methods.},
doi = {10.3390/s19040960},
journal = {Sensors},
number = 4,
volume = 19,
place = {Switzerland},
year = {2019},
month = {2}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record
https://doi.org/10.3390/s19040960

Save / Share:

Works referenced in this record:

Efficient strategies for low-statistics nuclear searches
journal, June 2006

  • Klimenko, A. V.; Priedhorsky, W. C.; Hengartner, N. W.
  • IEEE Transactions on Nuclear Science, Vol. 53, Issue 3
  • DOI: 10.1109/TNS.2005.862860

Deep Reinforcement Learning: A Brief Survey
journal, November 2017

  • Arulkumaran, Kai; Deisenroth, Marc Peter; Brundage, Miles
  • IEEE Signal Processing Magazine, Vol. 34, Issue 6
  • DOI: 10.1109/MSP.2017.2743240

Radiological Source Detection and Localisation Using Bayesian Techniques
journal, November 2009


Kernel-Based Machine Learning for Background Estimation of NaI Low-Count Gamma-Ray Spectra
journal, June 2013

  • Alamaniotis, M.; Mattingly, J.; Tsoukalas, L. H.
  • IEEE Transactions on Nuclear Science, Vol. 60, Issue 3
  • DOI: 10.1109/TNS.2013.2260868

A Markovian Decision Process
journal, January 1957


Smart radiation sensor management
journal, September 2008

  • Cortez, R.; Papageorgiou, Xanthi; Tanner, Herbert
  • IEEE Robotics & Automation Magazine, Vol. 15, Issue 3
  • DOI: 10.1109/MRA.2008.928590

Human-level control through deep reinforcement learning
journal, February 2015

  • Mnih, Volodymyr; Kavukcuoglu, Koray; Silver, David
  • Nature, Vol. 518, Issue 7540
  • DOI: 10.1038/nature14236

Networked sensing systems for detecting people carrying radioactive material
conference, June 2008

  • Chandy, Mani; Pilotto, Concetta; McLean, Ryan
  • 2008 Fifth International Conference on Networked Sensing Systems (INSS), 2008 5th International Conference on Networked Sensing Systems
  • DOI: 10.1109/INSS.2008.4610916

Maximum Likelihood Localization of Radioactive Sources Against a Highly Fluctuating Background
journal, December 2015

  • Bai, Er-wei; Heifetz, Alexander; Raptis, Paul
  • IEEE Transactions on Nuclear Science, Vol. 62, Issue 6
  • DOI: 10.1109/TNS.2015.2497327

Adaptive Bayesian Sensor Motion Planning for Hazardous Source Term Reconstruction
journal, July 2017


Examination of Count-Starved Gamma Spectra Using the Method of Spectral Comparison Ratios
journal, August 2007

  • Pfund, David Michael; Runkle, Robert C.; Anderson, Kevin K.
  • IEEE Transactions on Nuclear Science, Vol. 54, Issue 4
  • DOI: 10.1109/TNS.2007.901202

Iterative Estimation of Location and Trajectory of Radioactive Sources With a Networked System of Detectors
journal, April 2013


Q-learning
journal, May 1992

  • Watkins, Christopher J. C. H.; Dayan, Peter
  • Machine Learning, Vol. 8, Issue 3-4
  • DOI: 10.1007/BF00992698

Information driven search for point sources of gamma radiation
journal, April 2010


Cooperation between an unmanned aerial vehicle and an unmanned ground vehicle in highly accurate localization of gamma radiation hotspots
journal, January 2018

  • Lazna, Tomas; Gabrlik, Petr; Jilek, Tomas
  • International Journal of Advanced Robotic Systems, Vol. 15, Issue 1
  • DOI: 10.1177/1729881417750787

Examination of Count-starved Gamma Spectra Using the Method of Spectral Comparison Ratios
conference, October 2006

  • Pfund, David M.; Runkle, Robert C.; Anderson, Kevin K.
  • 2006 IEEE Nuclear Science Symposium Conference Record
  • DOI: 10.1109/nssmic.2006.356110

    Works referencing / citing this record:

    Kernel-based Gaussian process for anomaly detection in sparse gamma-ray data
    journal, January 2020