skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Automated data processing architecture for the Gemini Planet Imager Exoplanet Survey

Abstract

The Gemini Planet Imager Exoplanet Survey (GPIES) is a multiyear direct imaging survey of 600 stars to discover and characterize young Jovian exoplanets and their environments. We have developed an automated data architecture to process and index all data related to the survey uniformly. An automated and flexible data processing framework, which we term the Data Cruncher, combines multiple data reduction pipelines (DRPs) together to process all spectroscopic, polarimetric, and calibration data taken with GPIES. With no human intervention, fully reduced and calibrated data products are available less than an hour after the data are taken to expedite follow up on potential objects of interest. The Data Cruncher can run on a supercomputer to reprocess all GPIES data in a single day as improvements are made to our DRPs. A backend MySQL database indexes all files, which are synced to the cloud, and a front-end web server allows for easy browsing of all files associated with GPIES. To help observers, quicklook displays show reduced data as they are processed in real time, and chatbots on Slack post observing information as well as reduced data products. Together, the GPIES automated data processing architecture reduces our workload, provides real-time data reduction, optimizesmore » our observing strategy, and maintains a homogeneously reduced dataset to study planet occurrence and instrument performance.« less

Authors:
 [1];  [2];  [3];  [4];  [5];  [1];  [6];  [7];  [8];  [2];  [3];  [5];  [9];  [10]
  1. UC Berkeley, Department of Astronomy, Berkeley, California
  2. Space Telescope Science Institute, Baltimore, Maryland
  3. Cornell University, Sibley School of Mechanical and Aerospace Engineering, Ithaca, New York
  4. UCLA, Department of Physics and Astronomy, Los Angeles, California
  5. Stanford University, Kavli Institute for Particle Astrophysics and Cosmology, Stanford, California
  6. Jet Propulsion Laboratory, California Institute of Technology, Pasadena, California
  7. National Research Council of Canada Herzberg, Victoria, British Columbia
  8. Université de Montréal, Institut de Recherche sur les Exoplanètes, Départment de Physique, Montréal,
  9. University of California San Diego, Center for Astrophysics and Space Science, La Jolla, California
  10. SETI Institute, Carl Sagan Center, Mountain View, California
Publication Date:
Research Org.:
Lawrence Berkeley National Laboratory-National Energy Research Scientific Computing Center
Sponsoring Org.:
USDOE
OSTI Identifier:
1524052
Resource Type:
Journal Article
Journal Name:
Journal of Astronomical Telescopes, Instruments, and Systems
Additional Journal Information:
Journal Volume: 4; Journal Issue: 01; Journal ID: ISSN 2329-4124
Country of Publication:
United States
Language:
English

Citation Formats

Wang, Jason J., Perrin, Marshall D., Savransky, Dmitry, Arriaga, Pauline, Chilcote, Jeffrey K., De Rosa, Robert J., Millar-Blanchaer, Maxwell A., Marois, Christian, Rameau, Julien, Wolff, Schuyler G., Shapiro, Jacob, Ruffio, Jean-Baptiste, Maire, Jérôme, and Marchis, Franck. Automated data processing architecture for the Gemini Planet Imager Exoplanet Survey. United States: N. p., 2018. Web. doi:10.1117/1.JATIS.4.1.018002.
Wang, Jason J., Perrin, Marshall D., Savransky, Dmitry, Arriaga, Pauline, Chilcote, Jeffrey K., De Rosa, Robert J., Millar-Blanchaer, Maxwell A., Marois, Christian, Rameau, Julien, Wolff, Schuyler G., Shapiro, Jacob, Ruffio, Jean-Baptiste, Maire, Jérôme, & Marchis, Franck. Automated data processing architecture for the Gemini Planet Imager Exoplanet Survey. United States. doi:10.1117/1.JATIS.4.1.018002.
Wang, Jason J., Perrin, Marshall D., Savransky, Dmitry, Arriaga, Pauline, Chilcote, Jeffrey K., De Rosa, Robert J., Millar-Blanchaer, Maxwell A., Marois, Christian, Rameau, Julien, Wolff, Schuyler G., Shapiro, Jacob, Ruffio, Jean-Baptiste, Maire, Jérôme, and Marchis, Franck. Mon . "Automated data processing architecture for the Gemini Planet Imager Exoplanet Survey". United States. doi:10.1117/1.JATIS.4.1.018002.
@article{osti_1524052,
title = {Automated data processing architecture for the Gemini Planet Imager Exoplanet Survey},
author = {Wang, Jason J. and Perrin, Marshall D. and Savransky, Dmitry and Arriaga, Pauline and Chilcote, Jeffrey K. and De Rosa, Robert J. and Millar-Blanchaer, Maxwell A. and Marois, Christian and Rameau, Julien and Wolff, Schuyler G. and Shapiro, Jacob and Ruffio, Jean-Baptiste and Maire, Jérôme and Marchis, Franck},
abstractNote = {The Gemini Planet Imager Exoplanet Survey (GPIES) is a multiyear direct imaging survey of 600 stars to discover and characterize young Jovian exoplanets and their environments. We have developed an automated data architecture to process and index all data related to the survey uniformly. An automated and flexible data processing framework, which we term the Data Cruncher, combines multiple data reduction pipelines (DRPs) together to process all spectroscopic, polarimetric, and calibration data taken with GPIES. With no human intervention, fully reduced and calibrated data products are available less than an hour after the data are taken to expedite follow up on potential objects of interest. The Data Cruncher can run on a supercomputer to reprocess all GPIES data in a single day as improvements are made to our DRPs. A backend MySQL database indexes all files, which are synced to the cloud, and a front-end web server allows for easy browsing of all files associated with GPIES. To help observers, quicklook displays show reduced data as they are processed in real time, and chatbots on Slack post observing information as well as reduced data products. Together, the GPIES automated data processing architecture reduces our workload, provides real-time data reduction, optimizes our observing strategy, and maintains a homogeneously reduced dataset to study planet occurrence and instrument performance.},
doi = {10.1117/1.JATIS.4.1.018002},
journal = {Journal of Astronomical Telescopes, Instruments, and Systems},
issn = {2329-4124},
number = 01,
volume = 4,
place = {United States},
year = {2018},
month = {1}
}

Works referenced in this record:

Astropy: A community Python package for astronomy
journal, September 2013


Limb polarization of Uranus and Neptune: I. Imaging polarimetry and comparison with analytic models
journal, May 2006


Performance of the Gemini Planet Imager’s adaptive optics system
journal, January 2016

  • Poyneer, Lisa A.; Palmer, David W.; Macintosh, Bruce
  • Applied Optics, Vol. 55, Issue 2
  • DOI: 10.1364/AO.55.000323

Campaign Scheduling and Analysis for the Gemini Planet Imager
journal, June 2013

  • Savransky, Dmitry; Macintosh, Bruce A.; Graham, James
  • Proceedings of the International Astronomical Union, Vol. 8, Issue S299
  • DOI: 10.1017/S1743921313007916

Reconnaissance of the hr 8799 Exosolar System. ii. Astrometry and Orbital Motion
journal, April 2015


Experimental Design for the Gemini Planet Imager
journal, June 2011

  • McBride, James; Graham, James R.; Macintosh, Bruce
  • Publications of the Astronomical Society of the Pacific, Vol. 123, Issue 904
  • DOI: 10.1086/660733

First light of the Gemini Planet Imager
journal, May 2014

  • Macintosh, B.; Graham, J. R.; Ingraham, P.
  • Proceedings of the National Academy of Sciences, Vol. 111, Issue 35
  • DOI: 10.1073/pnas.1304215111

XSEDE: Accelerating Scientific Discovery
journal, September 2014

  • Towns, John; Cockerill, Timothy; Dahan, Maytal
  • Computing in Science & Engineering, Vol. 16, Issue 5
  • DOI: 10.1109/MCSE.2014.80

Detection and Characterization of Exoplanets and Disks Using Projections on Karhunen-LoÈVe Eigenimages
journal, August 2012


Fundamental Limitations of high Contrast Imaging set by Small Sample Statistics
journal, August 2014


Discovery of a Substellar Companion to the Nearby Debris disk host hr 2562
journal, September 2016

  • Konopacky, Quinn M.; Rameau, Julien; Duchêne, Gaspard
  • The Astrophysical Journal, Vol. 829, Issue 1
  • DOI: 10.3847/2041-8205/829/1/L4

Polarimetry with the Gemini Planet Imager: Methods, Performance at First Light, and the Circumstellar ring Around hr 4796a
journal, January 2015

  • Perrin, Marshall D.; Duchene, Gaspard; Millar-Blanchaer, Max
  • The Astrophysical Journal, Vol. 799, Issue 2
  • DOI: 10.1088/0004-637X/799/2/182

Improving and Assessing Planet Sensitivity of the GPI Exoplanet Survey with a Forward Model Matched Filter
journal, June 2017

  • Ruffio, Jean-Baptiste; Macintosh, Bruce; Wang, Jason J.
  • The Astrophysical Journal, Vol. 842, Issue 1
  • DOI: 10.3847/1538-4357/aa72dd

Apodized Pupil lyot Coronagraphs for Arbitrary Apertures. iii. Quasi-Achromatic Solutions
journal, February 2011

  • Soummer, Rémi; Sivaramakrishnan, Anand; Pueyo, Laurent
  • The Astrophysical Journal, Vol. 729, Issue 2
  • DOI: 10.1088/0004-637X/729/2/144

Computer vision applications for coronagraphic optical alignment and image processing
journal, January 2013

  • Savransky, Dmitry; Thomas, Sandrine J.; Poyneer, Lisa A.
  • Applied Optics, Vol. 52, Issue 14
  • DOI: 10.1364/AO.52.003394