catsHTM: A Tool for Fast Accessing and Cross-matching Large Astronomical Catalogs

Soumagnac, Maayane T.; Ofek, Eran O.

doi:10.1088/1538-3873/aac410

catsHTM: A Tool for Fast Accessing and Cross-matching Large Astronomical Catalogs

Journal Article · Tue Jun 12 00:00:00 EDT 2018 · Publications of the Astronomical Society of the Pacific

DOI:https://doi.org/10.1088/1538-3873/aac410· OSTI ID:1543942

Soumagnac, Maayane T. ^[1]; Ofek, Eran O. ^[1]

Weizmann Inst. of Science, Rehovot (Israel)

Fast access to large catalogs is required for some astronomical applications. Here we introduce the catsHTM tool, consisting of several large catalogs reformatted into HDF5-based file format, which can be downloaded and used locally. To allow fast access, the catalogs are partitioned into hierarchical triangular meshes and stored in HDF5 files. Several tools are provided to perform efficient cone searches at resolutions spanning from a few arc-seconds to degrees, within a few milliseconds time. The first released version includes the following catalogs (by alphabetical order): 2MASS, 2MASS extended sources, AKARI, APASS, Cosmos, DECaLS/DR5, FIRST, GAIA/DR1, GAIA/DR2, GALEX/DR6Plus7, HSC/v2, IPHAS/DR2, NED redshifts, NVSS, Pan-STARRS1/DR1, PTF photometric catalog, ROSAT faint source, SDSS sources, SDSS/DR14 spectroscopy, SkyMapper, Spitzer/SAGE, Spitzer/IRAC galactic center, UCAC4, UKIDSS/DR10, VST/ATLAS/DR3, VST/KiDS/DR3, WISE and XMM. We provide Python code that allows to perform cone searches, as well as MATLAB code for performing cone searches, catalog cross-matching, general searches, as well as load and create these catalogs.

View Accepted Manuscript (DOE)

Research Organization:: Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States). National Energy Research Scientific Computing Center (NERSC)

Sponsoring Organization:: USDOE

OSTI ID:: 1543942

Journal Information:: Publications of the Astronomical Society of the Pacific, Journal Name: Publications of the Astronomical Society of the Pacific Journal Issue: 989 Vol. 130; ISSN 0004-6280

Publisher:: Astronomical Society of the Pacific (ASP)Copyright Statement

Country of Publication:: United States

Language:: English

References (31)

SkyMapper Southern Survey: First Data Release (DR1) Wolf, Christian; Onken, Christopher A.; Luvaul, Lance C. Publications of the Astronomical Society of Australia, Vol. 35 https://doi.org/10.1017/pasa.2018.5	journal	January 2018
The AKARI/IRC mid-infrared all-sky survey Ishihara, D.; Onaka, T.; Kataza, H. Astronomy and Astrophysics, Vol. 514 https://doi.org/10.1051/0004-6361/200913811	journal	May 2010
The XMM-Newton serendipitous survey : VII. The third Rosen, S. R.; Webb, N. A.; Watson, M. G. Astronomy & Astrophysics, Vol. 590 https://doi.org/10.1051/0004-6361/201526416	journal	April 2016
Probabilistic multi-catalogue positional cross-match Pineau, F. -X.; Derriere, S.; Motch, C. Astronomy & Astrophysics, Vol. 597 https://doi.org/10.1051/0004-6361/201629219	journal	January 2017
Gaia Data Release 1 : Summary of the astrometric, photometric, and survey properties Brown, A. G. A.; Vallenari, A.; Prusti, T. Astronomy & Astrophysics, Vol. 595 https://doi.org/10.1051/0004-6361/201629512	journal	November 2016
The third data release of the Kilo-Degree Survey and associated data products de Jong, Jelte T. A.; Kleijn, Gijs A. Verdoes; Erben, Thomas Astronomy & Astrophysics, Vol. 604 https://doi.org/10.1051/0004-6361/201730747	journal	August 2017
The VizieR database of astronomical catalogues Ochsenbein, F.; Bauer, P.; Marcout, J. Astronomy and Astrophysics Supplement Series, Vol. 143, Issue 1 https://doi.org/10.1051/aas:2000169	journal	April 2000
The SIMBAD astronomical database: The CDS reference database for astronomical objects Wenger, M.; Ochsenbein, F.; Egret, D. Astronomy and Astrophysics Supplement Series, Vol. 143, Issue 1 https://doi.org/10.1051/aas:2000332	journal	April 2000
The FIRST Survey: Faint Images of the Radio Sky at Twenty Centimeters Becker, Robert H.; White, Richard L.; Helfand, David J. The Astrophysical Journal, Vol. 450 https://doi.org/10.1086/176166	journal	September 1995
The NRAO VLA Sky Survey Condon, J. J.; Cotton, W. D.; Greisen, E. W. The Astronomical Journal, Vol. 115, Issue 5 https://doi.org/10.1086/300337	journal	May 1998
2MASS Extended Source Catalog: Overview and Algorithms Jarrett, T. H.; Chester, T.; Cutri, R. The Astronomical Journal, Vol. 119, Issue 5 https://doi.org/10.1086/301330	journal	May 2000
The Galaxy Evolution Explorer : A Space Ultraviolet Survey Mission Martin, D. Christopher; Fanson, James; Schiminovich, David The Astrophysical Journal, Vol. 619, Issue 1 https://doi.org/10.1086/426387	journal	January 2005
The Two Micron All Sky Survey (2MASS) Skrutskie, M. F.; Cutri, R. M.; Stiening, R. The Astronomical Journal, Vol. 131, Issue 2 https://doi.org/10.1086/498708	journal	February 2006
Spitzer Survey of the Large Magellanic Cloud: Surveying the Agents of a Galaxy?s Evolution (SAGE). I. Overview and Initial Results Meixner, Margaret; Gordon, Karl D.; Indebetouw, Remy The Astronomical Journal, Vol. 132, Issue 6 https://doi.org/10.1086/508185	journal	January 2006
The First Release COSMOS Optical and Near‐IR Data and Catalog Capak, P.; Aussel, H.; Ajiki, M. The Astrophysical Journal Supplement Series, Vol. 172, Issue 1 https://doi.org/10.1086/519081	journal	September 2007
Point Sources from a Spitzer IRAC Survey of the Galactic Center Ramírez, Solange V.; Arendt, Richard G.; Sellgren, Kris The Astrophysical Journal Supplement Series, Vol. 175, Issue 1 https://doi.org/10.1086/524015	journal	March 2008
Probabilistic Cross‐Identification of Astronomical Sources Budavári, Tamás; Szalay, Alexander S. The Astrophysical Journal, Vol. 679, Issue 1 https://doi.org/10.1086/587156	journal	May 2008
The Palomar Transient Factory: System Overview, Performance, and First Results Law, Nicholas M.; Kulkarni, Shrinivas R.; Dekany, Richard G. Publications of the Astronomical Society of the Pacific, Vol. 121, Issue 886 https://doi.org/10.1086/648598	journal	December 2009
Searchable Sky Coverage of Astronomical Observations: Footprints and Exposures Budavári, Tamás; Szalay, Alexander S.; Fekete, György Publications of the Astronomical Society of the Pacific, Vol. 122, Issue 897 https://doi.org/10.1086/657302	journal	November 2010
The Palomar Transient Factory photometric catalog 1.0 Ofek, E. O.; Laher, R.; Surace, J. Publications of the Astronomical Society of the Pacific, Vol. 124, Issue 918 https://doi.org/10.1086/666978	journal	August 2012
The Wide-Field Infrared Survey Explorer (Wise): Mission Description and Initial On-Orbit Performance Wright, Edward L.; Eisenhardt, Peter R. M.; Mainzer, Amy K. The Astronomical Journal, Vol. 140, Issue 6 https://doi.org/10.1088/0004-6256/140/6/1868	journal	November 2010
The Fourth us Naval Observatory ccd Astrograph Catalog (Ucac4) Zacharias, N.; Finch, C. T.; Girard, T. M. The Astronomical Journal, Vol. 145, Issue 2 https://doi.org/10.1088/0004-6256/145/2/44	journal	January 2013
The Tenth data Release of the Sloan Digital sky Survey: First Spectroscopic data from the Sdss-Iii Apache Point Observatory Galactic Evolution Experiment Ahn, Christopher P.; Alexandroff, Rachael; Allende Prieto, Carlos The Astrophysical Journal Supplement Series, Vol. 211, Issue 2 https://doi.org/10.1088/0067-0049/211/2/17	journal	March 2014
C ³ , A Command-line Catalog Cross-match Tool for Large Astrophysical Catalogs Riccio, Giuseppe; Brescia, Massimo; Cavuoti, Stefano Publications of the Astronomical Society of the Pacific, Vol. 129, Issue 972 https://doi.org/10.1088/1538-3873/129/972/024005	journal	December 2016
The second data release of the INT Photometric Hα Survey of the Northern Galactic Plane (IPHAS DR2) Barentsen, Geert; Farnhill, H. J.; Drew, J. E. Monthly Notices of the Royal Astronomical Society, Vol. 444, Issue 4 https://doi.org/10.1093/mnras/stu1651	journal	September 2014
The VLT Survey Telescope ATLAS Shanks, T.; Metcalfe, N.; Chehade, B. Monthly Notices of the Royal Astronomical Society, Vol. 451, Issue 4 https://doi.org/10.1093/mnras/stv1130	journal	June 2015
Finding counterparts for all-sky X-ray surveys with Nway: a Bayesian algorithm for cross-matching multiple catalogues Salvato, M.; Buchner, J.; Budavári, T. Monthly Notices of the Royal Astronomical Society, Vol. 473, Issue 4 https://doi.org/10.1093/mnras/stx2651	journal	October 2017
SkyQuery: Federating Astronomy Archives Budavari, Tamas; Dobos, Laszlo; Szalay, Alexander S. Computing in Science & Engineering, Vol. 15, Issue 3 https://doi.org/10.1109/MCSE.2013.41	journal	May 2013
The UKIRT Infrared Deep Sky Survey (UKIDSS) Lawrence, A.; Warren, S. J.; Almaini, O. Monthly Notices of the Royal Astronomical Society, Vol. 379, Issue 4 https://doi.org/10.1111/j.1365-2966.2007.12040.x	journal	August 2007
Cross Catalogue Matching with Virtual Observatory and Parameterization of Stars Malkov, O.; Dluzhnevskaya, O.; Karpov, S. Open Astronomy, Vol. 21, Issue 3 https://doi.org/10.1515/astro-2017-0390	journal	January 2012
Version 1 of the Hubble Source Catalog Whitmore, Bradley C.; Allam, Sahar S.; Budavári, Tamás The Astronomical Journal, Vol. 151, Issue 6 https://doi.org/10.3847/0004-6256/151/6/134	journal	May 2016

Cited By (6)

Optical follow-up of the neutron star-black hole mergers S200105ae and S200115j Anand, Shreya; Coughlin, Michael W.; Kasliwal, Mansi M. arXiv https://doi.org/10.48550/arxiv.2009.07210	text	January 2020
Astronomical data fusion: recent progress and future prospects — a survey Yu, Ce; Li, Bingyao; Xiao, Jian Experimental Astronomy, Vol. 47, Issue 3 https://doi.org/10.1007/s10686-019-09633-z	journal	June 2019
Transient processing and analysis using AMPEL: alert management, photometry, and evaluation of light curves Nordin, J.; Brinnel, V.; van Santen, J. Astronomy & Astrophysics, Vol. 631 https://doi.org/10.1051/0004-6361/201935634	journal	November 2019
Transient processing and analysis using AMPEL: alert management, photometry, and evaluation of light curves Nordin, J.; Brinnel, V.; Van Santen, J. Deutsches Elektronen-Synchrotron, DESY, Hamburg https://doi.org/10.3204/pubdb-2019-04287	text	January 2019
Toward the Measurement of the Mass of Isolated Neutron Stars: Prediction of Future Astrometric Microlensing Events by Pulsars Ofek, Eran O. The Astrophysical Journal, Vol. 866, Issue 2 https://doi.org/10.3847/1538-4357/aadfeb	journal	October 2018
Transient processing and analysis using $\texttt{AMPEL}$: Alert Management, Photometry and Evaluation of Lightcurves Nordin, J.; Brinnel, V.; van Santen, J. arXiv https://doi.org/10.48550/arxiv.1904.05922	text	January 2019

Similar Records

Census of the Young Stellar Population in the Galactic H ii Region Sh2-242

Journal Article · Wed Apr 01 00:00:00 EDT 2020 · The Astronomical Journal (Online) · OSTI ID:23013157

The Large Sky Area Multi-Object Fibre Spectroscopic Telescope (LAMOST) Quasar Survey: Quasar Properties from Data Release Two and Three

Journal Article · Tue May 01 00:00:00 EDT 2018 · The Astronomical Journal (Online) · OSTI ID:22897468

Related Subjects

79 ASTRONOMY AND ASTROPHYSICS
Astronomy & Astrophysics

catsHTM: A Tool for Fast Accessing and Cross-matching Large Astronomical Catalogs

Citation Formats

References (31)

Cited By (6)

Similar Records

Related Subjects