skip to main content


Title: Crowdsourcing quality control for Dark Energy Survey images

We have developed a crowdsourcing web application for image quality control employed by the Dark Energy Survey. Dubbed the "DES exposure checker", it renders science-grade images directly to a web browser and allows users to mark problematic features from a set of predefined classes. Users can also generate custom labels and thus help identify previously unknown problem classes. User reports are fed back to hardware and software experts to help mitigate and eliminate recognized issues. We report on the implementation of the application and our experience with its over 100 users, the majority of which are professional or prospective astronomers but not data management experts. We discuss aspects of user training and engagement, and demonstrate how problem reports have been pivotal to rapidly correct artifacts which would likely have been too subtle or infrequent to be recognized otherwise. We conclude with a number of important lessons learned, suggest possible improvements, and recommend this collective exploratory approach for future astronomical surveys or other extensive data sets with a sufficiently large user base. We also release open-source code of the web application and host an online demo versionat
ORCiD logo [1]
  1. The Ohio State Univ., Columbus, OH (United States). et al.
Publication Date:
Report Number(s):
arXiv:1511.03391; FERMILAB-PUB-15-487; BNL-113170-2016-JA
Journal ID: ISSN 2213-1337; 1476392
Grant/Contract Number:
AC02-07CH11359; SC00112704; FG02/91ER40690; AC02/98CH10886; PHYS/1066293; AST/1138766; AYA2012/39559; ESP2013/48274; FPA2013/47986; SEV/2012/0234; AC02-06CH11357
Accepted Manuscript
Journal Name:
Astronomy and Computing
Additional Journal Information:
Journal Volume: 16; Journal Issue: C; Journal ID: ISSN 2213-1337
Research Org:
Argonne National Lab. (ANL), Argonne, IL (United States); Brookhaven National Laboratory (BNL), Upton, NY (United States); SLAC National Accelerator Lab., Menlo Park, CA (United States); Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
Sponsoring Org:
USDOE Office of Science (SC), High Energy Physics (HEP) (SC-25); National Science Foundation (NSF); USDOE Office of Science (SC), Basic Energy Sciences (BES) (SC-22); Science and Technology Facilities Council; University of Illinois-Urbana-Champaign - National Center for Supercomputing Applications; University of Chicago - Kavli Institute for Cosmological Physics
Contributing Orgs:
Country of Publication:
United States
79 ASTRONOMY AND ASTROPHYSICS; Surveys; Information systems Crowdsourcing; Human-centered computing Collaborative filtering; 97 MATHEMATICS AND COMPUTING; Human-centered computing: Collaborative filtering; Information systems: Crowdsourcing; surveys
OSTI Identifier:
Alternate Identifier(s):
OSTI ID: 1336176; OSTI ID: 1346698; OSTI ID: 1356682