skip to main content

DOE PAGESDOE PAGES

Title: Overview of the interactive task in BioCreative V

Fully automated text mining (TM) systems promote efficient literature searching, retrieval, and review but are not sufficient to produce ready-to-consume curated documents. These systems are not meant to replace biocurators, but instead to assist them in one or more literature curation steps. To do so, the user interface is an important aspect that needs to be considered for tool adoption. The BioCreative Interactive task (IAT) is a track designed for exploring user-system interactions, promoting development of useful TM tools, and providing a communication channel between the biocuration and the TM communities. In BioCreative V, the IAT track followed a format similar to previous interactive tracks, where the utility and usability of TM tools, as well as the generation of use cases, have been the focal points. The proposed curation tasks are user-centric and formally evaluated by biocurators. In BioCreative V IAT, seven TM systems and 43 biocurators participated. Two levels of user participation were offered to broaden curator involvement and obtain more feedback on usability aspects. The full level participation involved training on the system, curation of a set of documents with and without TM assistance, tracking of time-on-task, and completion of a user survey. Here, the partial level participationmore » was designed to focus on usability aspects of the interface and not the performance per se. In this case, biocurators navigated the system by performing pre-designed tasks and then were asked whether they were able to achieve the task and the level of difficulty in completing the task. In this manuscript, we describe the development of the interactive task, from planning to execution and discuss major findings for the systems tested.« less
Authors:
 [1] ;  [2] ;  [3] ;  [4] ;  [5] ;  [4] ;  [6] ;  [7] ;  [8] ;  [5] ;  [9] ;  [10] ;  [1] ;  [11] ;  [5] ;  [12] ;  [13] ;  [14] ;  [15] ;  [16] more »;  [17] ;  [18] ;  [11] ;  [3] ;  [19] ;  [20] ;  [21] ;  [22] ;  [23] ;  [24] ;  [25] ;  [26] ;  [20] ;  [27] ;  [28] ;  [29] ;  [30] ;  [31] ;  [22] ;  [32] ;  [33] ;  [34] ;  [35] ;  [36] ;  [37] ;  [38] ;  [39] ;  [39] ;  [39] ;  [40] ;  [35] ;  [41] ;  [42] ;  [1] ;  [43] ;  [1] « less
  1. Univ. of Delaware, Newark, DE (United States)
  2. Taipei Medical Univ., Taipei (Taiwan)
  3. Univ. de Santiago, Aveiro (Portugal)
  4. Univ. of Manchester, Manchester (United Kingdom)
  5. Univ. Nacional Autonoma de Mexico, Ciudad de Mexico (Mexico)
  6. BMD Software, Aveiro (Portugal)
  7. Newcastle Univ., New Castle (United Kingdom)
  8. Rutgers Univ. - Camden, Camden, NJ (United States)
  9. Oregon State Univ., Corvallis, OR (United States)
  10. National Taitung Univ., Taitung (Taiwan)
  11. Fraunhofer Institute for Algorithms and Scientific Computing, St. Augustin (Germany)
  12. SourceData, Heidelberg (Germany)
  13. Univ. of Birmingham, Birmingham (United Kingdom); Univ. Hospitals Birmingham NHS Foundation Trust, Birmingham (United Kingdom)
  14. Univ. of Bonn, Bonn (Germany)
  15. Univ. of Copenhagen, Copenhagen (Denmark)
  16. Ecole Polytechnique Federale de Lausanne (EPFL), Geneva (Switzerland)
  17. Univ. of New South Wales NSW, Sydney, NSW (Australia)
  18. SRI International, Menlo Park, CA (United States)
  19. Univ. of Oxford, Oxford (United Kingdom)
  20. Ontario Institute for Cancer Research, Toronto, ON (Canada)
  21. Cardiff Univ., Cardiff (United Kingdom)
  22. Bharathiar Univ., Tamil Nadu (India)
  23. Hellenic Centre for Marine Research, Crete (Greece)
  24. Max Planck Institute for Marine Microbiology, Bremen (Germany)
  25. Georgetown Univ., Washington, D.C. (United States)
  26. Univ. of Zurich, Zurich (Switzerland)
  27. Aix-Marseille Univ., Marseille (France)
  28. Univ. of Aveiro, Aveiro (Portugal)
  29. Taipei Medical Univ. Graduate Institute of Biomedical Informatics, Taipei (Taiwan)
  30. Univ. of Cambridge, Cambridge (United Kingdom)
  31. Academia Sinica, Taipei (Taiwan)
  32. Freelance Scientific Curator, Cleveland, OH (United States)
  33. Univ. of Bedfordshire, Bedford (United Kingdom)
  34. Oregon Health & Science Univ., Portland, OR (United States)
  35. California Inst. of Technology (CalTech), Pasadena, CA (United States)
  36. Univ. de Montreal, Montreal (Canada)
  37. Medical College of Wisconsin, Milwaukee, WI (United States)
  38. Reed Elsevier, Philadelphia, PA (United States)
  39. European Bioinformatics Institute (EMBL-EBI), Hinxton (United Kingdom)
  40. Roche Innovation Center Basel, Basel (Switzerland)
  41. National Institute of Health, Bethesda, MD (United States)
  42. Univ. of Missouri, Columbia, MO (United States)
  43. The MITRE Corp., Bedford, MA (United States)
Publication Date:
Grant/Contract Number:
SC0010838
Type:
Accepted Manuscript
Journal Name:
Database
Additional Journal Information:
Journal Volume: 2016; Journal ID: ISSN 1758-0463
Publisher:
Oxford University Press
Research Org:
Univ. of Deleware, Newark, DE (United States)
Sponsoring Org:
USDOE Office of Science (SC)
Country of Publication:
United States
Language:
English
Subject:
96 KNOWLEDGE MANAGEMENT AND PRESERVATION
OSTI Identifier:
1360096

Wang, Qinghua, Abdul, Shabbir S., Almeida, Lara, Ananiadou, Sophia, Balderas-Martínez, Yalbi I., Batista-Navarro, Riza, Campos, David, Chilton, Lucy, Chou, Hui -Jou, Contreras, Gabriela, Cooper, Laurel, Dai, Hong -Jie, Ferrell, Barbra, Fluck, Juliane, Gama-Castro, Socorro, George, Nancy, Gkoutos, Georgios, Irin, Afroza K., Jensen, Lars J., Jimenez, Silvia, Jue, Toni R., Keseler, Ingrid, Madan, Sumit, Matos, Sérgio, McQuilton, Peter, Milacic, Marija, Mort, Matthew, Natarajan, Jeyakumar, Pafilis, Evangelos, Pereira, Emiliano, Rao, Shruti, Rinaldi, Fabio, Rothfels, Karen, Salgado, David, Silva, Raquel M., Singh, Onkar, Stefancsik, Raymund, Su, Chu -Hsien, Subramani, Suresh, Tadepally, Hamsa D., Tsaprouni, Loukia, Vasilevsky, Nicole, Wang, Xiaodong, Chatr-Aryamontri, Andrew, Laulederkind, Stanley J. F., Matis-Mitchell, Sherri, McEntyre, Johanna, Orchard, Sandra, Pundir, Sangya, Rodriguez-Esteban, Raul, Van Auken, Kimberly, Lu, Zhiyong, Schaeffer, Mary, Wu, Cathy H., Hirschman, Lynette, and Arighi, Cecilia N.. Overview of the interactive task in BioCreative V. United States: N. p., Web. doi:10.1093/database/baw119.
Wang, Qinghua, Abdul, Shabbir S., Almeida, Lara, Ananiadou, Sophia, Balderas-Martínez, Yalbi I., Batista-Navarro, Riza, Campos, David, Chilton, Lucy, Chou, Hui -Jou, Contreras, Gabriela, Cooper, Laurel, Dai, Hong -Jie, Ferrell, Barbra, Fluck, Juliane, Gama-Castro, Socorro, George, Nancy, Gkoutos, Georgios, Irin, Afroza K., Jensen, Lars J., Jimenez, Silvia, Jue, Toni R., Keseler, Ingrid, Madan, Sumit, Matos, Sérgio, McQuilton, Peter, Milacic, Marija, Mort, Matthew, Natarajan, Jeyakumar, Pafilis, Evangelos, Pereira, Emiliano, Rao, Shruti, Rinaldi, Fabio, Rothfels, Karen, Salgado, David, Silva, Raquel M., Singh, Onkar, Stefancsik, Raymund, Su, Chu -Hsien, Subramani, Suresh, Tadepally, Hamsa D., Tsaprouni, Loukia, Vasilevsky, Nicole, Wang, Xiaodong, Chatr-Aryamontri, Andrew, Laulederkind, Stanley J. F., Matis-Mitchell, Sherri, McEntyre, Johanna, Orchard, Sandra, Pundir, Sangya, Rodriguez-Esteban, Raul, Van Auken, Kimberly, Lu, Zhiyong, Schaeffer, Mary, Wu, Cathy H., Hirschman, Lynette, & Arighi, Cecilia N.. Overview of the interactive task in BioCreative V. United States. doi:10.1093/database/baw119.
Wang, Qinghua, Abdul, Shabbir S., Almeida, Lara, Ananiadou, Sophia, Balderas-Martínez, Yalbi I., Batista-Navarro, Riza, Campos, David, Chilton, Lucy, Chou, Hui -Jou, Contreras, Gabriela, Cooper, Laurel, Dai, Hong -Jie, Ferrell, Barbra, Fluck, Juliane, Gama-Castro, Socorro, George, Nancy, Gkoutos, Georgios, Irin, Afroza K., Jensen, Lars J., Jimenez, Silvia, Jue, Toni R., Keseler, Ingrid, Madan, Sumit, Matos, Sérgio, McQuilton, Peter, Milacic, Marija, Mort, Matthew, Natarajan, Jeyakumar, Pafilis, Evangelos, Pereira, Emiliano, Rao, Shruti, Rinaldi, Fabio, Rothfels, Karen, Salgado, David, Silva, Raquel M., Singh, Onkar, Stefancsik, Raymund, Su, Chu -Hsien, Subramani, Suresh, Tadepally, Hamsa D., Tsaprouni, Loukia, Vasilevsky, Nicole, Wang, Xiaodong, Chatr-Aryamontri, Andrew, Laulederkind, Stanley J. F., Matis-Mitchell, Sherri, McEntyre, Johanna, Orchard, Sandra, Pundir, Sangya, Rodriguez-Esteban, Raul, Van Auken, Kimberly, Lu, Zhiyong, Schaeffer, Mary, Wu, Cathy H., Hirschman, Lynette, and Arighi, Cecilia N.. 2016. "Overview of the interactive task in BioCreative V". United States. doi:10.1093/database/baw119. https://www.osti.gov/servlets/purl/1360096.
@article{osti_1360096,
title = {Overview of the interactive task in BioCreative V},
author = {Wang, Qinghua and Abdul, Shabbir S. and Almeida, Lara and Ananiadou, Sophia and Balderas-Martínez, Yalbi I. and Batista-Navarro, Riza and Campos, David and Chilton, Lucy and Chou, Hui -Jou and Contreras, Gabriela and Cooper, Laurel and Dai, Hong -Jie and Ferrell, Barbra and Fluck, Juliane and Gama-Castro, Socorro and George, Nancy and Gkoutos, Georgios and Irin, Afroza K. and Jensen, Lars J. and Jimenez, Silvia and Jue, Toni R. and Keseler, Ingrid and Madan, Sumit and Matos, Sérgio and McQuilton, Peter and Milacic, Marija and Mort, Matthew and Natarajan, Jeyakumar and Pafilis, Evangelos and Pereira, Emiliano and Rao, Shruti and Rinaldi, Fabio and Rothfels, Karen and Salgado, David and Silva, Raquel M. and Singh, Onkar and Stefancsik, Raymund and Su, Chu -Hsien and Subramani, Suresh and Tadepally, Hamsa D. and Tsaprouni, Loukia and Vasilevsky, Nicole and Wang, Xiaodong and Chatr-Aryamontri, Andrew and Laulederkind, Stanley J. F. and Matis-Mitchell, Sherri and McEntyre, Johanna and Orchard, Sandra and Pundir, Sangya and Rodriguez-Esteban, Raul and Van Auken, Kimberly and Lu, Zhiyong and Schaeffer, Mary and Wu, Cathy H. and Hirschman, Lynette and Arighi, Cecilia N.},
abstractNote = {Fully automated text mining (TM) systems promote efficient literature searching, retrieval, and review but are not sufficient to produce ready-to-consume curated documents. These systems are not meant to replace biocurators, but instead to assist them in one or more literature curation steps. To do so, the user interface is an important aspect that needs to be considered for tool adoption. The BioCreative Interactive task (IAT) is a track designed for exploring user-system interactions, promoting development of useful TM tools, and providing a communication channel between the biocuration and the TM communities. In BioCreative V, the IAT track followed a format similar to previous interactive tracks, where the utility and usability of TM tools, as well as the generation of use cases, have been the focal points. The proposed curation tasks are user-centric and formally evaluated by biocurators. In BioCreative V IAT, seven TM systems and 43 biocurators participated. Two levels of user participation were offered to broaden curator involvement and obtain more feedback on usability aspects. The full level participation involved training on the system, curation of a set of documents with and without TM assistance, tracking of time-on-task, and completion of a user survey. Here, the partial level participation was designed to focus on usability aspects of the interface and not the performance per se. In this case, biocurators navigated the system by performing pre-designed tasks and then were asked whether they were able to achieve the task and the level of difficulty in completing the task. In this manuscript, we describe the development of the interactive task, from planning to execution and discuss major findings for the systems tested.},
doi = {10.1093/database/baw119},
journal = {Database},
number = ,
volume = 2016,
place = {United States},
year = {2016},
month = {9}
}