Usability 101: A Pilot Test of DOE PAGES^Beta

The Department of Energy (DOE) Office of Scientific and Technical Information (OSTI) is committed to developing science search tools that meet the needs of users and support the dissemination of DOE research and development (R&D) results.  In August 2014, the Department launched the DOE Public Access Gateway for Energy and Science^Beta (DOE PAGES^Beta), a portal to journal articles and accepted manuscripts resulting from DOE-funded research that was developed and is maintained by OSTI.  Soon after the launch of DOE PAGES^Beta, OSTI invited a group of key stakeholders to provide what proved to be valuable feedback about the resource.  In November 2015, as part of OSTI's efforts to continuously improve our scientific and technical information offerings, a team of OSTI librarians conducted a more formal pilot usability study of DOE PAGES^Beta to gauge user satisfaction.  The study included evaluating current search features, ease of navigation, and the overall site experience, with the end goal of identifying usability issues to inform enhancements. 

So what is usability testing?  It is essentially making sure something works well that a person of average experience can use the product for its intended purpose without excessive hindrance and frustration.  Products created with usability in mind are helpful to the target audience and easy to learn and navigate, and they guide the user efficiently and effectively to the information they need.  Our goal is to ensure that DOE PAGES^Beta meets all those criteria.  To that end, we defined our target audiences and asked selected members of those audiences to participate in a guided exploration of DOE PAGES^Beta while OSTI staff observed.  We also invited feedback regarding their needs and expectations of the product.  We recorded our observations and their feedback to help identify needed improvements to DOE PAGES^Beta.

The pilot usability test was conducted at the University of Tennessee College of Communication and Information-Oak Ridge National Laboratory User eXperience Laboratory in Knoxville.  We recruited volunteer participants with backgrounds and abilities representative of our target audiences:  researchers and the scientific community, librarians, and the general public.  The test was based on the "think aloud" methodology, a technique designed to elicit spontaneous thoughts, impressions, and suggestions from participants; they were instructed to complete tasks on DOE PAGES^Beta and to "think out loud" as they interacted with the product.  They were given basic instructions and then left alone to complete the tasks.  Software was used to record both the participants' on-screen actions as well as audio and video of the individuals as they completed their tasks. 

Participants were asked to complete six tasks aimed at determining the ease of use of some of the basic search functions as well as some of the more specialized features of DOE PAGES^Beta.  These included having a participant find articles by a specific author, generating a citation for a specific record, and finding and downloading an accepted manuscript.  Finally, participants were given a short post-test questionnaire to elicit feedback about their overall impression of the site, with questions such as "How easy was it to navigate the site?" "What did you like most about the site?" and "Is there anything you would change?"

Overall, the results of the test were encouraging, with all six participants able to complete the six assigned tasks with little to no difficulty.  Most of the participants expressed satisfaction with the navigation of DOE PAGES^Beta, stating that it was easy to search and retrieve relevant records.  There was also constructive feedback from the participants, a very important component to any usability study; these comments are being addressed and will help direct enhancements and improvements to DOE PAGES^Beta and OSTI's other science search tools. 

Usability studies are iterative processessoliciting feedback, making changes, then going back to the users to invite responses to these changes.  This pilot study has been a great learning experience for OSTI, and it will be used to inform future usability studies on other search tools such as SciTech Connect, OSTI's primary repository for DOE science, technology, and engineering research information.  Meanwhile, OSTI encourages users to provide feedback about our science search tools via the contact link at each of our product sites.  Comments and suggestions are always appreciated and will help contribute to ongoing improvements to OSTI's collections.