OSTI's Web Metrics - Partially Measuring Success

OSTI creates and deploys web-based information products to accomplish its mission.  One way to measure the success of this approach is to use web metrics to gauge and analyze the usage of the information we disseminate via our web- based products.

An “information transaction” is the largest and most broadly defined web metric we track at OSTI. We define an information transaction as a discrete information exchange between an information patron and OSTI’s suite of web-based information services. An information transaction occurs when our web servers deliver information to satisfy a user’s request.  Requests can take a variety of forms.  Sometimes a user’s request might be a search of a product such as Science Accelerator that brings back a hit list.  Following up on this information transaction, a user might click on an item in the hit list to call up a single bibliographic citation delivered via a component database of Science Accelerator such as the Information Bridge or Energy Citations Database.  Next, the user might call up the full text of a technical report.  Another type of information transaction would occur when a user requests tens, hundreds, or thousands of bibliographic citations via our OAI or MARC xml services.

In 1994, OSTI launched and hosted the first DOE web home page. From this small web presence, OSTI served 300,000 information transactions. In FY 2008, OSTI served over 84,000,000 information transactions, a 28,000% increase.

While information transactions are of great use and give us a great metric relative to information dissemination, it is useful to more closely examine certain kinds of transactions.  One such metric is full-text downloads. We define a full-text download as the delivery of a textual document to an information patron. In FY 2003, on the Information Bridge product, OSTI’s premier product for DOE research report literature with full text availability, over 225,000 full-text downloads occurred.  In FY 2008, 12.5 million downloads occurred, and through August in FY 2009 over 25.5 million downloads occurred.

You can tell a lot about OSTI’s success in web-based dissemination from just these two metrics. However, much of our web-based dissemination cannot be measured. For example, one of the tenets of Science.gov is to introduce users to the underlying sources of data that feed the Science.gov federated search. This introduction to specific databases benefits users who may have a particular interest in the information provided by one agency. Therefore, after performing a search on Science.gov and discovering the underlying agency database, the user may go that database and not return to Science.gov. While they may repeatedly use the underlying agency database that they found through Science.gov, our metrics here at OSTI will never show this usage.  Yet, our goal to disseminate information as widely as possible is being achieved.

Another example of our “untraceable success” occurs when our content is downloaded and subsequently distributed by another information dissemination vehicle. Especially when users download a significant body of technical reports, it is safe to assume that they are motivated by some subsequent use of the downloaded information.  Here again we are accomplishing our mission each time a document that was originally downloaded from OSTI is distributed downstream, even though we cannot track these downstream metrics.

Web metrics are not the only measure of success we are interested in, although they are a key part of telling the story of our growth relative to information dissemination in the Internet Age. 

Mark Martin

OSTI