Technology

An earlier article derived the Knowledge Investment Curve.

Information sharing is an integral part of the R&D process. Thus, decision makers affect the pace of scientific progress when they determine the fraction of R&D dollars dedicated to sharing knowledge. Think of it this way: A program for sharing knowledge derived from scientific research has much in common with a scientific research program itself in that they share the common goal of advancing science. 

by David Wojick and Peter Lincoln

As OSTI Director Walt Warnick likes to say, today's Web is like the Model T Ford -- revolutionary but ready for vast improvement. This is especially true when it comes to making the Web work for science and technology. In that spirit I want to describe a new kind of Web Portal, one which has yet to be built. It is called the X-Portal.

by David Kaiser (MIT) and Luis Bettencourt (LANL)

For some time, we and OSTI have been interested in the question of how new scientific ideas spread. What does it take for the “next big thing” to leap from one person’s head to an active community of researchers? Do those shared ideas or techniques bind the community together more tightly than before, perhaps even helping to define a new research field that didn’t exist before? And if so, how might we detect and measure such shifts in the space of researchers and ideas

OSTI is driven! We are fully committed to providing scientists and researchers with the social networking tools and services that can make it easier for them to more rapidly advance their scientific research. We have a number of exciting ongoing initiatives in support of accelerating the evolution of science. Here are ten that come to mind:

by Walt Warnick and Sol Lederman

OSTI has embraced a new paradigm for sharing scientific and technical information (STI). Historically, OSTI has fulfilled its mission of providing STI to scientists, researchers, and the public by hosting, or collecting, documents and/or metadata. OSTI's new paradigm is to make content searchable that is often hosted by others; today, OSTI connects those seeking the content with the organizations that host it.

by Walt Warnick and Sol Lederman

While federated search is a core technology that OSTI employs to tackle challenges of sharing knowledge, the technology isn't perfect.  OSTI aggressively uses federated search because it does what no other search technology can do—inexpensively making dozens of non-Googleable databases searchable via a single query.  Two nagging limitations of federated search are that it can take 30 seconds to execute a search—which seems slow in the digital age—and hit lists are not exhaustive. 

by Walt Warnick and David Wojick

The life of every person in the world today has been shaped by successive technological transformations. The printing press transformed communication and education, beginning in the mid 15th century. Sailing and navigation technology of the late 15th century allowed Europeans to learn about other continents, beginning the global network of trade. Metal tools and firearms technology of the early 17th century enabled Europeans to colonize other continents and spread the fruits of European technology around the world. Railroads transformed transportation beginning in the early 19th century, and the telephone transformed communication in the latter part of that century. The automobile transformed transportation beginning early in the 20th century. These are but a few of the notable transformations that profoundly reshaped the way people live.

Today, all of OSTI's information products are on the web. This is in sharp contrast to the situation as recently as the mid-1990s, when OSTI had no products on the web.

First becoming popular in 1994, the web quickly emerged as a transformational technology, and its potential for reshaping OSTI was apparent. Recognizing the opportunity to advance the OSTI mission, OSTI set out to capitalize on it as quickly as resources would allow by producing web applications to disseminate all manner of scientific and technical information (STI). A steady progression of new OSTI products addressed the various forms of STI: technical reports, e-prints, conference proceedings, accomplishments, patents, and project descriptions . To make it easy for users who want to search through all these products at once, we introduced the DOE Science Accelerator, which is powered by our special web architecture called federated search. Reaching out beyond DOE, we initiated a collaboration with other agencies to allow users to search their R&D results along with DOE's; thus emerged Science.gov. Most recently, we took collaboration world wide by federating the best information sources from governments around the world, WorldWideScience which makes searchable about the same quantity of science as does Google.

This is the third, and final, article in a series. The first article provided an overview of the E-print Network. The second article discussed the special harvested component of the E-print Network in depth. This article provides a tour of the E-print collections which are federated. Hopefully, once you finish reading this article and this series, you will appreciate the innovation and hard work that has gone into producing the premier federated search application for searching E-prints.

by Walt Warnick and Sol Lederman

This is the final article in a series about the limitations of the crawl and index approach to searching scientific content and the advantages of federated search. Part 1 identified a number of issues with Google and the other crawlers, and showed why researchers and the science attentive citizenry don't rely on "Googleable" content to meet their needs for quality scientific and technical information. Part 2 explained how federated search, by providing access to "non-Googleable" content, overcomes the Google limitations. This article highlights three important applications, developed and maintained by OSTI, that demonstrate how federated search is going beyond crawling to advance science.

by Walt Warnick and Sol Lederman

This is the second in a three part series of articles about the deficiencies of web crawling and indexing, the superiority of federated search to the serious researcher, and the value of OSTI federated search applications in advancing science. Part 1 identified a number of serious limitations of Google and the other crawlers. This article shows how federated search overcomes these limitations. The final article in the series highlights a number of federated search applications and databases that OSTI makes available to the public.

by Walt Warnick and Sol Lederman

The web is growing.

For providing searchable access to the content that matters the most to scientists and researchers, Google and the other web crawlers can't keep up. Instead, growing numbers of scientists, researchers, and science attentive citizens turn to OSTI's federated search applications for high quality research material that Google can't find. And, given fundamental limitations on how web crawlers find content, those conducting research will derive even more benefit from OSTI's innovation and investment in federated search in the coming years.

In Part 1 of this series I provided an overview of the technology that drives the E-print Network. In this article I will provide some detail about how the harvested collection, the "E-prints on Web Sites" component of the E-print Network, is constructed. In Part 3, I will discuss the technology of the portion of the E-print Network that relies on federated search of databases.

 Part of OSTI's R&D aims at understanding how scientists use information. This goal was originally articulated by OSTI's Thurman Whitson, who has since retired. To that end we have begun to look at the different kinds of information provided by the different Web-based science resources. Different kinds of information imply different uses. It is not that one resource is better than another overall, it is that they are very different and support different uses.

The E-print Network is one of OSTI's most popular and powerful research offerings yet few of its users know about the advanced technology that drives it and makes it simple to use. Professional researchers in basic and applied science are able to access over 5 million e-prints gathered from nearly 28,000 world-wide databases and web-sites. Numerous OSTI innovations ensure that the E-print Network's documents are of extremely high quality, are highly relevant to researchers, and are easy and quick to find. This is the first in a series of articles about the technology behind this very important component of the Science Accelerator. This article serves as an overview; subsequent articles will provide more technical information.

Federated search is very much at the heart of OSTI's ability to realize its mission. OSTI provides a simple description of what federated search is and how it works in the OSTI environment. The best way to experience the tremendous value of federated search at OSTI is to try several of OSTI's flagship applications:

• Science Accelerator


• Science.gov


• WorldWideScience.org
  
  

OSTI provides searchable access to vast collections of DOE research results, project descriptions, and accomplishments. Making R&D findings available and usable is central to OSTI's mission:

The mission of the Office of Scientific and Technical Information (OSTI) is to advance science and sustain technological creativity by making R&D findings available and useful to Department of Energy (DOE) researchers and the American people.