Open Innovation Enabled by Global Networking of
Science and Technical Knowledge
Collaborative Expedition Workshop
National Science Foundation, Arlington, VA
Walt Warnick, Ph.D.
Director, Office of Scientific and Technical Information
U.S. Department of Energy
OSTI Mission
To advance science and sustain technological creativity by making R&D findings available and useful to DOE researchers and the public
Science progresses as knowledge is shared
OSTI Corollary: If the sharing of knowledge is accelerated, then discovery is accelerated
"If I have seen further, it is by standing on the shoulders of giants."
– Isaac Newton 1676
Profound implications for everyone in the information business.
How do we help researchers find the information they need for making life-altering discoveries?
We need innovative discovery tools to help researchers find the information they need
Often the knowledge scientists need resides in distant communities
• Genomics
• Spectroscopy
• Materials Science
• Nanoscience
• Cosmology
• Cognitive Neuroscience
• Radiology
• Meterology
• Sterochemistry
• Immunology
• Bioinformatics
• Limnology
• Astrodynamics
• Cybernetics
• Oceangraphy
• Condensed Matter Physics
• Plasma Physics
• Seismology
• Geology
• Acoustics
• Agricultural Engineering
• Cryogenics
… because traditional "discovery tools" are not yet doing the job
We must dispel the misperception that traditional search engines can find whatever we seek
• Google
• Yahoo!
• bing msn LiveSearch
Much of science is non-Googleable
The Deep Web is Huge
Surface Web Deep Web
Most STI is in databases in the Deep Web
– govenment databases
– journals
In fact, the vast majority of science information is in databases within the deep web – or the non-Googleable web – where popular search engines cannot go
We in the information business need to recognize this gap between availability and need, and seize the opportunity to …
Provide science information consumers with better tools
The web is transformational technology for sharing knowledge
The web is still young and will certainly hold surprises as it evolves
Just as another well-known transformational technology held surprises …
1903
1918
2010
Eclipsing Current Search Technology
Google is capitalizing on this early era of web technology and is hugely successful, powering more than half the world’s searching
But we must remember that we are just in the beginning of this transformation. Further technological transformations may very well eclipse today’s search technology!
A new, promising technology is now emerging: federated search
We need systems, such as federated search, that probe the deep web
Federated search drills down to the deep web where scientific databases reside
(1) A user types in a query, hits send
(2) Query travels to server in Oak Ridge, TN
(3) In Oak Ridge, the query is replicated 61 times and fanned out to servers of national databases around the globe
(4) At each National server, a search is executed in real time
(5) Results transit back to Oak Ridge where they are collected & sorted
(6) A relevant, authoritative result list is returned to the user
Federated search involves a series of steps to create a novel web architecture:
OSTI federated search solutions
Science Accelerator is the DOE contribution to Science.gov
Integrates key DOE databases
Science.gov is the U.S. contribution to WorldWideScience.org
Integrates 14 U.S. science agencies – 200 million pages of science information
WorldWideScience.org
Integrates science information issued by over 60 Nations – 400 million pages of global science information
OSTI, through federated search, ensures access to non-Googleable science
Volume of Content Made Searchable by OSTI
WorldWideScience.org:
400,000,000 pages of Global Scientific and Technical Information (STI)
These web-available pages would fill 62,000 traditional 2-feet deep file drawers.
Science.gov:
200,000,000 pages of U.S. Government STI
These web-available pages would fill 33,000 traditional 2-feet deep file drawers.
STIP Collection:
11,400,000 pages of U.S. Department of Energy STI
These web-available pages would fill 1,900 traditional 2-feet deep file drawers.
Amount of Data Transferred in FY08: 9.95 terabytes
Through OSTI products, librarians, researchers and the public can access a science page count comparable to, but not duplicative of, Google's entire science content
WorldWideScience.org History
Concept introduced by OSTI Director, Walt Warnick, June 2006, Bethesda, Maryland
Bilateral U.S.(DOE)/U.K. (British Library) partnership, January 2007, London
Demonstration of first prototype, June 2007, Nancy, France
Multilateral governance structure WorldWideScience Alliance, established June 2008, Seoul
Common ingredient: International Council for Scientific and Technical Information (ICSTI)
• Searches 61 science databases and portals sponsored by governments and national institutions in 61 countries
• Covers scientific literature from over three-fourths of the world’s population
• Includes a vast quantity of science (over 400 million pages), much of which is grey literature
• Proving WWS "deep web" value, recent analysis shows only 3.5% overlap with Google and Google Scholar
• Current research in multilingual translations technologies will enable searching of non-English databases from within applications such as WWS
• Prototype allows users to select their preferred language. Queries are translated into the languages of the databases being searched and results are then returned in the user's language
• We are committed to launching multilingual WorldWideScience.org at the ICSTI Meeting in Helsinki in June 2010
Accelerates scientific networking
• A scientist in the United States can find the works of scientists in the same field from 60 other countries—all in a single search
• Alerts feature can keep one scientist apprised of the new work of other specific scientists
• Real-time translations of non- English queries and non-English databases will connect scientists in unprecedented ways
WorldWideScience.org search results
ETDEWEB
Bibliographic Citation
Fulltext Document
Is there a better solution for a high quality science search tool just over the horizon?
We think so …
Live Federated Search Tools + Crawled Indexes
For Example:
WorldWideScience.org + crawled indexes
The stage is set for the future
We are ready to scale up our efforts in federated search
A billion-page, high quality science search tool may be available soon to spread ideas, increase learning, and further accelerate the progress of science.
Cognition Budget
Making more info available is not enough
It must be presented more conveniently – easier and faster to find
To this end, relevancy ranking is being reinvented for federated searching
Try WorldWideScience.org
Simply put, we intend to make more science accessible to more people more conveniently than has ever been done before.
Thumbnail panels:


