Office of Scientific and Technical Information
Mobile Gallery Subscribe to RSS OSTIblog Get Widgets Get Alert Services
OSTI Facebook OSTI Twitter Google + Bookmark and Share (Link will open in a new window) Go to Videos
Science Accelerator
Find DOE Collections

August 2009


Searching OSTI products using the deep web.

Listen Now

If you search on the internet using commercial search engines such as Google or Yahoo or Microsoft's search engine, you are typically searching what is known as the surface web. These are pages that are able to be crawled by these search engines and indexed by these search engines. When you do a search on those commercial engines, then its automatically gathering information on what they are able to crawl.


There is another part of the internet which is actually many believe is 10 to a hundred to a thousands times larger that the surface web, called the deep web. And this is where a lot of the most significant scientific information resides in databases in the deep web. And, so, this information is generally considered to be 'non-googleable', to use a colloquial term about the capabilities of certain search engines to search in the deep web. And it is federated searching technology that OSTI has pioneered along with many of our partners to make accessible this information in the deep web that is not Googleable.

At OSTI we use this federated searching technology in our web products such as Science.gov, ScienceAccelerator, WorldWideScience.org to reach this non-googleable information. And in many ways, these are sort of cascading products.WorldWideScience.org is using federated searching technology to search non-googleable information all across the world. Science.gov is using federated searching technology to search non-googleable information across U.S. science agencies, the federal agencies. And we have the ScienceAccelerator at OSTI which unifies OSTI's web products. We put it under this, what we call sort or our goober product, that is the ScienceAccelerator allowing deep web access to all of OSTI's databases simultaneously.

 

Some links on this page may take you to non-federal websites. Their policies may differ from this site.