August 2009

Listen Now
If you search on the internet using commercial search engines such as Google or Yahoo or Microsoft's search engine, you are typically searching what is known as the surface web. These are pages that are able to be crawled by these search engines and indexed by these search engines. When you do a search on those commercial engines, then its automatically gathering information on what they are able to crawl.
There is another part of the internet which is
actually many believe is 10 to a hundred to a thousands
times larger that the surface web, called the deep web.
And this is where a lot of the most significant scientific
information resides in databases in the deep web.
And, so, this information is generally considered to be
'non-googleable', to use a colloquial term about the
capabilities of certain search engines to search in the deep
web. And it is federated searching technology that OSTI has
pioneered along with many of our partners to make accessible
this information in the deep web that is not Googleable.
At OSTI we use this federated searching technology in our web products such as Science.gov, ScienceAccelerator, WorldWideScience.org to reach this non-googleable information. And in many ways, these are sort of cascading products.WorldWideScience.org is using federated searching technology to search non-googleable information all across the world. Science.gov is using federated searching technology to search non-googleable information across U.S. science agencies, the federal agencies. And we have the ScienceAccelerator at OSTI which unifies OSTI's web products. We put it under this, what we call sort or our goober product, that is the ScienceAccelerator allowing deep web access to all of OSTI's databases simultaneously.


