Search Architecture: What Is under the Hood?

Science Conference Proceedings makes electronic "papers" and articles searchable from many different conferences. By making all the papers and articles searchable via a single query, Science Conference Proceedings virtually integrates the various conferences. Having performed a search, a user can follow a hyperlink in a hit list. The user then views the paper or article on the server affiliated with that conference.

Thus, Science Conferences allows the information patron to search multiple data sources with a single query from the user interface. While the user gets a seamless, Google-like search and retrieval experience, sophisticated Web technology is used behind the scenes. Such sophistication is necessitated by the fact that different conference organizers use different mechanisms to publish proceedings. The publication mechanisms fall into one of two general categories. First, "papers" and articles can be published in databases or portals. Typically, such databases and portals have their own search engine, and they are often not readily crawled and indexed. Alternatively, "papers" and articles can be published as simple Web site documents. In order to make all the proceedings searchable and retrievable, Science Conference Proceedings implements a blend of federated search to search databases and portals and Web harvesting technologies to make Web site documents searchable.

Federated Search

When the information patron enters a query in the search box, the query is sent to every individual database or portal searched by Science Conference Proceedings. The individual data sources send back to Science Conference Proceedings a list of results from the search query. The information patron can review this hit list and travel to the host site of a particular hit for more detailed information.

Web Harvesting

In addition to this federated approach Science Conference Proceedings also searches an internally maintained index of harvested Web content. This internal index is very specific to the domain of Science Conference Proceedings and the Web addresses, or URLs, which have been indexed have been pre-selected and screened before being added to the internal Science Conference Proceedings index.

Whether the search results come via federated search or Web harvesting, Science Conference Proceedings then ranks the hits and presents them to the user in relevance order.

This process allows Science Conference Proceedings some key advantages when compared with general purpose crawler-based search engines. Federated search does not place any requirements or burdens on owners of the individual data sources, other than handling increased traffic. Federated searches are inherently as current as the individual data sources, as they are searched in real time. Web harvesting-based searches focus exclusively on quality conference proceedings.