Page Content

The Harvesting Initiative

OSTI's Harvesting Initiative officially launched in January 2002 when the National Renewable Energy Laboratory (NREL) became the first site to successfully submit metadata via this new process.

Harvesting sites permit OSTI's automated harvesting system to run, on a regular basis, a query with a data range against their web servers.  The query triggers a site-generated program that retrieves new and updated records directly from the site's STI database, converting them, on-the-fly, to XML format.  The records are automatically uploaded into E-Link at OSTI, processed, link-checked, and sent to Energy Citations and other appropriate databases.

After the initial set-up is complete, harvesting sites can realize the following benefits:

• A great saving of staff time, since keying individual records (for unclassified, unlimited information) into E-Link is no longer necessary
• Automatic updating of OSTI's records when new information is added at the site.  The updating and editing are done in the site's database and picked up the next time OSTI's harvester runs
• Assurance that the site's record and OSTI's record will remain in "version" harmony
• Feedback on which records loaded into E-Link and which did not.  The OSTI ID for each record loaded is also provided back to the site.

Details of the harvesting process

Status of sites involved in harvesting

Original Overview Document - 02/2003, Revision 1

Harvesting Diagram: The Data Flow Process

Learn about STIP text-only page

Meeting Notes text-only page

STIP Contacts text-only page

STIAB text-only page

STIP Archive text-only page

Harvesting Initiative text-only page

Reference copy of Subject Categories

Reference copies of all DOE 241 forms and XML DTD

Reference copy of DOE Order 241.1A

Reference copy of Admin Change to DOE Order 241.1A

Current Version of 241.1 Guide

Revised marked-up 241.1 Guide

STIPWorks
(Password Required)

OSTI Home

Authorities

E-Link Home

DOE Home

OSTI's Product Flyers

DOE Order 241.1A on DOE Directives Web

Website Policies and Important Links

Science.gov

USA.gov