# Eliminate crawling of full-text documents. # 7-30-2001 Neal Ensor User-agent: * Disallow: /disconps.html # 02/23/2007 NE removed by request for Jeff Given #Disallow: /servlets/purl/ #Disallow: /gpo/servlets/purl/ #Disallow: /doe/servlets/purl/ #Disallow: /etde/servlets/purl/ #Disallow: /dublincore/servlets/purl/ #Disallow: /infobridge/servlets/purl/ #Disallow: /dublincore/gpo/servlets/purl/ # WH 04/16/2007 Comment out innovation #Disallow: /innovation/research/ # 11/10/2006 # 01/30/2007 NE took out restrictions on ECD per Jeff Given #Disallow: /energycitations/product.biblio.jsp #Disallow: /energycitations/servlets/purl/ Disallow: /energycitations/searchresults.jsp Disallow: /bridge/searchresults.jsp Disallow: /energycitations/header.jsp # 07/13/2007 NE take out access to single-page searching Disallow: /bridge/singlepagesearch.jsp Disallow: /energycitations/singlepagesearch.jsp # 11/04/2008 WH Accomplishments pdfs are redirected Disallow: /accomplishments/pdf/ # 06/14/2010 Disallow: /opennet/document/ # 08-10-2010 NE by request Disallow: /bridge/servlets/purl/10124657-jJc0rL/ Disallow: /bridge/servlets/purl/10124657-jJc0rL/index.jsp Disallow: /bridge/servlets/purl/10124657-jJc0rL/10124657.pdf Disallow: /bridge/servlets/purl/5693826-AHoMmd/ Disallow: /bridge/servlets/purl/5693826-AHoMmd/5693826.pdf # 09-11-2010 NE by request Disallow: /bridge/servlets/purl/29371-bi6yXR/webviewable/ # 5-24-2011 NE do not allow access to webinator (from Lance) Disallow: /cgi-bin/texis/webinator # 9-20-2011 NE adding more crawler exclusions Disallow: /energycitations/excel/ Disallow: /energycitations/downloads/ Disallow: /bridge/downloads/ # 1-30-2012 NE added for Sol Disallow: /cgi-bin/eprints/redirectEprintsUrl # 03-14-2012 WH Retired products Disallow: /graylit/ Disallow: /RETIRED/ # 5-4-2012 NE block ERSD admin Disallow: /ersdadmin/ # 5-8-2013 NE block parts of scitech Disallow: /scitech/excel.jsp # 6-26-2012 NE adding site maps for this domain Sitemap: http://www.osti.gov/eprints/topicpages/sitemap.xml Sitemap: http://www.osti.gov/eprints/topicpages/sitemap-2.xml Sitemap: http://www.osti.gov/eprints/topicpages/sitemap-3.xml Sitemap: http://www.osti.gov/eprints/topicpages/sitemap-4.xml Sitemap: http://www.osti.gov/eprints/topicpages/sitemap-5.xml Sitemap: http://www.osti.gov/eprints/topicpages/documents/sitemap.xml Sitemap: http://www.osti.gov/sciencelab/sites/www.osti.gov.sciencelab/sitemaps/sitemap.xml Sitemap: http://www.osti.gov/bridge/sitemap Sitemap: http://www.osti.gov/energycitations/sitemap User-agent: Slurp Crawl-delay: 5 # 03/18/2010 WH User-agent: dotbot Disallow: /