Accelerating Science Discovery - Join the Discussion

OSTIblog Articles in the DOE Data Explorer (DDE) Topic

DOE Data Explorer: Restructured and Redesigned to Better Reflect Data Relationships

by Sara Studwell 31 Aug, 2017 in

DOE Data ExplorerResearch data is being produced at a rapidly increasing rate, and we at the Department of Energy (DOE) Office of Scientific and Technical Information (OSTI) recognize the importance of making data findable and accessible and are committed to being active in the data community.  The DOE Data Explorer (DDE), launched in 2008, is a search tool enabling users to locate and access research data resulting from DOE funding.  Additionally, OSTI joined DataCite in 2011 and established the DOE Data ID Service, through which OSTI assigns persistent identifiers, known as digital object identifiers (DOIs), to datasets submitted by DOE and its contractor and grantee researchers to help increase access to digital data from DOE-funded research.

As product manager for DDE, my goal is to ensure that DDE reflects the ways in which researchers organize their data so that users can easily find or discover data of interest to them.  So, after OSTI held meetings with researchers and stakeholders and listened to what they need in a data search and discovery tool, we developed a plan to reorganize and redesign DDE to better reflect inherent relationships between data objects.  I presented this plan at the SciDataCon conference in September 2016 and subsequently co-authored a paper with colleagues Carly Robinson and Jannean Elliott that further explained the steps being taken to update DDE.

Through a multi-phase restructuring, we are creating meaningful contextual relationships between data objects to make the data more accessible; phase I was completed in August 2017, and phase II is...

Related Topics: DOE Data Explorer (DDE)


Exploring DOE’s Data

by Jannean Elliott 15 Dec, 2014 in

Alternate Text PlaceholderI’ve always been a “window shopper.”  I don’t want to go in and find the store directory, follow the little map, go up the escalator and through the racks…unless the window displays tell me it will probably be worth my time.  I tend to approach databases the same way; I want to know what’s in there.  Not only do I want some reassurance that what I need is there, but I also want to see if there’s information I may not have realized I need yet.

If you can relate, then you will love the inside view the DOE Data Explorer (DDE) offers with its (what else!) Explore feature.  Choose an Explore option from the DDE homepage to check out the most recently added content, browse the titles of every dataset or data collection, see which organizations are sponsoring what data, or discover subject areas into which those data are grouped.  The Other Organizations option will show you the originating research organizations and the host websites.

Let’s check out an example.  Maybe you’re interested in astrophysics.  “Doesn’t the Department of Energy focus mainly on natural gas resources or solar power…stuff like that?” you think.  “I should probably check the NASA homepage for astrophysics data.” But first you decide to take a quick peek at the Subject Categories option, and you find “Astronomy and Astrophysics” third from the top of the list.  Selecting that category lists 30 collections of astronomy and astrophysics data described in DDE.  Now you can explore High-Energy Cosmic Ray Event Data from the Pierre...

Related Topics: data, datasets, Digital Object Identifier, explore, search, window shopping, DOE Data Explorer (DDE)


Before and after CrossRef

Doctor at University of Oxford


It is truly wonderful when something comes along that speeds access to science. Such is the case with CrossRef’s linking network for scholarly literature. Anyone that has ever done a literature search prior to 2000 is completely blown away today when they encounter the time saved and the quality of CrossRef’s linking service. I vividly recall my own literature review for my PhD dissertation almost 40 years ago and I want to share my story.

For many long and miserable days and nights for a solid month I practically lived at the University of Maryland’s Engineering and Physical Sciences Library plowing through a massive set of numerous volumes of citation indices looking up keywords related to my dissertation. My topic Secondary deflections and lateral stability of beams was based on my research at the U.S. Naval Research Laboratory. None of my faculty had specialized in such topics and so my task was monumental-- do a full blown literature review from scratch. I would write down suspected relevant citations and walk through the extensive stacks of the library where I could locate the journal, find the right volume of the...

Related Topics: CrossRef, dissertation, dois, FundRef, journal articles, literature review, SciTech Connect, DOE Data Explorer (DDE), ETDEWEB


DOE Data Explorer

Early in May the new design and the expanded search functionalities of the DOE Data Explorer were launched.  The major upgrade continues this month with the addition of customization features that enhance your interaction with the DDE database.  You may be familiar with some of these from OSTI’s other information products.  For example, you can now download retrieved records into an Excel spreadsheet format or create an account to store your searches in a personal “library.”  But brand new is the ability to log on and choose how you wish to view DDE’s search results!  Use the standard list format to get a preview of each record’s abstract/description.  This view, available with or without an account, can be sorted by relevance or alphabetically by title. 

Choose the detailed, tabular display to see more results per page (up to 100 records at a time) and to view at a glance the OSTI ID, title, content type of the data collection or dataset, and the sponsor organization.  Each of the columns can be used as a sorting choice, further increasing the ways you can view your retrieved information.

If you didn’t have time to take a look at the new release of the DOE Data Explorer earlier in the summer, we encourage you to do so now.  Revisit the OSTIblog article of June 20, 2013 to read more about the search capabilities DDE now offers and the larger scope of its content.  And, as always, let us know if you have comments or questions.

Related Topics: non-text information, DOE Data Explorer (DDE)


A (re)Birth Announcement for the DOE Data Explorer

Alternate Text PlaceholderA database and its supporting website can get periodic makeovers and sometimes it can even undergo rebirth!  The DOE Data Explorer (DDE) has just emerged from a rebirth process, and we are proud to announce its transformation.  The first version of DDE was launched in 2008 with the mission of guiding users to collections of publicly available, DOE-sponsored data and other non-text information.  Hundreds of websites were researched in order to find these collections at DOE’s labs, program offices, and user facilities, at data centers, at colleges and universities, on private sector websites such as SciVee, and across all science disciplines.  The mission has not changed, but the content has grown to include individual datasets within collections.  Now DOE boasts a new website design, better navigation, enhanced search functionality, and new features to help you analyze your search results.

The most obvious change in design, of course, is in the color scheme and the clean lines of the new pages.  DDE took inspiration from OSTI’s recently launched SciTech Connect, opting for a design that clearly says “family look and feel.”  An exciting part of the new “feel” appears on the left side of your screen every time you do a search.  Like SciTech Connect, DDE automatically breaks down the results of the search into groupings that allow you to shortcut through a long list of citations and go directly to the subset of your choice.  In DDE the groupings are based on the types of data and non-text items that were retrieved by your search term.  Search on the word “...

Related Topics: data, data sets, datacite, Digital Object Identifier, dois, non-text information, redesign, DOE Data Explorer (DDE)


Exploring DOE Data Treasures

treasure chest

There are databases, and then there are treasure maps. The DOE Data Explorer (DDE) merges the two concepts into a product offering the best of both. DDE’s database provides the features needed for simple retrieval or advanced searching. The treasure map aspect comes from DDE’s content, which links you to collections of data and non-text information wherever those collections reside.

Instead of sailing the seven seas, you can browse DDE’s seven types of content. Choose “Browse by Content Type” from the drop down menu on the DDE homepage and hit the “Submit” button. Each one of the seven categories shown on the results page will open a list of every collection tagged in DDE with that particular type of content. Naturally, “Numeric Files/Datasets” is a content type with a huge “footprint.”  High-Energy Physics (HEP) data from experiments at DOE’s National Laboratories, Scientific User Facilities, CERN, and prestigious universities vie for the most visible presence on the treasure map with terabytes of climate and environmental data gathered around the globe and made available by DOE Data Centers such as the Carbon Dioxide Information Analysis Center (CDIAC) and the archive for the Atmospheric Radiation Measurement (ARM) Program.

Browse “Interactive Data Maps” to discover sophisticated, layered views of everything from nuclides to energy resource data (solar, wind, geothermal, etc.) to genetic details of plant and animal kingdoms, to carbon sinks, the ocean floor, and drought-stricken land....

Related Topics: cern, national laboratories, DOE Data Explorer (DDE)


Science Accelerator Brings You More Access and More to Access

You can now have multiple access points to Science Accelerator at your fingertips. Just download the new tabbed widget and you will have access to search Science Accelerator, to the RSS feed, and to the Science Accelerator Alerts.  Download via the 'Get Widget Options' link or by placing the inclusion code in the online location of your choice.

When you use the widget search feature, a federated search provides one-stop simultaneous searching of multiple networked data resources, including the newly-added resources -- DOE Data Explorer and DOE Green Energy.

DOE Data Explorer contains collections of scientific research data such as computer simulations, numeric data files, figures and plots, interactive maps, multimedia, and scientific images that have been generated in the course of DOE-sponsored research in various science disciplines.  These publicly available data collections support DOE research results that are well documented in journal articles, conference literature, and technical reports that are available via the Science Accelerator.

DOE Green Energy is a portal to information about various forms of green energy, including solar, wind, bioenergy, and others. It provides access to DOE technical report literature, green energy patent information, and other green energy results from research and development conducted throughout the Department and by DOE-funded awards at universities.  It contains both current research and historical research.


Related Topics: accelerator, access, accomplishments, citation clustering, conferences, customizing e-prints information, management, patents, projects, reports, science, search, software, winners, DOE Data Explorer (DDE), DOE Green Energy, Science Accelerator


DOE Data: Would We, Could We...?

I can’t remember how it went now, but as a child I skipped rope to a rhyme that included “would I, could I” somewhere in it.  Recently questions were asked about OSTI’s involvement with scientific research data.  Is OSTI planning to become a repository for numeric data?  Are we going to issue Digital Object Identifiers (DOIs) for datasets, and would we be telling people how to manage their data?  For some reason, the questions triggered the memory of that old refrain, but now I was thinking from an OSTI perspective, “would we, could we…?”

Fortunately, I’m much clearer about OSTI’s answer to those questions than I am about the conclusion of that old rhyme.  In order, the answers are a simple no, maybe, and no.

I’m in a position to know these answers because of my tasks here at OSTI.  I work with the Scientific and Technical Information Program (STIP) that handles policies and processes for information submissions to OSTI.  I’m also the product manager for the DOE Data Explorer and an OSTI point of contact for a related, ongoing STTR grant.

If you wonder why anyone would think to ask if OSTI has plans to begin taking in data, the question is, no doubt, triggered by the revision currently underway of the STI directive DOE O 241.1A.  That directive basically says that an announcement notice (citation/bibliographic record) for any scientific and technical information resulting from DOE-funded R&D must be submitted to OSTI.  For technical reports and, when possible, for other document types, that announcement notice contains a URL that links to the PDF document.  OSTI’s databases allow users to search both the citation in the database as well as the full text of the document, whether it resides at...

Related Topics: data, dois, r&d, stip, sttr, DOE Data Explorer (DDE), Scientific and Technical Information Program Website


Surviving a Technological Transformation

by Dr. Walt Warnick 02 Sep, 2008 in Technology

The life of every person in the world today has been shaped by successive technological transformations. The printing press transformed communication and education, beginning in the mid 15th century. Sailing and navigation technology of the late 15th century allowed Europeans to learn about other continents, beginning the global network of trade. Metal tools and firearms technology of the early 17th century enabled Europeans to colonize other continents and spread the fruits of European technology around the world. Railroads transformed transportation beginning in the early 19th century, and the telephone transformed communication in the latter part of that century. The automobile transformed transportation beginning early in the 20th century. These are but a few of the notable transformations that profoundly reshaped the way people live.

Today it is the Internet transformation, especially the Web. As a leader in making the Web work for DOE science, OSTI is embedded in the Internet transformation and OSTI itself is being transformed. Our dual core mission -- getting DOE results out to the scientific community and beyond, and getting the community's results into DOE -- has not changed. But the technology we apply to that mission has changed a lot. By carefully adopting Internet technology, and even pioneering new advances in that technology to meet our needs, OSTI achieves its mission better than ever before and has achieved a series of impressive "firsts."

I think all of us at OSTI would agree that getting this far has not been easy. If there is one word that describes what it has been like to be embedded in the Internet transformation, it is "turbulent." In this regard, the Internet transformation is much like the technological transformations that preceded it. Those embedded in transformations find themselves in a rapidly changing world which challenges them to find their own...

Related Topics: milestones, DOE Data Explorer (DDE), Science Conference Proceedings


Discover the data behind DOE publications!

If you're ready to discover data, then OSTI's newest product is ready for you!  The DOE Data Explorer (DDE) is a unique tool that identifies collections of DOE-sponsored numeric files, figures and data plots, multimedia and images, computer simulations, specialized databases, and interactive data maps. Browse, run a quick search, or advanced search, then click a link to results. You'll be amazed at the data you can freely see and use, the highly specialized interfaces developed by the owners of the data that will help you delve deeper into their collections, and the software toolkits that allow you to manipulate, compare, visualize, download, and re-use the data.

The DOE Data Explorer will guide you to data collections at national laboratories, data centers, scientific user facilities, colleges and universities ...and across all of the science areas with DOE involvement.  The DOE Data Explorer development team sifted through hundreds of these websites so that you would not have to, selecting each collection for inclusion according to strict criteria. 

DOE has several data centers  that provide excellent collections and expert services. Each of these centers specialize in data belonging to a specific subject area or scientific discipline. The DOE Data Explorer will help you find those centers and their collections. However, its unique usefulness is in helping you find the collections that are NOT in a data center.  In addition, what if you want to do cross-disciplinary research?  Or what if you don't even know what data might be out there or what discipline it might belong to? You need a data discovery tool that will allow you to see ALL of DOE's data - regardless of scientific discipline, regardless of format, and, even,...

Related Topics: data, national laboratories, DOE Data Explorer (DDE)