OSTI’s Committee of Visitors, An Update

"The unexamined life is not worth living."  So says Plato's Socrates in the Apology.   His self-examination led to extreme humility (or to an extreme irony) when Socrates confessed to his accusers that the only knowledge he had was knowledge of his own ignorance.  No one we know of came away from a Socratic cross-examination in one piece, but they would at least have known their own limits.  And in knowing their limits, or their ignorance, they would somehow be better.

That's really the reason we open ourselves up to honest reviews of our own performance, or open our programs up to honest review by outsiders.  Now there are two ways to go about such reviews.  One is to gather your amen corner around you and have them tell you how great you are and what progress you are making and how important you are, etc. etc.  You can then announce to the world that you are a smashing success.  The other way is to gather serious, knowledgeable, and thoughtful people and let them ask hard questions; ask them to put you through a Socratic dialogue.  You'll almost always discover that there is room for improvement, if you choose the latter course. 

OSTI chose the latter course when it had a Committee of Visitors (CoV) review its programs.  A previous CoV report some years ago had proved helpful.  I felt it was time for another review. 

After meeting with OSTI staff for a full two days, the committee provided us with 10 individual reports.  There was praise for the enthusiasm, professionalism and entrepreneurial spirits of the OSTI staff.  And although there were certainly questions and criticisms, no one felt these took away from the outstanding work the staff was doing.

A summary of the review is as follows:

Taken as a whole, the ten reviews shared a consistent message:  OSTI needs to grapple with and resolve the balance between its mission to provide ready access to DOE R&D results and its more entrepreneurial mission of making all science information available to the world, both founded on the (as far as I know) noncontroversial notion that discovery is accelerated through the dispersal of knowledge.   Many, if not all, other issues noted in the charge are related to this question of mission balance.  As these were independent reviews with no consensus even permitted by the structure of the CoV, there are various approaches to the mission balance issue.   Indeed the approach to questions such as "who is the customer," "what products should be emphasized," and "where should resources be focused" tend to be structured by how one first addresses the question of mission balance.  However, within the diversity of thoughtful opinions on OSTI's mission balance, there was general agreement that resolution of this concern was less a resource issue than one of management focus. 

As one can imagine, a summary of 10 detailed reports has to skim the surface.  The reviewers looked at a common set of questions, but naturally they would not come up with a common set of answers.  Still, the diversity wasn't so great that common themes couldn't be seen, and mission balance was certainly a theme.

There were others, but let's just touch on this one.

The CoV allowed OSTI the chance to take a step back and see how it was serving its primary customer, DOE.  This focus gave impetus and force behind a set of actions.

As an initial step, OSTI senior management and I met to address the CoV feedback.  The individual CoV team member reports were broken down into discrete recommendations/advice.  These were then grouped into the following broad themes:

• Improve Collection of DOE STI (includes Improving the Understanding of Program Needs and Improving STI Gathering from Labs)
• Digitize Legacy Collection
• Improve Classified Collection/Access
• Product Improvement
• Metrics
• Open Publication Model

For each theme, strategies and actions were identified, target dates determined, and leadership responsibilities defined.  Progress has been monitored on a quarterly basis. Many suggestions and ideas were put forward by the CoV and have been adopted by OSTI.  For example, gather far better data on collection efforts from the National Laboratories; coordinate more closely with program offices in DOE; strengthen the relationship with STIP, the Scientific and Technical Information Program; improve the classified collection through greater interface with NNSA; improve metrics on customer use of OSTI. These are a sample of some of the suggestions from the CoV that OSTI is working through.

I can say that OSTI has made real progress on these and other responses to the COV and work continues as OSTI is making additional progress.

Finally, I think the CoV itself was an outstanding group.  Here is the membership:

Each of these individuals made a real contribution to helping OSTI move forward.  We greatly appreciate their work.  They acted more like Socrates than an amen corner.