Report of Findings and Recommendations for the Review of the Business Case for the Corporate R&D Portfolio Management Environment (PME)

 

 

DESCRIPTION:  This report is a compilation of findings and recommendations from the review of the Business Case for the Corporate R&D Portfolio Management Environment dated August 15, 2000.  Twelve participants throughout the Department of Energy (DOE) conducted the review from the period November 20, 2001 to February 1, 2002.

 

Working Group:  The Business Case Validation Working Group team members were selected for their experience with the PME SIM, their knowledge of the Department’s R&D management needs and practices, their cognizance of on-going initiatives affecting the implementation of PME, and their subject matter expertise.  The review team consisted of the following members:

 

Co-Lead - Gary Louie, PNNL

Co-Lead - Vince Dattoria, SC

Steve Baker, ME

Steve Binkley, NNSA

Jose Elizondo, ID

Dan Hitchcock, SC

Brian Kendrick, ORNL

Laura Liles, LANL

Gary Peabody, CH

Dave Smith, EE

Marshall Thompson, SLAC

Trudy Wood, ME

 

Background and Scope:  Approximately 18 months ago, the Business Case for the Corporate R&D Portfolio Management Environment was published to document the current R&D management practices of the Department, identify alternatives, and to serve as a management-decision document.  The Corporate R&D PME project was approved in NovDecember 2000.  At the request of the Corporate R&D PME Project Manager, a Business Case Validation Working Group was formed to perform a rapid, but systematic review of the business case.  The review team was asked to document changes that have occurred over an 18-month period that impact PME.  As a result, the focus of the review is validation of the assumptions and analysis that resulted in the Business Case and the related PME requirements to make recommendations on necessary modifications or updates in the development of the PME project baseline plans and schedules.

 

During the review, two types of findings became evident.  The first set contains grammatical or editorial suggestions.  These findings primarily are limited to a specific occurrence in the Business Case and do not have material impacts on the Business Case content.  This set of suggestions is somewhat self-explanatory and is contained in Appendix A.    The second type of findings are those that have a substantial impact on the meaning and content and typically affect several areas within the business case.  This group of findings is contained in Appendix B.

 

 

Appendix A

 

Editorial Changes

 

Section 2, page 2-1

 

The processes used to manage R&D projects at by DOE and its predecessors have evolved over the past 50 years.

 

Section 2, page 2-13

 

 

 

 

Section 2, page 2-15

 

The process phases and activities just described reflect a highly manual and paper-intensive overall process with limited information system support. Therefore, an assessment of the DOE systems used to support R&D management processes was performed to provide a picture of the technology supporting the R&D project management environment. The assessment validated the existence of a manual process activity that requires each DOE program organization, operations office, and R&D facility to produce and process R&D proposal, execution, tracking and reporting, and program management information with minimal system automation. The annual estimated costs associated with the systems identified in the survey iscosts associated with the systems identified in the survey are $12.387 million. There were no corporate data linkages reported in the survey information that indicated current or planned system interfaces for the systems identified.

 

 

Section 4, page 4-6

 

4.3 Alternative 3 – Corporate R&D Portfolio Management Environment

 

This alternative would include all the process improvements and system enhancements of the Enhanced Base Case. In addition, it would include DOE-wide data dictionaries, combined with a project unique identifier Project Identification Number (PIN) service. This would enable powerful web-based query and analysis of the R&D project information residing in computers across the DOE complex, allowing authorized users to gather data from the original sources in real time and develop customized reports. This added functionality would integrate research project information across the DOE complex and enable improvements in efficiency across all R&D management processes.

 

Section 4, page 4-8

 

The solution would be able to handle all proposal types, even unsolicited proposals, including grants, field work proposals, and plans (whether they are generated from universities, private firms, or DOE R&D facilities), without requiring all data fields to be completed. It would also handle batch submissions. To facilitate review, the system would include a registry of institution and principal investigator background data (e.g., curriculum vitae or resume). In addition, it would be set up to provide electronic signature, approval, and concurrence capabilities.

 

Appendix A, page A-4, section A.3.1

 

A.3.1 The Challenge

 

Participants from the laboratories (business systems and accounting personnel) and DOE headquarters (programs, budget, and finance personnel) were concerned that much of the data provided via the proposal submission process and the in the responses to the R&D tracking data at year end were redundant. From the DOE perspective, there was no way to search proposed work. The ability to compare proposed work to funded work and related costs, at a project level defined by the program office, did not exist.

 

Appendix E, page E-9, section E.5.2.2

 

E.5.2.2 Alternative 2 – Enhanced Base Case

 

Recurring Costs:

 

Recurring costs for Alternative 2 are shown in Exhibits F-3 and F-5 found in Appendix F.

Hardware Maintenance: Hardware maintenance is estimated at 15 percent of the initial server hardware costs for an average industry-standard 24-hour, 7-day service agreement with an average response time. Utility software is also included at $.03 million per year. The total recurring cost for hardware maintenance is $0.12 million annually for a total 5-year cost of $0.56 million.  The recurring cost for a maintenance contract is $0.12 million annually.  The total recurring cost for hardware maintenance is $0.15 million for a total 5-year cost of $0.56 million.

 

Appendix E, page E-10, section E.5.2.2

 

Ongoing Training and Travel: Ongoing training and travel includes a yearly workshop and annual modifications/updates to the web-based training for a total of $0.05 million annually in year 4 and year 5 for a total 5-year impact of $0.10 million.

 

 

 

 

 

Appendix B

 

Substantive Findings and Recommendations

 

General:  The Business Case was researched and created with the participation of over 100 representatives throughout the Department drawing on their vast experience and collective knowledge.  It reflected one of the most comprehensive studies of the Department’s R&D management policies and practices of its time drawing on the best available information.  However, recent activities have transpired since the completion of the Business Case that have a substantial affect on the planning and implementation of the PME project.

 

 

1.      Other Initiatives

a.      Finding:  There are other initiatives that affect the PME project beyond those identified in the Business Case.

b.      Description:  Public Law 106-107 is affecting several initiatives in the Federal Government.  One such initiative in DOE is the Procurement Modernization (PROCMOD) project.  The PROCMOD project has just completed its SIM process.  One of the main goals of the PROCMOD project is to streamline and modernize the procurement process.  This may be a functional overlap with one of the goals of PME to serve as a receipt system for electronic proposals.  Since PROCMOD was not in existence, PROCMOD’s predecessor, IIPS was identified in the PME Business Case. 

c.      Recommendation:  Consideration should be given to coordination with other initiatives that appear to have functional overlaps and Integration Plans should be developed that delineate responsibilities of the systems and reduce or eliminate duplicative work. 

 

It is important to note that DOE is in the process of developing an e-commerce solution (i.e., Procurement Modernization SIM) to cover every aspect of acquisition and financial assistance actions, including the award and administration functions related to R&D awards.  The DOE e-commerce solution will comply with the requirements of PL 106-107, e-Grants (formerly known as the Federal Commons), and Federal acquisition requirements.  The solution will be integrated with BMIS-Phoenix and other Departmental and Government-wide systems.  DOE’s e-commerce solution should include the receipt of acquisition and financial assistance proposals and provide for the merit review process, reporting, funding, and the close out of financial assistance and acquisition instruments.  It appears that: 1) the PME R&D solution will provide for the receipt of facility management contractors’ annual R&D proposals, work authorizations, financial instructions, etc.; 2) BMIS will provide for the financial functions; and 3) the PME R&D solution will provide for the R&D project reporting.   This leads one to assume that somehow these various DOE solutions (i.e., systems) will be integrated to permit one to query, analyze, and summarize R&D projects.   If this assumption is correct, it should be described in the Business Case.  If not correct, the relationships of these systems need to be clarified.   It is recommended that the various DOE system managers agree on what each solution will provide and the systems integration process.  It may be necessary to use a common or virtual data warehouse for this integration. 

 

2.      Schedule

a.       Finding:  The timeline for the planning and development phases for the three modules was aggressive and currently does not reflect the impact of the major changes to date.

b.      Description:  The Business Case has a proposed schedule for the implementation of the PME project.    The Business Case identifies drivers such as Public Law 106-107 that are influencing the milestones.  The requirements of P.L. 106-107 will now be handled by PROCMOD. 

c.      Recommendation:  PME should review the language and intent of these drivers and revisit its original timeline and resource needs.  Specific adjustments should be made for the delayed start of the project relative to the timeline in the Business Case and a more lengthy requirements gathering and planning phase for each of the three modules.  Since it appears that the user and systems requirements are critical influences on the implementation of PME, these phases have a dramatic impact on the ultimate success of PME.

 

3.      Program Management Organization

a.      Finding:  The PME Program Management structure outlined in the Business Case is under staffed in dedicated resources

b.      Description:  Section 7 of the Business Case discusses an action plan for PME.  Among the items is an organizational structure and responsibilities for implementing PME.  In the Business Case, staffing for the various implementation milestones takes the form of dedicated PME staff, support contractors, and temporary, transitory staff from other offices within DOE.  Reliance on non-dedicated staff for critical tasks increases the level of risk for the project.  These risk areas are most evident in the requirements planning and design phases for the three modules. 

c.      Recommendation:  Consideration should be given to including additional dedicated resources, either Federal or contractor personnel, to facilitate a smooth implementation.  Subject matter experts are still critical participants in the requirements planning and design phase, and they should be included on an as-needed and dedicated basis on focused working groups that are formed and chartered with a specific purpose for a specific time period.  There is also value in establishing a high-level senior management board that oversees and directs the PME project, such as a Board of Directors.

 

4.      Acquisition Strategy

a.      Finding:  The acquisition strategy limits available sources.

b.      Description:  Paragraph 7.1.3 of the Business Case discusses the acquisition strategy.  Where new procurement is necessary, the strategy is to use the General Services Administration (GSA) schedule where possible, to purchase commercial off-the-shelf (COTS) products. 

c.      Recommendation:  Given the current support contracts and the recent volume-based pricing negotiated by the Department, consideration should be given to alternative acquisition sources beyond GSA to ensure the best pricing and value for the Government.

 

5.      Hardware/Software Specifications

a.      Finding:  The configurations in the Business Case are out dated.

b.      Description:  Paragraph G.2 of the Business Case discusses the Hardware/Software Specifications developed using the prevailing state of technology and representative equipment configurations at the time. 

c.      Recommendation:  Given the rapid advances in technology and the mainstream equipment refresh cycles, consideration should be given to updating both the equipment processing capabilities and representative configurations to reflect the current state-of-the-art.  These configurations should also be reviewed for the required level of performance for a PME system once a determination has been made regarding what data will be housed in a PME database on a PME server and what data will be accessed on other servers integrated with PME.

 

 

Return to CME Final Business Case dated 8/15/00