Below is a case-study of the construction Hanford Vitrification Plant. This is a mega-construction project with the latest estimated cost to of $33-42 billion. The text below is taken from GAO-06-602t, which is a government accountability report. Even though I’ve modified the text, it has largely been by removing it for clarity.

At the end there’s a link to my marked up version of this case study. If you choose to review this as a case study, here are some rough directions, though the whole article is well worth your time.

Hanford Construction Cost

Overview

The Department of Energy (DOE) manages Hanford’s tank waste through two main contracts: a tank farm operations contract to maintain safe storage of the waste and to prepare it for retrieval, and a construction contract with Bechtel to design, construct, and commission the operation of a waste treatment plant.

The Hanford waste treatment construction project includes the construction of three primary processing facilities, a large analytical laboratory, and 23 supporting buildings on a 65 acre site. The three primary processing facilities are:

• the pretreatment facility, which receives the waste from the tank farms and separates it into its low-activity and high-level waste components;

• the high-level waste facility that immobilizes high-level waste for offsite disposal through a process known as vitrification, which mixes nuclear waste with molten glass; and:

• the low-activity waste facility, which vitrifies the low-activity waste for onsite disposal.

The waste treatment plant facilities are large and complex. For example, Bechtel estimates that the completed project will contain almost 270,000 cubic yards of concrete and nearly a million linear feet of piping. The largest building, the pretreatment facility, has a foundation the size of four football fields and is expected to be 12- stories tall.¹

Cost and Schedule

In 2000 at initial award, the contract price was $4.3 billion, including contractor fee and project contingencies. In 2003, Bechtel revised the estimate to $5.7 billion, based on changes DOE wanted to make in plant capacity and to correct for estimating errors and other problems that were already occurring on the project. In December 2005 estimate of the cost to complete the project, an estimate that DOE has not yet approved, totals about $10.5 billion plus contractor fee. Bechtel is still revising its estimate of the project costs, and the final estimate will very likely be higher. For example, in a February 2006 hearing before the Senate Armed Services Committee, the Secretary of Energy said that the final cost for the project could be nearly $11 billion.

Figure 1: Progression of Cost Estimates for WTP Construction Project.

Note: These cost estimates do not include contractor performance fees and are not adjusted for inflation, about 15% over the time period.

In 2000, the estimated date to complete the construction of the waste treatment project was 2011. This date corresponded to the work schedule agreed to by DOE in the Tri-Party Agreement under which DOE was to begin operating the waste treatment facilities by 2011. However, Bechtel’s latest estimate, not yet approved by DOE, is that the construction project will be completed by 2017 or later, at least a 6-year extension and a 50 percent increase in the project’s schedule, (it was finally completed in 2023).

Furthermore, the revised cost and schedule estimates Bechtel developed in December 2005 are not final and will likely increase further. At least through the rest of 2006, DOE and Bechtel will continue to address identified technical and safety issues and incorporate additional design changes into its estimates. For example, Bechtel is currently reviewing several technical issues recently raised by a panel of experts DOE invited to study the project. Bechtel plans to incorporate changes resulting from the review into a new cost estimate. This revised estimate is expected to be complete in late May 2006. Once that estimate is available and DOE has completed its review of the estimate, DOE and Bechtel will need to agree on a revised contract price that incorporates any changes made to the project, including any changes to the fee that Bechtel can potentially earn. DOE officials do not expect to have these activities completed until late 2006

Shared Responsibility

Bechtel made a number of miscalculations on a broad range of activities when developing and revising its cost estimates for the project. Specifically, we found that Bechtel:

• underestimated by more than 50 percent the engineering hours needed to design the facilities (a small portion of this increase was due to changes in seismic design criteria). The current estimate for design hours is now over 14 million hours².
• underestimated the cost of key commodities like steel. Steel prices climbed sharply once project construction started³.
• incorrectly assumed that it could obtain an exception to the fire code and avoid applying a protective coating on some of the structural steel used in the facilities and instead use a less expensive sprinkler system⁴.

Bechtel also incorrectly estimated the amount of contingency funds that would be needed to account for project uncertainties. In 2000, Bechtel estimated that $500 million in contingency was needed. However, in its December 2005 estimate, Bechtel proposed that a total of $2.8 billion in contingency be allocated to the project⁵. The $2.8 billion in contingency funds included $1.76 billion to address technical and programmatic risks outside the current scope of the project and an additional reserve of about $1 billion for potential future problems not yet identified.

Finally, Bechtel was ineffective at ensuring that the completed facilities would meet nuclear safety requirements. In March 2006, DOE’s Office of Enforcement issued a report documenting a number of different safety problems with the construction project, including a failure to (1) include safety requirements in design documents⁶, (2) identify and use the correct design codes and safety standards, and (3) track design changes to ensure purchased materials and supplies were consistent with those changes⁷. These failures led to significant problems. For example, Bechtel ordered approximately 70 tanks with incorrect structural specifications to ensure the quality of their welds. These tanks, that will be located in inaccessible areas of the waste treatment plant, were in various stages of fabrication. Had this problem not been identified, the quality of welds for all of these tanks could have been flawed. One tank had already been installed using these incorrect specifications before the problem was discovered. The tank was installed because neither the supplier nor Bechtel had performed the required weld inspection. Furthermore, when the welds were first repaired the subcontractor used incorrect welding rods, requiring more rework to repair the repairs.

In addition, in September 2005, Bechtel discovered errors that had been made in structural steel calculations for the laboratory facility[^2]. These potentially serious errors included design specifications that were incorrect and discrepancies between engineering calculations and design drawing specifications, which led to replacing steel already purchased and correcting hundreds of engineering drawings. Of significant concern, a 2005 DOE-sponsored survey found that some construction and engineering employees were reluctant to raise safety concerns to Bechtel management, fearing reprisal[^25]. [^2]: An example of skill deficit in engineering, possibly compounded by motivated reasoning. [^25]: An example of commitment to ignorance, Bechtel management created this environment and it succeeded.

In our view, DOE’s management of the project has been flawed, as evidenced by (1) adopting a fast-track approach to design and construction activities that both created and exacerbated problems and (2) failing to exercise adequate and effective oversight of contractor activities, both of which contributed to cost and schedule increases.

DOE’s decision to pursue a fast-track, design-build approach under which technology development, facility design, and construction activities were carried out concurrently has proven to be regrettable⁸. DOE adopted the fast-track approach because of commitments made under the Tri-Party Agreement to have facilities operating by 2011, and to treat all of the tank waste by 2028⁹. However, using a fast-track approach for nuclear facilities is considered “high risk,” and is not recommended for designing and constructing one-of-a-kind, or first-of-a-kind complex nuclear facilities. DOE’s own project management guidance cautions against using this approach for complex facilities. For example, DOE Order 413.3 cautions that a design-build approach should only be used in limited situations, such as when work scope requirements are well defined, projects are not complex, and technical risks are limited.

Furthermore, the project approach included optimistic assumptions¹⁰ that virtually every major safety, technology, regulatory, and nuclear material acquisition uncertainty could be resolved while facilities were being constructed at an unusually fast pace for the largest, most complex, first-of-a-kind, nuclear waste treatment plant in the United States¹¹. Less than one year after construction began, DOE was already experiencing problems with construction activities outpacing design¹², technology problems that were affecting the critical path of the construction project, contractor safety control inadequacies, and outdated facility seismic criteria. Despite these problems, DOE insisted on continuing its fast-track design-build approach under its accelerated cleanup plan until early 2005¹³. At that point, the effect of these and other unresolved issues, contractor performance problems, and signs of significant cost growth and schedule delays caused DOE to direct Bechtel to significantly slow construction, rework the design, and reevaluate safety, seismic, and regulatory requirements.

Under nuclear industry guidance, which recommends that facility design be essentially complete before construction begins, major environmental, technological, and regulatory issues can be resolved in advance of construction. The benefit of this process is that most uncertainties are resolved before major capital is at risk, and the potential for project delay is significantly reduced¹⁴. On this project, under the fast track approach, actual schedule delays of more than two years have occurred, contributing to more than 1,000 workers being laid off, and work on the two largest waste treatment facilities coming to a halt.

GAO, the Safety Board, and others have criticized DOE in the past for using the fast-track approach for large, complex first-of-a-kind nuclear cleanup facilities. We issued reports in 1993, and again in 1998, that were critical of DOE for using an approach that differs so significantly from nuclear industry guidelines for constructing complex nuclear facilities. The Safety Board cautioned in June 2002, and again in March 2004, that a fast-track, design-build approach could lead to expensive plant modifications or to the acceptance of increased public health and safety risks. In June 2004, we recommended that DOE avoid using a fast-track approach to designing and constructing its complex nuclear facilities. The department accepted this recommendation, but apparently believes that it does not apply to this project. At the time of our 2004 report, the department could not identify a single instance where it had successfully used the approach to construct a large, complex nuclear cleanup facility. Despite the fact that DOE has never been successful with this approach on any complex nuclear cleanup project, Bechtel reported in its most recent cost and schedule estimate that a “fast-track engineering, procurement, and construction” approach is a standard commercial approach for large projects and the best approach for a schedule-driven project.

DOE’s lack of oversight of Bechtel’s activities has also been unfortunate. DOE did not ensure adherence to normal project reporting requirements and, as a result, status reports provided an overly optimistic assessment of progress on the project. For example, in January 2005, DOE’s project status report indicated that costs and scheduled work to date were proceeding as planned. However, Bechtel was not providing accurate information¹⁵. The project almost always appeared to be on schedule because Bechtel adjusted the project baseline schedule to match actual project results. In addition, DOE headquarters oversight officials were generally unaware¹⁶ of the full extent of the problems with the project.

DOE is responsible for ensuring that its activities follow nuclear safety requirements and generally receives no outside regulatory oversight of nuclear safety. Contributing to the problem, DOE’s internal safety oversight had been significantly reduced since 2000. Key responsibilities to ensure quality control of contractors were placed under the responsibility of the DOE project manager who also had primary responsibility for meeting project cost and schedule targets¹⁷. In late 2003, DOE began recognizing some of the nuclear safety problems on the project but many of these problems dated back to 2002, or earlier. Finally, in 2005 and 2006, according to the WTP project manager, DOE withheld a total of $800,000 in performance fee from Bechtel for industrial and nuclear safety problems, but problems continued. In 2006, DOE assessed a civil penalty of $198,000 for a number of nuclear safety violations¹⁸. DOE also recently increased the number of staff assigned to oversee safety activities.

Technical challenges

In 2002, the Safety Board began expressing concerns that the seismic standards used to design the facilities were not based on the most current ground motion studies and computer models, and were not based on geologic conditions present directly under the construction site. After more than 2 years of analysis and discussion, DOE contracted for a new seismic analysis that confirmed the Safety Board’s concerns that the seismic criteria were not “sufficiently conservative” for the two largest treatment facilities–the high-level waste facility and the pretreatment facility. Revising the seismic criteria caused Bechtel to recalculate thousands of engineering estimates and to rework[^30] thousands of design drawings to ensure that tanks, piping, cables, and other equipment in these facilities were adequately anchored. Bechtel determined that the portions of the building structures already constructed were sufficiently robust to meet the new seismic requirements. By December 2005, however, Bechtel estimated that engineering rework and other changes to tanks and other equipment resulting from the more conservative seismic requirement would increase project costs by about $750 million to $900 million and result in a 26 month schedule delay.

In 2003, potential problems with the pulse jet mixers caused project construction delays. Bechtel initially planned to rely on computer modeling to confirm that the mixer would successfully keep the tank waste uniformly mixed. However, because these mixers were designed to be placed in “black cells” in the pretreatment facility where they could not be repaired or modified after operations began because of the high levels of radiation in the cells, mixer failure was considered high risk. Given this risk, in April 2003, just 9 months before the design configuration for the mixers was to be completed, Bechtel decided to conduct laboratory tests of the mixers to ensure that they would successfully mix the tank waste. Based on laboratory performance testing, Bechtel found that the mixers did not adequately work. Consequently, the mixers had to be re-designed. The tanks that were to house the mixers also had to be redesigned with greater structural support to accommodate more forceful mixing pumps and other modifications. DOE spent about two years addressing problems with the pulse jet mixers. According to DOE’s project manager, Bechtel has completed the testing and design modifications for the mixers. As of May 2005, this problem had contributed more than $300 million to the project’s cost growth.

In June 2004, we reported on the possibility of hydrogen gas building up in the plant’s tanks, vessels, and piping systems, and noted that the buildup of flammable gas in excess of safety limits could cause significant safety and operational problems. Although DOE and Bechtel have been aware of this problem since 2002, the problem has not been fully resolved. As of March 2006, Bechtel continued to assess how to resolve this technical problem but has not identified final solutions. In April 2005, Bechtel estimated that this problem contributed about $90 million to the project’s cost growth.

In March 2006, an external technology review identified another technological problem called “line plugging,” involving the potential that solid and liquid radioactive and hazardous wastes could plug waste treatment facility piping systems during treatment operations. Described as the most serious problem the external group identified, the report emphasized that unless corrected, this flaw could prevent the plant from operating successfully. The review concluded that the treatment plant’s piping systems could begin plugging within days to a few weeks of operational start up. The external review did not estimate the potential cost and schedule impact of correcting this problem, but concluded that DOE identify and consider the corrective actions needed to resolve the problem. Bechtel plans to address these actions in its final cost and schedule estimate due in late May 2006¹⁹.

Here’s a link to my marked up version.