Why Projects Fail—and What World-Class Teams Do Differently

We have spent four articles building up the toolkit: integration, scope management, monitoring and controlling, change governance, and stakeholder engagement. The uncomfortable question remains: if all these tools exist, why do projects keep failing? The answer is not technical — it is human.

---

Why This Question Matters

In the 1820s, George Stephenson built a railway from Liverpool to Manchester. It cost 45% more than budget and was delayed repeatedly as the route crossed the treacherous Chat Moss bog. Nearly two centuries later, The Economist observed that the management of large-scale projects seems to have improved remarkably little.

This is not a fringe opinion. The evidence is overwhelming and spans industries, geographies, and eras:

• The reconstruction of Wembley Stadium — a £750 million project — faced mounting losses when the cost of steel doubled and additional labour was required to meet the deadline for the FA Cup Final.
• A £6 billion project to digitise the medical records of 50 million Britons was significantly over budget and postponed by several months.
• The FBI abandoned a $170 million internal IT project two years after problems had first been identified.
• The Standish Group found that in 2004, only 29% of IT projects "succeeded" — down from 34% in 2002. Cost overruns averaged 56% of original budgets, and projects took 84% more time than originally scheduled.
• The Baku–Tbilisi–Ceyhan oil pipeline — a $4 billion project — was several months overdue and 5–10% over budget, despite being considered a relative success.

The Pattern: Projects in every sector — construction, IT, energy, defence — consistently overshoot on time and budget. The tools and methodologies exist. The failure is in application, and the root cause is almost always human.

---

The Root Causes: Psychology Before Process

1. Optimism Bias and Self-Serving Bias

The most fundamental reason projects fail at the forecasting stage is not incompetence — it is psychology. Max Bazerman of the Harvard Business School describes a phenomenon called "self-serving bias," which he uses to explain why good accountants produce bad audits. The same mechanism explains why good project managers produce bad forecasts.

To secure a project, bidders make overly optimistic assumptions about costs, time, and revenues. This is not always deliberate deception — it is often unconscious motivated reasoning. The project team wants the project to succeed, and this desire subtly distorts their estimates.

This bias is amplified in the public sector, where accountability to the ultimate paymaster (the taxpayer) is less rigorous than in private enterprise. It is further amplified on prestige projects — like Wembley Stadium — where bidders chase glory almost as much as commercial gain.

2. Forecasting Accuracy Has Not Improved

A landmark study published in the Journal of the American Planning Association examined 210 major rail and road projects across 14 countries. The findings were stark:

The study's lead author, Bent Flyvbjerg of Aalborg University in Denmark, concluded that forecasts on large projects are no more accurate now than they were 30 years ago. The tools have improved. The accuracy has not. This strongly suggests that the problem is not technical but behavioural.

Project Team — Wants Approval

Unconscious — Optimism Bias

Overly Favourable — Forecasts

Project Approved — on False Premises

Reality Diverges — from Forecast

Cost Overrun & — Schedule Delay

Post-Hoc — Rationalisation

Relationship details

From	Relationship	To
Project Team — Wants Approval	leads to	Unconscious — Optimism Bias
Unconscious — Optimism Bias	leads to	Overly Favourable — Forecasts
Overly Favourable — Forecasts	leads to	Project Approved — on False Premises
Project Approved — on False Premises	leads to	Reality Diverges — from Forecast
Reality Diverges — from Forecast	leads to	Cost Overrun & — Schedule Delay
Cost Overrun & — Schedule Delay	leads to	Post-Hoc — Rationalisation
Post-Hoc — Rationalisation	No systemic — correction	Project Team — Wants Approval

3. Initiation Is Separated from Execution

The Project Management Institute identifies five project phases: initiation, planning, execution, control, and closure. Problems arise most frequently when initiation gets separated from execution — when the people who promise the project are not the people who deliver it.

This separation creates a structural incentive for optimism at the front end (to win the bid) and a structural impossibility at the back end (to deliver against those optimistic promises).

4. Poor Scope Definition Before Estimation

Kul Uppal's analysis of major engineering and construction projects identifies two primary root causes of project cycle time problems:

1. Poor definition of project requirements (project scope of work) prior to preparing the project cost estimate. 2. Failure to recognise invalid assumptions behind these project requirements.

These are Front End Loading (FEL) failures. When scope is insufficiently defined before the estimate is prepared, the estimate is not wrong — it is fiction. And once a fictional estimate becomes an approved budget, every subsequent variance is measured against an imaginary baseline.

5. Failure to Learn from Past Projects

J.S. Busby's research on post-project reviews highlights a pervasive organisational failure: most organisations do not systematically learn from completed projects. Post-project reviews, when they occur at all, tend to be superficial, politically constrained, and disconnected from the planning processes of future projects.

Lavingia addresses this directly in his PMP framework by mandating:

The Accountability Gap: Business evaluation is conducted 1–2 years after project completion, and the project sponsor is responsible for this review. This practice brings accountability into the overall project management process. Yet most organisations skip it — because by the time the evaluation is due, the sponsor has moved on, organisational attention has shifted, and nobody wants to revisit a project that is already "done."

Initiation

Planning

Execution

Project Completion

Post‑Project — Assessment

Business — Evaluation — (1–2 Years Later)

The Accountability Gap

Most organisations stop here

Relationship details

From	Relationship	To
Initiation	leads to	Planning
Planning	leads to	Execution
Execution	leads to	Project Completion
Project Completion	leads to	Post‑Project — Assessment
Post‑Project — Assessment	leads to	Business — Evaluation — (1–2 Years Later)
Project Completion	leads to	The Accountability Gap
The Accountability Gap	leads to	Business — Evaluation — (1–2 Years Later)
Most organisations stop here	leads to	Project Completion

---

What World-Class Teams Do Differently

The evidence is not entirely bleak. Some organisations have developed cultures and systems that consistently beat the odds. Here is what sets them apart.

1. They Treat Project Management as a Core Competence

The Economist observes that companies are increasingly organising themselves around projects rather than ongoing operations. The examples are instructive:

For these firms, project management is not a support function — it is a competitive weapon.

2. They Embed Project Managers in Sales Teams

One of the most innovative elements of Siemens' initiative was introducing project managers to the company's sales teams. The purpose was to temper the sales team's more extravagant promises with realistic execution assessments before the bid was submitted — directly attacking the optimism bias at its source.

This requires a delicate balance: rein in the promises enough to make them deliverable, but not so much that the bid is no longer competitive. The project manager in this role is not a controller — they are a reality check embedded in the commercial process.

3. They Decentralise Authority to Project Managers

BP's transformation of its exploration division, BPX, provides a structural model. BP converted BPX into a portfolio of projects, each operating as a more or less autonomous unit — a structure the company describes as an "asset federation."

Under this model:

• Asset/project managers cannot rely on head office for support
• They are required to build their own self-sufficient teams
• They have full accountability for project outcomes

This decentralisation eliminates one of the most common project pathologies: decision bottlenecks at head office that delay execution while preserving the illusion of central control.

4. They Get the Contract Right

The winner of the PMI Project of the Year award was the Saudi Aramco Haradh gas pipeline — a $2 billion project to build a gas terminal deep in the Saudi desert, 10 kilometres from the nearest road. It was completed six months ahead of schedule and 27% under budget.

The project was lauded specifically for the way its original contracting document defined the mix of contracts best suited to accomplish the project's objectives. This is not glamorous work — it is the painstaking, front-end discipline of matching the contract structure to the project's risk profile.

5. They Close the Learning Loop

Lavingia's framework mandates that lessons learned are both sought (at the start of new projects) and shared (at the end of completed projects). World-class organisations do not treat this as optional documentation — they treat it as organisational infrastructure.

The mechanisms include:

• Peer reviews — where participants not associated with the current project constructively challenge assumptions, alternatives, decision logic, and the proposed path forward
• Pre-funding assessments — where project progress and quality are rated against a database of similar past projects
• Post-project assessments — where actual outcomes are compared to approved AFE data and used to update the estimating database
• Business evaluations — where the project sponsor is held accountable for financial outcomes 1–2 years after completion

Lessons Learned — (SEEK) — Start of new project

Peer Review — During FEL

Pre-Funding — Assessment — End of Phase 3

PROJECT — EXECUTION

Post-Project — Assessment — End of project

Business — Evaluation — 1-2 years post

Lessons Learned — (SHARE) — End of project

Relationship details

From	Relationship	To
Lessons Learned — (SEEK) — Start of new project	leads to	Peer Review — During FEL
Peer Review — During FEL	leads to	Pre-Funding — Assessment — End of Phase 3
Pre-Funding — Assessment — End of Phase 3	leads to	PROJECT — EXECUTION
PROJECT — EXECUTION	leads to	Post-Project — Assessment — End of project
Post-Project — Assessment — End of project	leads to	Business — Evaluation — 1-2 years post
Business — Evaluation — 1-2 years post	leads to	Lessons Learned — (SHARE) — End of project
Lessons Learned — (SHARE) — End of project	Feed into next — project's Seek phase	Lessons Learned — (SEEK) — Start of new project

---

The Pitfalls: Why Even Aware Organisations Still Fail

1. Knowing the bias does not eliminate the bias. Organisations that understand optimism bias often believe they have inoculated themselves against it. They have not. Structural countermeasures — like embedding project managers in sales teams or using reference-class forecasting — are required. Awareness alone is insufficient.

2. Post-project reviews are conducted as ceremonies, not investigations. When post-project reviews are politically constrained — when the findings must not embarrass senior sponsors — they produce sanitised lessons that change nothing. Effective reviews require psychological safety and organisational authority to act on findings.

3. "Lessons learned" databases are built but not queried. Many organisations invest in capturing lessons learned. Far fewer invest in ensuring those lessons reach the teams who need them. A lessons-learned repository that nobody searches before starting a new project is an archive, not a management tool.

4. Success is defined too narrowly. A project completed on time and on budget is not necessarily a success. If the business case assumed 10,000 daily users and the facility attracts 2,000, the project failed — regardless of how well the execution was managed. Flyvbjerg's research shows that forecasting errors on the demand side (revenue, usage, throughput) are often larger than those on the cost side.

5. Accountability decays over time. The further removed a review is from the project's completion, the weaker the accountability. Sponsors change roles. Teams disperse. Organisational priorities shift. The 1–2 year business evaluation — the most consequential review in Lavingia's framework — is the one most likely to be skipped.

---

The Formula for Improvement

Lavingia distils the path to improved profitability through project management into four interdependent elements:

Structured Project — Management Process — (Five-Phase PMP)

Management's Active — Involvement — (Accountability, Leadership, — Resources, Behaviours)

Value Improving — Practices & Best — Practices — (Applied at the right time)

Total Cost — Management — (Converting optimised — scope into cost & schedule)

Improved ROCE — Better, Cheaper, — Faster, Safer — Projects

Relationship details

From	Relationship	To
Structured Project — Management Process — (Five-Phase PMP)	leads to	Improved ROCE — Better, Cheaper, — Faster, Safer — Projects
Management's Active — Involvement — (Accountability, Leadership, — Resources, Behaviours)	leads to	Improved ROCE — Better, Cheaper, — Faster, Safer — Projects
Value Improving — Practices & Best — Practices — (Applied at the right time)	leads to	Improved ROCE — Better, Cheaper, — Faster, Safer — Projects
Total Cost — Management — (Converting optimised — scope into cost & schedule)	leads to	Improved ROCE — Better, Cheaper, — Faster, Safer — Projects

The formula is simple. Implementation is the challenge. A company that consistently selects the right projects and executes them with excellence can improve ROCE and ultimately total shareholder return (TSR). In a competitive business environment, this can mean the difference between a profitable company and one that becomes a takeover target.

---

Key Takeaways

• Project failure is predominantly a human problem, not a technical one. Optimism bias, self-serving bias, and separation of initiation from execution are the primary root causes.
• Forecasting accuracy has not improved in 30 years despite dramatic improvements in tools and methodologies. The problem is behavioural, not technical.
• Poor scope definition before estimation means the approved budget is fiction — and every subsequent variance is measured against an imaginary baseline.
• World-class organisations treat project management as a core competence, not a support function. They embed PM into sales, decentralise authority, and get the contract structure right.
• The learning loop must be closed. Lessons learned must be both sought at the start of new projects and shared at the end of completed ones. Post-project assessments and business evaluations are the mechanisms — but they only work if they carry real accountability.
• The four elements of improvement — structured PMP, management involvement, value improving practices, and total cost management — are individually necessary and collectively sufficient. Skip any one and the system degrades.
• Success must be measured against the business case, not just the execution plan. A project delivered on time and on budget against a flawed forecast is still a failure.

---

This concludes the Project Implementation Masterclass series. The five articles — Integration, Scope Management, Monitoring & Controlling, Change Control & Stakeholders, and Why Projects Fail — form a complete framework for understanding how projects are planned, tracked, governed, and improved. The tools exist. The knowledge exists. The differentiator, as always, is discipline.