You can’t afford to neglect data

Martin Paver argues that, in addressing the reasons for project failure, forensic insight is key.

I recall my frustration, while leading a large project, at reviewing a database of hundreds of generic, partially relevant lessons and having to confirm that I would not repeat the mistakes of the past. Despite my reservations, as project manager, it was my job to confirm that I had taken the lessons on board. Although I ticked the box, I felt uncomfortable. My daily quest was to get under the skin of some of these lessons, but it was harder than it should have been.

In another role, I was presented with a lessons-learned document from a multimillion-pound project. Without board-level commitment to learn from these lessons, and limited organisational appetite to address the underlying challenge, the report was filed away. We lost these insights, and all enthusiasm to share lessons quickly fell away.

I have since collated a dataset of more than 10,000 lessons. From this, it is clear that organisations’ ability to learn from lessons varies dramatically.

There is a huge body of work on the reasons for project failure. In 1995, Martin Cobb wrote: “We know why projects fail; we know how to prevent their failure – so why do they still fail?” Reports have been published globally on how to address this paradox, but the same themes still haunt us.

Taking a small subset of recent public projects – FiReControl, Edinburgh Trams, eCare, Registers of Scotland, BBC Digital Media and NHS 24 – the cost of failure amounts to around £1bn. Themes are similar, and despite assurances that lessons were learned, the evidence suggests otherwise. In the current economic climate, are we not obligated to revisit this challenge? Does this require transformational change?

Many organisations lack the evidence to justify this change. By looking at the challenge through a data lens, we have an opportunity to bring transparency, establish trends, identify correlations, discover how lessons snowball, and, more importantly, shine a light on the cost of failure to learn from lessons. It is too easy to tick the box that lessons have been learned.

Lessons tend to be managed in one basket, yet segmentation holds the key to success. Many lessons are about the failure to follow the accepted body of knowledge, specific details relating to technical know-how or the results of ‘fast fail’ experiments. The solution to each of these is different. For example, not following the APM Body of Knowledge may be an issue of priorities and motivational factors, rather than knowledge management.

Some projects are once-in-a-career events. NHS 24 is a good example; in the report from the Public Accounts Committee, its chief executive commented: “This is a once-in-a-decade event. It is not something that we do regularly.” But, in reality, NHS 24 is a complex call centre. Call centres are being delivered across the world on a regular basis, creating insights, recipes, challenges and lessons. The frequency and relevance of lessons depend on perspective.

For many organisations, the impact of not acting on lessons isn’t evident, which makes it difficult to prioritise and create the business case for change. Organisations don’t tend to assess the degree to which negative lessons are avoidable.

Some lessons emerge because of ‘black swans’, which are argued to be unknowable events, but as the sample size grows, the unknown becomes more knowable.

Lessons often lack forensic insight; for example, on first inspection, the lesson may be attributed to the quality of the plan, but on further examination, it may be an issue of complexity and emergence. If lessons identified lack rigour, they also lack insight.

Data holds the key to creating momentum from which change and knowledge management methods can be applied. Imagine it is 2030 and a project manager can look three to six months ahead and visualise how their project will unfold. It may not be that far into the future that project managers are judged on performance based on managing events before they unfold.

It’s a future that is unstoppable; our challenge will be how we engage with it.

Martin Paver is founder of Projecting Success, an APM Registered Project Professional and a chartered engineer