This Year's Benefits Summit from the PMO SIG's Perspective, 23rd June 2016

The Benefits Summit 2016 took place on a dreadful weather day, in London, on the 23rd June. Once everyone managed to battle through the rain beaten tube lines, broken umbrellas, and traffic, Bruce Phillips gave the delegates a rousing introduction reiterating the theme for the day ‘Value over Delivery’, encouraging them to network and build relationships with their peers, take the opportunity to learn more about our trade, and use the day to fill in any gaps in their benefits management armoury.

Neil White took the stage to introduce the National Audit Office’s Joanna Lewis, but managed to demonstrate his knowledge in popular culture by introducing Leona Lewis instead. Down with the kids indeed!

Joanna Lewis declined to do a song and dance but instead told us about the perception of delivery success or failure by citing the O2 as an example of a project perceived as a failure in terms of its delivery but now is one of the most visited tourist landmarks in London, and astronomically popular for gigs and events. The project delivered the anticipated benefits in the end… but was it value for money? It depends I suppose if you can imagine the London skyline without it!

Interestingly, at the National Audit Office, there is no standardised RAG scoring utilised. Instead a Delivery Confidence Assessment has been put in place that takes into account all the aspects of large scale change such as inter-dependability with other things going on (e.g. defence projects) political weight and the economic predictions for the expected life of the activity. This tool has helped to move delivery performance out of the opaque world in which it generally lives, but did highlight that a third of all projects being delivered were at red or red/amber status. Oops!

Despite the good work carried out by the National Audit Office to answer the value for money question, the work is not yet done. The question remains; how do we look at long term benefits across the wider transformation portfolios ensuring clear accountability for ownership whilst maintaining delivery and the perception of success? One to ponder in the bath later methinks.

Next we saw Claire Ross, the benefit manager take the stand to tell us about how the Met Office builds (or attempts anyway) to build a justifiable business case for an upgrade to their super computer (what has to happen every five years or so to keep them competitive by the way).

Claire reminded delegates that the Met Office is fundamentally an information provider, and that the number attached to the socio-economic benefit for the upgrade the last time was £2m. Not a small number! But what’s in it for me and how do you prove that by having the ability to utilise more complex prediction models it will effectively ‘add to the bottom line’ for the treasury to give the green light?

It’s hard.

Consider the Met Office not as the weather app that sits on your phone, but as an information provider just for a second. An information provider that sells its information to a broad set of customers. One of those customers is your local supermarket who will utilise the information it receives for the next bank holiday to stack its shelves in order that you purchase more burgers and hotdog buns because the weather forecast says 22 degrees and clear… If instead the weather turns to storms and lightning, the impact on that customer could be severe.

Now turn that logic onto airports, who decide not to put anti-icing on their planes but the weather takes a turn and drops to 2 degrees. It’s not safe to fly without it, so the impact? Newsworthy delays, maybe even cancellations, and ruined holidays, meeting or family gatherings.

The stakes are high for the Met Office to get this right. With the purchase of a new super computer, the estimate is that weather forecasting gets 24hrs more accurate every 10 years.

The Met Office look at the rate of false alarms as a metric they use government data on the impact of real weather (road and public transport for example). But focus on the loss of information value – because there is a lack of understanding on how forecasting works a perfect forecast can easily be turned into useless information by the end user.

Climate change and global economics also play a part which is difficult to predict well enough to create a benefits model that can be placed in front of the Treasury.

Adding measurement points that the Met Office know is easy for them to measure internally using all sorts of tools and methods that exist.
The customer bit though… surveys, stats per sector, demographics, are all hard to get, usually out of date, and not overly accurate.

The fact is, a lot of the above is theoretical, it belongs in the realms of could, should, might, so the Met Office continue to work to refine the benefits profile for their business case, whilst providing 92% accurate forecasts. They do this by answering one key question that stuck with me: “Have we succeeded sufficiently?”.

Maybe try reframing your project; you know, the one that was complicated to deliver and was beset with issues from the start.

The coffee break was sharply followed by a number of short sharp case studies with a panel for quizzing the presenters at the end.

Network Rail were the first up with their Orbis Programme – a technology enabled transformation activity with long term benefits. Their challenge: to ensure owners have the confidence and commitment to realise the long term benefits.

Their answer: a benefits charter and profile which was signed and treated as a contract for the duration of the benefits measurement period, detailing what capability will be delivered, what local change is needed, what will be used to measure. It also includes the governance arrangements, performance management approach, and makes use of a healthy dose of competitive culture to their advantage.

A nice, clean way to answer the challenge – but tricky if Stakeholder Engagement is not one of your key skills.

Next up, EDF who have figured out a way to model their benefits with the support of BMT HiQ-Sigma. Across the key streams of work; Prevent, Remove, Harden, Understand and xxxx two of them were focused on the benefits of keeping the power plants in full working order and safe for the public.

They developed a number of KPIs to support the ongoing measuring of the plants not just on the benefits profile of the activity; in addition to project performance metrics.

The interesting part of this was the role changed for the team involved from working at the project level to realising that they could not do the work in isolation of other work happening and then without seeing the big picture at portfolio level.

The journey from project to programme to portfolio and back cemented one thing; spend as much effort developing your benefits as you do developing your costs profile.

DVLA came up next to talk about their implementation of Benefits Realisation Management. As usual the buy-in from the organisation is key for the survival of this activity, so the biggest piece of advice is to define the value of implementing BRM and keep it simple so people understand the journey.

The DVLA made it clear that they were providing tools and information for their leaders to make informed decisions for the good of the organisation. They utilised Lessons Learned to support the validation of the original benefits profile versus the results thereby providing analytical assurance of delivery.

Staying on the road, Highways England followed the DVLA supported by Wovex who explained to the delegates that benefits management is like dealing with treasure – you just have to figure out how much is out there and available to find.

Wovex’s view is that there is an unnecessary fog around benefits management and the clarity around profiles, dependencies and complexity. This is largely because benefits are often not linked to business cases, they are over-engineered, and are neither tailorable to suit the situation or drive good practice.

To clear the fog, organisations needs to make benefits management low cost to the business, get certainty of the results, and create clarity – clarity, confidence, certainty are the key words to think about.

Following the presentations, the panel got together and delegates were given an opportunity to ask questions. Most will be covered off by the Benefits SIG but a few notable ones are detailed here for completeness.

Even if we didn’t have benefits processes, wouldn’t benefits be achieved anyway? The view from Highways England was that because the benefits define the monitoring and therefore the level of rigour and focus required, that it would be unlikely that the benefits would be delivered effectively or in a visible way.

How early on should benefits be considered? EDF consider that ‘it’s too early to define benefits’ is frankly, a cop-out. You can always define what does this mean for me, my team or my organisation – it might not be validated early on but there is definitely opportunity to understand the why. Network Rail and the DVLA concurred with this point of view with the addition of stopping or removing the ability to start working on projects before a (simple) benefits case is defined.

During the next session, a Benefits SIG Committee member provided a detailed overview of Outcome Relationship Models (ORM) developed by Atkins and tested on several of their military programmes of work.

The premise around mapping relationships instead of standard benefits maps, is sound with milestones, interim outcomes, strategic direction, benefits, dependencies, all working together to demonstrate the theory of change.

Once a ORM diagram exists, it is used as a ‘poster’ for the project which details how important an outcome is, what controls are in place, and the measurement approach at a glance. A complex map to create, one not to be constrained by size (some are bigger than A1!).

I found this session personally a little heavy on detail so would have liked more of an overview with an opportunity to find out more at one of the workshops, but I like the content and the approach very much. I question if a lot of organisations that are early in their maturity and work hard to avoid creating a project plan would take the time to understand and build something like this instead though. I will definitely be exploring the topic more!

After a hearty lunch – I was in the group that got to go to the restaurant which was very pleasant and included amazing chocolate mousse cake, the workshops started. You can read about my workshop The PMO and its Role in Benefits Management here.
 

Emma-Ruth Arnaz-Pemberton
PMO SIG Chair