System Error – We should learn to love our IT failures – Planning to fail is success
This post was originally placed on the Linked In forum relating to the recent Institute of Government report – System Error – Fixing the flaws in Government IT. I was surprised not to get any comments so I thought to re-post here to see if we can get some debate going.
—————————————
I have worked on many IT projects in the public and private sector and in my experience, there is no real difference in levels of success between both.
The best learning I did on how to stop IT failure was at Nat West Corporate Bank many years ago. I was asked by the IT Director of the time to review the portfolio of 400 or so projects to see why there were always about 8 projects in crisis. What could be done to improve? Turns out, nothing could be done – plans were fine, methodologies were good, none of the people needed fixing. But we did fix it and here’s how.
All 400 projects had a risk register. In those projects, there were risks that were high impact but low probability. For example – project delayed because Thames floods – impact Very High, probability 1%. The problem was, when you get so many low probability risks together, then you actually end up with mathematical certainty that something will go wrong. In the case above, a number of projects were seriously impacted not because the Thames flooded, but because the servers were securely stored in the basement and when someone set off the sprinklers, 400,000 gallons of water duly turned the server room into a spark lit scene from Aliens 3.
So how did the Nat West team improve their success rate – by forming a hit squad who could go in at short notice to fix projects that were starting to or had gone wrong.
In 2011, how many public sector IT projects will get into problems because – the requirements change, key people move on, ministers change their mind, the idea proves to be not feasible in it’s original design etc? Answer – a lot, about the same number as in the private sector.
Having just finished running one of the most complex IT programmes I have ever encountered – businesslink.gov.uk – the programme’s key success was that we planned for failure. Of course we had releases that got delayed, ministerial changes, stakeholder problems etc. but when they arose, we had plans already created that detailed how we would deal with a problem of that type. Stakeholder and PR messages were already in place and resources were available to be allocated quickly to the remedial plan.
External review proved a valuable tool. Each year on businesslink.gov.uk the programme does an OGC Gate 0 review. It is a tough process but each time it was done, the delivery team found the results helpful, even if some of the recommendations were tough.
The methodologies mentioned in the report are good methodologies. The combination interesting. But those alone will not stop an over enthusiastic cleaner from polishing the sprinklers on the top floor of your office.
Sorry for the length of this post tried to keep it short.
Interesting post. What makes you feel the businesslink.gov.uk programme has been a success?
Steph Gray
March 11, 2011 at 10:52 pm
Steph, thanks – good point.
For me, the definition of success would be did you hit your objectives? In the case of businesslink.gov.uk, the objectives were:
1. Save businesses time and money in their dealings with government.
2. Help businesses start and grow.
3. Migrate 95% of online services for business from departmental web sites to businesslink.gov.uk by March 2011 – about 160 services in all.
The total benefits in relation to the first two were:
Established businesses – about £1bn per year.
Startup businesses – about £5bn per year.
Over 40,000 startups each year rate the site as ‘Very Helpful’ or ‘Critical’ to their ability to start up.
Both are good returns for a service that costs £20m a year to run. The benefits are measured by phoning up about 8,000 businesses and doing a 30 minute questionnaire with about 1,000 businesses. The businesses provide the benefit figures directly.
The third objective will be hit by March this year.
The web site has risen in natural traffic from 2m visits a year in 2006 to about 20 million visits in 2010. Not quite facebook but certainly the largest service in the UK in it’s vertical.
Happy to provide further info if that helps.
ddinsdale
March 13, 2011 at 3:00 pm
Great post David and as you know I am a big fan of the businesslink.gov website and what it has achieved.
I was struck by your final comment about the cleaner, and the contrast with the planning for the possibility of the Thames flooding. And I was also thinking about the current Japanese disaster and what we have been reminded about regarding Chernobyl. In Chernobyl the ‘plan’ was absolutely fine, it was the people on the ground who failed to follow the plan and thus caused the explosion etc. Ditto the Gulf oil spill last year.
And my point is?
I believe that a recurring weakness of many big projects is the failure to consult the people doing the job lower down the chain of command.
Decisionmakers are usually insulated from what is going on on the ground. They focus on feedback from user groups. They also get feedback from their own team, but any feedback from lower down the ranks is mediated by managers who understandably are trying to make their own teams look good. If you are dealing with numerous IT suppliers, it gets even more complicated.
There is nothing quite like a boss like you going and “walking the floor” as they say. If you asked the team in the basement what they thought the risks were, perhaps sprinklers would have been on their list. Too often, they even say “well the last two times it was the sprinklers” and the top management team are not even aware of these problems.
So consult everyone all the time? Of course not. But I know that every time my own company is asked how it would improve a project that we are involved in, we have lots of easy wins to suggest as well as views on more fundamental issues. I am a big believer in transparency and I think this approach results in a far better (and less expensive) outcome at the end of the day.
Rory MccGwire
March 17, 2011 at 11:00 am