Monday, March 13, 2017

What we can learn from large, failed IT Projects

HOLMES system (Home Office Large Major Enquiry System) 
This is the first blog I am planning in an occasional series on lessons we can learn – good and bad - from commercial/public sector system implementations; looking outside the charity sector for best practice and alternative approaches we might take.


On Computing's website last week, there was an interesting article which says how “Accenture 'underestimated the complexity' of failed [£50m] national IT system for Police Scotland”. Whilst this is clearly on a different scale to all NFP CRM projects, there are some good lessons to consider regardless of what size project you are managing.1 The following are some of the key lessons I think we can take:

“Early disagreements between Police Scotland and Accenture led to a breakdown in relationships and a loss of trust that never fully recovered”.

This is a classic problem and shows how important client-supplier relationships are and supplier management for the client. One of the few things we can say with (99%!) certainty is that something will go wrong sometime during any CRM implementation, and therefore if you have set off on the right foot and started developing good relationships with your supplier, then when the beep hits the fan, you should be in a better place to manage such instances far more appropriately.

“Within weeks [of the contract award], and despite 18 months of pre-award discussions, Accenture and Police Scotland disagreed about whether the proposed system would deliver the requirements set out in the contract.”

First, if you are really arguing about scope and requirements “within weeks” of a contract award, then you have probably not managed the procurement appropriately for your requirements, and/or you are kicking off the project in the wrong way. And/or you have not created an appropriate Business Case pre-project which should have declared what the benefits were which you were expecting, and thus what you were looking for from a system.

In addition to initiating the implementation correctly, it shows how difficult and yet how important your procurement process and any associated tender (ITT) document can be. You should be able to define the fundamental requirements and scope enough so that you are not arguing so early, but I am not convinced that a long, complicated ITT document would necessarily mean you would avoid such issues.

For most CRM projects for charities, the procurement process should be about finding the right supplier, the right business partner for you, and their software/solution is part of that. But I do not think that, for most charities with their resources and needs, that a tender document can be certain to (or even should) cover the whole scope, encompass all functional and non-functional requirements, and do it to such a level that no disagreements will come later.

There are increasingly good reasons, therefore, especially for larger CRM projects and especially when building on CRM platforms, for doing an initial ‘discovery’ phase after the initial contract award which will spend more, detailed time detailing what is required from the rest of the project: what, when, how and who is responsible. To a certain level at least. And at least so that the client and supplier know more than they did pre-sale and so the supplier can refine and provide more accurate cost/time estimates on the agreed scope/requirements. Even if you are planning an Agile implementation thereafter, this ‘waterfall’ part of the implementation is well worth while.

“The programme was complicated and highly ambitious [and] the idea was to merge more than 130 legacy systems.”

Any IT/CRM programme where you know it is going to be ‘complicated and highly ambitious’ needs to be thought through more carefully before implementation. Can you really manage such a project in ‘one go’? Should you instead break it down into blocks/smaller sub-projects (regardless of project management methodology), thus considering how some aspects of it could be de-risked and the different blocks made less complex?

If you are going to merge several/many legacy systems, then this is a wonderful opportunity to break down the implementation into multiple blocks, prototype, do early go-lives for limited users/functionality, learn lessons as you go along. Even if those lessons show the project will be more complex than first thought, then get this lesson sooner rather than later.

You can still be considering ‘the overall’ project at the same time (e.g. considering how you can do merging/de-duping from each legacy system into one, looking at overall visions/benefits) whilst you are doing one or two of the early migrations.


"The two parties had initially believed that the majority of the system could be based on an existing IT system that Accenture had delivered elsewhere. But… as the design and development process developed, it became apparent that Accenture would need to develop significantly more than what had originally been anticipated."

If a supplier says the majority of their system can be based on an existing system they have, well, if that is the case, couldn’t you be shown that system up-front? And if so, then both sides could tell far sooner (i.e. pre-sale) if it really is viable to be re-used? Similarly, if it is that good, then couldn’t a simpler ‘fit gap analysis’ help in the first place? If that shows it is a larger gap than expected then, again, find out early.

If you are buying a CRM system based on a platform and you are being told there is 'an existing system' you can use, then ask for more evidence, do more work on investigations, plan how you are going to run the project accordingly and ideally, consider if the contract can reflect that.

And if your supplier does not have experience in your sector, with your requirements, then although that certainly does not necessarily mean that they could not do it, it does show that you need to be more careful about the procurement process.


“After Police Scotland tested the system, Accenture estimated that meeting the requirements of the contract would take an additional two-and-a-half years, with the live date pushed back to April 2018, almost four years later than originally planned. [And] Police officers found 12 critical errors that made the system unusable, with 76 defects in total that required more work.”

This shows that ideally you do not wait until UAT to test such fundamental issues. Ask how the design is being done earlier. Are there critical factors you can test earlier, even in a waterfall project? An ‘agile’ or iterative approach (even a hybrid waterfall/agile methodology) can also help mitigate some of the risks of doing UAT close to a go-live which might mean that only then will you find issues that could push back go-live so significantly.


“Accenture underestimated the complexity [of the project].”

This sums it all up and it is sadly something I have seen on a number of charity CRM implementations by (large and small) suppliers and business partners. Maybe they don’t know your business/sector well enough, maybe they have under-quoted even if they do know it, maybe you as a client didn’t explain the complexity well enough. I hope that all the above points should help mitigate some of this possibility.


1. I want to emphasise that this blog post is in no way suggesting that only Accenture have issues such as this! Clearly you could have the same issues with any supplier. And, of course, I don’t have any knowledge of this specific project except for what the article provided – but I do think there are generic points we can learn.