The 3 Most Important Things To Do in a Data Migration Project

Most times when you implement a new database, you will need to migrate the data from an existing database (or databases, or spreadsheets…) to your new system. And assuming you don’t have just a few hundred records – in which case, it might be far more pragmatic to re-key them into your new database – you will need to perform a data migration exercise. Below are what I consider the three most important things to do in a data migration project:

• Create a Mapping document. You need to define exactly where in your new database all the fields in your old database will be mapped to. Put it in writing. Review it, clarify it, validate it. You might also define specific rules so that data from certain old fields could get mapped to different new fields depending on the value in the old field, or certain records do/don’t get migrated at all depending on a particular code/field value.
• Test Test Test. The minimum number of ‘trial iterations’ you should do is two, and for more complicated data migrations you may require more. After each iteration, test. Test record counts, financial totals, eye-ball records for field level matching, do performance testing, stress testing, test business processes, security, do regression testing. And so on and so on. Test test test.
• Make sure you don’t treat a new database implementation purely as a data migration project - and don't do a data migration project without remembering why you are implementing the new database in the first place. It is easy to do, to get so involved with the granular level of data that you lose sight of why you are implementing a new database in the first place. Remember the project benefits. Of course the data migration is a key element in a database implementation, and it can make-or-break at least the initial phase of such an implementation, but it’s not the be-all-and-end-all.

There is of course far, far more to the actual database implementation and many factors from that will feed into the data migration exercise (e.g. workshops, database design, change management and so on). So the caveat to the above is that this is written purely from the specific angle of the data migraiton itself.

Thus, whereas the first two points above are the most important in terms of the specific exercise of data migration, there is no doubt that the final point is the more important for you as an organisation and for your business needs.