Data migration is a complex and rigorous process. There is typically a transition from some kind of system we have to manage for launching a new system.
A data migration methodology and best practice has evolved as we have worked on consulting projects to move data. We wanted to share our approach here. Each project is different, but we work from a framework in our consulting which sheds light to how we manage the process.
A Data Migration Consultant’s Approach
Typically, we are working with a customer and they have used a legacy database of some kind to manage a CRM system, content management system (CMS) or a business database. There are various reasons for migrating. Once this decision has been made, then the strategies for delivering the goal is mapped out in our AscendWorks project system.
Much like moving from one house to another, data migration is a project which has a sequence of steps.
Here are some of the issues we address:
- Customization of the new system. A new system affords the opportunity for a new design. Our customers have had a track record working their business process in an older system. They can clarify requirements and help design the layout and data relationships in the new system. This is an important step. Using the house analogy, we create blueprints, design and build the new house with all the rooms and the infrastructure for plumbing, electricity, airflow, etc. The data relationships and impacts are carefully explored.
- Creating a data mapping document. A collaborative online spreadsheet managed in Google Apps is created and managed by a lead technology consultant on our team. Each sheet contains information of tables and field mapping. We take information from the old system and new system and lay them out. We use a bottom-up design approach which assumes a predetermined data structure (e.g. a list of attributes or tables and attributes which you need to incorporate into a design). We use this when there is an existing database or data source such as a business intelligence system or a data integration project. We use the opposite top-down approach in the absence of an existing structure and base it on requirements. This process depends on your requirements and how to represent data. We then identify the attributes of the data, which will become attributes in tables. Either way, the data mapping process gets accomplished from your requirements and starting point.
- Collaboration on the data map. We share out our master data mapping document and invite clarification and feedback from our customers. This is a rigorous, but important step. It ensures there is complete clarity on where fields and tables go inside the new system. The new house is built. We are deciding on where furniture from the old house goes. Everything needs to fit and each item needs a decision on migration. This is the master requirements document we work from and get sign-off on.
- Writing custom developed scripts. Based on the data map, we prepare a SQL database and where we can manipulate and transform the data. Our technology consultants develop software scripts for migrating the data per each table and field requirement. It is checked and rechecked.
- Present, approve and launch. Our customer is invited in to look at the new database with the migrations. We anticipate feedback and make changes quickly as we get those pieces of information. It’s important to differentiate new requirements generated from clarity derived from experiencing the new system from missed requirements from the data map. Once we have the final approval, the new system is launched. Timing the adoption of the new system such as a CRM system by a team needs to be coordinated as part of the project plans.
There are many details we manage behind the scenes, but the goal is good design and requirements.
Our clients enjoy being able to move forward in a new system with their past data. Their assets live in a new context and setting.
What data would you like to migrate from an older system?
