Wayfare Consulting
136 S. 9th Street, Suite 450
Noblesville, IN 46060

Project Saves – 3TB Data Migration

Business Challenge

A data migration had been struggling for over two years. The 3tb database was supported on fragile hardware in an outdated model. The goal was to migrate the data into a redesigned data warehouse to allow much better and faster access. Data cleanup was necessary as well as reconfiguring the schema and reporting on the data as it was moved.

Solution

Wayfare asked for two weeks to evaluate the situation, determine a solution, and recast expectations. We went back to the basics establishing a solid project plan. We asked for and were approved to make a few personnel changes that focused on getting the right person in the right role. We were then able to build a solution that included checkpoints so that when a DB crash occurred, we didn’t have to restart at the beginning each time. We added error and DB integrity checking as well as regular system call outs to monitor progress. Where possible, we reported a problem and allowed the migration to continue with a checklist to fix data before (in case of rerun) and after, in case we finished with no fatal errors. We also added a performance specialist to speed up the migration through more efficient temporary DB and efficient DB calls.

The clients engaged our services to:

  • Build a project plan
  • Manage the team responsible for ETL and the DW build project

Results

Although we didn’t meet the delivery target of the team prior to taking over, we did establish a reasonable timeline and made steady progress going forward. The team was brought together and started performing with a few very clear goals and was able to start working together for the common good. The conversion ran 24/7 for more than two months.

Benefits

Objective

Benefits Achieved

Establish a reasonable timeline Created a project plan that included all of the work that needed to be done and worked with each role to determine the “most likely” estimates.
Make steady, documented progress Once we were able to track progress, we were able to determine where the slow calls were and focus appropriate resources to improve efficiency greatly reducing run time.
Eliminate the restart from the beginning This allowed a complete run-through of the data so that all problems, fixes, data checks, and efficiencies could be addressed for the final conversion.

 

Ready to start?