The Challenge

The top retailer in the US faced challenges with scalability, maintenance, performance, data recovery, disaster recovery, increased operational overheads and governance while running the data on legacy data warehouses. Moreover, ensuring a reduction in application costs and processing time was essential.

To overcome these challenges, the client required solutions for establishing a consolidated Modern Data Platform on to Google Cloud Platform.

The Objective

Migrate the complex on-premise teradata, two Hadoop clusters and an existing clickstream solution on Azure to Google Cloud Platform. Further leverage the performance of BQ to deliver a solution that will streamline the retail operational functions across the network.

Solution

• Google recommended Datametica as a migration partner to the client to work on this complex and extensive project.
• Using Eagle – our automated assessment tool, we first analyzed the existing data warehouse, created a well-planned strategy with optimized data model recommendations. This helped in setting up a successful foundation for migration within weeks.
• We then used Raven – our code conversion tool to translate the source workloads (ETL/ELT) to target workloads (ETL/ELT). This saved 70% of time as compared to traditional methods of data conversion and translation. Raven based truly automated translation, provides a uniform, clean and highly maintainable translated code-base.
• Using Pelican – our automated tool for data validation, we ensured confidence in decommissioning existing legacy warehouses comprising more than 20,000 DB objects and 50 TB of data. The overall reconciliation process was completed with greater efficiency, granular comparison, zero data copy, no coding and by using faster techniques.
• One of the key challenges of this migration was to integrate applications hosted on Mainframe. Datametica Mainframe parser was heavily used to integrate more than 1,000 mainframe feeds including packed binary coded decimal files.
• End-to-End DevOps services was another key deliverable of this mammoth migration program. DataMetica built systems and processes for environment provisioning & setup, development/delivery pipeline, CI/CD based deployment pipelines and monitoring & alerting mechanism.
• With our toolset and deep expertise in GCP, the task was accomplished 12 times faster compared to traditional methods.

Accelerator Used

Customer Benefit