MySQL Migration to HP Vertica for OpenX

single image
Client  
         :  OpenX

Industry       :  Hi-Tech

Practice Area  :  Big Data Solutions


Background:

OpenX is a global leader in digital advertising technology and services, enabling digital media companies manage and maximize their ad revenue. Their unique platform conducts 250 billion monthly transactions with 12 billion daily bids from buyers.

The MySQL Architecture at OpenX was not robust enough to handle high volume reporting and delivering BI query results in a timely manner. The current infrastructure had outgrown and the data loads from HADOOP and MySQL were enormous causing the BI reports to take too long and fail. There was uneven distribution of data on the MySQL Cluster resulting in poor performance. Estuate provided a solution by migrating analytics and reporting to HP Vertica cluster.

The Opportunity:

  • Load MySQL and HADOOP data to a database that supports real-time analytics and high-volume reporting.
  • Provide fast BI query results.
  • Implement a highly scalable solution.
  • Highly reliable and robust infrastructure.

The Estuate Solution:

  • Estuate provided a highly scalable solution with HP Vertica which performed extremely well addressing to all their business and technical needs.
  • Implemented a highly reliable system that was fast for loading and retrieving data for high volume reporting.
  • Prepared the environment for migration.
  • Migrated both fact and dimension tables to HP Vertica.
  • 30 TB data was extracted from MySQL cluster with Informatica and the large tables were spilt into chunks of 300-500GB csv files for bulk loads.
  • Data was cleansed and mapped
  • Developed Python and Perl load scripts to load data to HP Vertica.
  • Developed the physical design model based on the logical model to maintain dependencies of all business objects.
  • Performed projection design and segmented and partitioned the data to meet specific business needs.
  • Integrated HP Vertica with Tableau for custom BI reporting.
  • Post migration checks and sanity testing for reports and troubleshooting in case of issues.
single image

Business Impact:

  • Migration was successful and HP Vertica proved its ability to scale.
  • Integration with Tableau for BI reporting tool was successful and much faster due to HP Vertica’s columnar architecture.
  • HP Vertica was highly responsive to High-volume reporting and real-time analytics were easily carried out to benefit the users.
  • The project was completed on time and within the budget.