Helping a Government Agency Align for AI-Powered Impact
Artificial Intelligence
RevGen helped this government agency break down silos and accelerate AI opportunities through strategic collaboration
Delivering Analytic Ready Data Through a Snowflake Medallion Architecture
RevGen built a Snowflake Medallion architecture for our client to help consolidate historical on-premise data sources with new cloud platform data to deliver a comprehensive reporting and analytic environment.
Project Overview
Our client, a national manufacturer and wholesale distributor, wanted to rebuild their analytic and reporting architecture to drive greater data-driven decision making. They faced challenges with fragmented data across a legacy on-premise solution and new cloud platform that needed to be reconciled. So, they brought RevGen on board to consolidate disparate data sources into a single source of truth.
Client Challenge
In addition to the data reconciliation necessary, there were also challenges to be solved in building a new reporting model, and bringing on more analytics capabilities.
Historical data schema didn't align with new cloud ERP requirements
The legacy data had fundamental differences which needed to be understood and transformed to align with the new data model before it could be aggregated in the same tables.
Needed full redesign of the reporting data model
The consolidated data from both systems needed to be remodeled to ensure close alignment with the business-critical reports that they used to manage daily operations. Differences in functions between the two platforms meant reconfiguring data table relationships and dependencies.
Limited options for advanced analytics in new platform
Their platform selected for the new ERP included standard analytic capabilities. However, without an external platform such as Snowflake, the client could not leverage their operational data for advanced analytic applications including AI/ML.
Approach
We collaborated with stakeholders to define the data model requirements and transformation logic. Then, we used the client’s existing technology stack to build an automated data pipeline serving a Medallion architecture in Snowflake with consolidated historical and new ERP data prepared for reporting and analytics.
[Read More: Building a Data Pipeline for a Seamless ERP Migration]
Solution
By involving key stakeholders to understand the data mapping requirements, we were able to keep current reporting running smoothly while building a solution to take them into the future.
Comprehensive field level data mapping and requirements
RevGen led discovery sessions across all business areas to map the original reporting data model to the new requirements, leveraging the consolidated layer in the Medallion Gold layer.
Automated data pipeline with Matillion and Snowflake
We then built a twice-daily executing data pipeline using change data capture patterns to populate over 70 tables in the Bronze and Silver Medallion layers. This performed aggregations automatically, to build over 45 Gold layers for analytics.
100% data traceability from source to consumption
The Bronze, Silver, and Gold layers provide full traceability and audit capability on data throughout the pipeline.
Results
Because of the detailed and meticulous approach to the data reconciliation, the client was able to maintain current business functions and also had a smooth go live for the new ERP.
Improved confidence in data
Continuity in the business reporting and analytics through Snowflake enhanced confidence in the new ERP and helped ensure a smooth system go-live. With confidence in their data, the business was able to focus on data driven operational and process improvements.
Enhanced business controls with Snowflake framework
The consolidated data in Snowflake provided the base for data governance principles to maintain data quality and maximize the value of the new ERP data.
A secure and scalable framework for future development
The new architecture that includes the full code base, complete with templates and accelerators, empowers the client to extend the Medallion architecture as their business needs evolve over time. The analytically optimized data in the Gold layer is ready for use in advanced analytic models and AI applications.
Success Stories
Enabling a Major Telecommunications Provider to Prioritize and Operationalize AI
Artificial Intelligence
RevGen helped our client turn blue-sky ideas into a realistic, sequenced AI roadmap
Building a Data Pipeline for a Seamless ERP Migration
Technology Services
RevGen designed, built, and executed a Snowflake data pipeline to shift millions of client data records from legacy systems into a new cloud ERP platform.
Subscribe to our Newsletter
Get the latest updates and Insights from RevGen delivered straight to your inbox.