A full stack automation framework was designed and developed for the data warehouse testing program. Test strategy developed for Big Data, Data Lake, Cloud migration, BI report validation and rationalization included solutions and skill development.
With no in-house cloud testing solution, client testing team was forced to follow sampling manual testing approach for the validation of migrated data as part of the project. Infosys QA team addressed this inefficient process by working on a cloud solution that can automate cloud testing. This AWS cloud native solution was proactively offered to the client and was then implemented after a successful proof of concept (POC). Key features of the solution were:
- Cloud adoption leveraging private / public cloud
- Automated and optimized data validations in cloud using the cloud native solution
- Spark based utility which runs on Amazon Elastic MapReduce (EMR) cluster capable to validate the data stored in AWS S3 bucket, AWS Relational DBs and other external Java Database Compliant (JDBC) databases
Other solution features include:
- Full stack automation for in-sprint regression and automated Test Data Management (TDM)
- Workforce transformation with teams upskilled on diversified technology stack like ETL, Cloud, R, Amazon services, etc.
- Code tuning done on big data framework to incorporate multiple utilities in one framework using various technologies
- Centralized data management team for end-to-end data testing of data mart (big data) and data hub
- R-based data validation framework developed for validation
- DevOps adoption for shift-left quality measures
- Detailed reporting capability with drill down to each record