National Australia Bank (NAB) is a leading financial institution in Australia, renowned for exceptional service. Its 30,000 employees serve 9 million customers at over 900 global locations including Australia and New Zealand.
NAB’s regulatory reporting program addresses data quality and accuracy issues related to regulatory requirements. The program involved fetching data from over 25 sources to cloud databases. For effective validation, Infosys developed an automation framework to reduce manual effort and enable comprehensive data testing.
Key challenges
- Cloud testing involved key challenges in terms of effort and time for data validation as data from more than 25 heterogenous source systems was being ingested into cloud databases (AWS)
- Difficulty in reading the parquet/nested parquet files as they are in column oriented binary file format (non-human readable)
Ready to experience?
TALK TO EXPERTS
The Solution
Comprehensive automation framework for data validation across multiple sources hosted on cloud and on-premises
Infosys Cloud Testing practice, part of Infosys Cobalt, designed and developed a one touch automation tool, Automated Data Comparator (Auto DC) for the data warehouse testing program. The solution was developed using the technologies below:
- Language: Flask (Python), HTML, CSS, JavaScript (front-end coding)
- Database: Oracle, Redshift
- Python packages: PANDAS, DataCompy, Pytest, Logging, SubProcess
- Tools: Jenkins, JIRA, QTEST
This framework model was built for the following comparisons:
- File (parquet/CSV) to Database (cloud/on-prem)
- Database (cloud/on-prem) to Database (cloud/on-prem)
- File (parquet/CSV) to File (parquet/CSV)
End-to-end automated solution for multi-source cloud data validation
- Scalable, robust, fast, agile and automated data validation solution
- Integrates with Tricentis qTest
- Automates creation and execution of test cases
- Updates the status of test cases based on the results
Benefits
Zero defect delivery: 100% test coverage leading to zero production defects, Increase in test coverage which helps identify defects early, Incorrect data in production will lead client to report wrong values to the regulatory board and also it has a cascading impact of client paying penalties for being non-compliant.
High quality deliverables: Completely reusable components that can be customized as required with minimal effort, Flexibility to generate detailed test summary report after each test run.
Faster Time to market: Up to 80% effort saving in execution cycle for faster time to market
“At NAB, Infosys implemented a solution named Auto DC (Automated Data Comparator) for validation of data across heterogenous source systems. Auto DC was developed to communicate between on-premises and cloud-based assets and to ensure both source and target data is identical. This framework generates a comparison report with status based on the test results and enables us to identify defects earlier. It has reduced of test execution effort by 80%, helping NAB go to market faster.”
- Cathy J Jones, Quality Engineering Manager, National Australia Bank
