I am looking for a Test Strategy and approach to Data Conversion Testing between Systems. We have applications from Vendor and now the data needs to be moved to In-House Applications.
There will be ETL used as part of the development approach.
I need to define from the QA perspective an approach i could recommend and it has to be measurable. Will need to define Metrics around.
I'm working on a project similar to this now. We are taking a strategy that modifies our existing API so that it can read data from the other source as well and then verify results in both places.
I'm kind of in the opposite situation you are however. We are the source of the data and we are testing the results on the ETL process in the new schemas. We have a comprehensive API that is used in the ETL process itself, so making some extensions to the API testing framework to include validation from the post ETL locations is feasible. The API contains about 30,000 automated test cases so we are confident on the amount of data coverage we'll be getting.
After relying on our existing API automation for data validation our primary concern in the setup and deployment of the ETL process. We are prepping this ETL for delivery to customers.
This is a rather large project, so we are delivering metrics on performance, scalability, platform support, language support among a host of others.
[ QUOTE ]
JGBPhilly, can you share what kind of validation you are doing in the API with ETL please?
[/ QUOTE ]
Our testing framework for the API compares an expected result to an actual result returned by the API from the primary database. We are modifying this to compare against the field in the post ETL schema as well. Most of the ETL transformation relates to using this API to persist fields in the schema that are not persisted currently, essentially calculated fields that only live in memory.
The bulk of this work is handled with generated code based on XML schema data from the various databases.
[ QUOTE ]
These related to the transformation business rules?
[/ QUOTE ]
We have a limited number of elements that are currently unmapped to the API and are created in SQL by the ETL process itself. This is a manageable list and will be tested somewhat manually using direct SQL scripts.