Title: Generating data quality rules and integration into ETL process
Abstract: Many data quality projects are integrated into data warehouse projects without enough time allocated for the data quality part, which leads to a need for a quicker data quality process implementation that can be easily adopted as the first stage of data warehouse implementation. We will see that many data quality rules can be implemented in a similar way, and thus generated based on metadata tables that store information about the rules. These generated rules are then used to check data in designated tables and mark erroneous records, or to do certain updates of invalid data. We will also store information about the rules violations in order to provide analysis of such data. This could give a significant insight into our source systems. Entire data quality process will be integrated into ETL process in order to achieve load of data warehouse that is as automated, as correct and as quick as possible. Only small number of records would be left for manual inspection and reprocessing.
Publication Year: 2009
Publication Date: 2009-11-06
Language: en
Type: article
Indexed In: ['crossref']
Access and Citation
Cited By Count: 21
AI Researcher Chatbot
Get quick answers to your questions about the article from our AI researcher chatbot