SAP Data Services 4.x Cookbook

Chapter 7. Validating and Cleansing Data

Here are the recipes presented in this chapter:

• Creating validation functions
• Using validation functions with the Validation transform
• Reporting data validation results
• Using regular expression support to validate data
• Enabling dataflow audit
• Data Quality transforms – cleansing your data

Introduction

This chapter introduces the concepts of validating methods that can be applied to the data passing through ETL processes in order to cleanse and conform it according to the defined Data Quality standards. It includes validation methods that consist of defining validation expressions with the help of validation functions and then splitting data into two data sets: valid and invalid data. Invalid data that does not pass ...