Using variables to name jobs and transformations

Suppose that you have to process several files with sales, each file coming from a different branch of your business. You have three branches: A, B, and C. You load each file into a staging area, then you validate the data and load it into a final table. Supposing that you read one file at a time, a simplified version of your main Job could be as follows:

Sample ETL that loads sales data

Now let's add some complexity to the Job. Despite having the same kind of data, the files are in different locations and have different structure for each branch. For example, you could have Excel files or CSV ...

Get Learning Pentaho Data Integration 8 CE - Third Edition now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.