Working with data pipeline definition Files

The AWS Data Pipeline console provides us with three different options to get started with creating a new pipeline. You could use the architect mode, which is exactly what we ended up working with in the earlier section, or alternatively, use any one of the pre-defined templates as a boilerplate and build your pipeline fro them. Last but not the least, the console also provides you with an ability to upload your very own pipeline definition file, which is basically a collection of various pipeline objects and conditions written in a JSON format. In this section, we will be learning how to write our very own pipeline definitions and later, use the same for building a custom pipeline as well.

To start, ...

Get AWS Administration : The AWS Definitive Guide to core AWS service offerings and implementing AWS in your own environment now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.