Pipeline example

The following example is written in Java and shows constructing and running a pipeline with three transforms; they are:

  • Transform to read
  • Transform to count the data
  • Transform to write out the results of the data

The code for the same is as follows:

public static void main(String[] args) {           // create a pipeline parameterized by command-line flags            PipelineOptions options = PipelineOptionsFactory.fromArgs(args);            Pipeline p = Pipeline.create(options);            p.apply(TextIO.Read.from(“gs://...”))               // read input            p.apply(new CountWords())                           // do processing            p.apply(TextIO.Write.to(“gs://...”));               // write output            p.run();                                            // run the pipeline}

Get Cloud Analytics with Google Cloud Platform now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.