O'Reilly logo

Mastering Hadoop by Sandeep Karanth

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Pig performance optimizations

In this section, we will look at different performance parameters and how to tune them for optimized Pig script execution.

The optimization rules

Pig applies optimization rules on the generated logical plan for a Pig script. By default, all rules are enabled. The pig.optimizer.rules.disabled property can be used to disable rules. The –optimizer_off command-line option can also be used when executing a Pig script to disable rules. Some rules are mandatory and cannot be disabled. The all option disables all the non-mandatory rules:

set pig.optimizer.rules.disabled <comma-separated rules list>

Alternatively, you can use the following command:

pig –t|–optimizer_off [rule name | all]

Tip

FilterLogicExpressionSimplifier ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required