Posted on by & filed under Content - Highlights and Reviews, Programming & Development.

A guest post by Paul Lathrop, a software engineer for Krux Digital, Inc. In his prior life as an operations engineer, Paul specialized in building and/or breaking complex systems using Puppet for several companies including Digg, SimpleGeo, and Krux.

All problems in computer science can be solved by another level of indirection” — David Wheeler

The Puppet language is one of the most powerful aspects of Puppet. The syntax of the language is similar to a configuration file, making it approachable by operations engineers who are more comfortable tweaking files than they are writing code. Despite this superficial resemblance, the Puppet language is a powerful programming language that provides many of the constructs and data types programmers are used to from other languages. One of the ways to take configuration management to the next level is to leverage Puppet’s language to build powerful abstractions over the components that make up an infrastructure. By building on Puppet’s native resource types, you can create abstractions that describe an infrastructure in terms of the components used to create the platform that colleagues build their software on top of.

Building good abstractions is an iterative process that begins with some design decisions. Some questions are:

  • What are the top-level entities to deal with?
  • Do you think of these systems in terms of services, roles, or something else?
  • Can a given server have multiple services/roles/other?
  • What are the reusable components that make up those services/roles/other?

I’ve built several infrastructures based upon the answers to these questions, and I’ve found that a powerful and flexible pattern is to consider each server (whether physical or virtual) to have a single role: an application server, for example, or a static assets server. These roles are built up from a number of components that are abstractions built on top of Puppet’s native resource types. Common components are things like Apache site, supervised service, or at a lower level, a component directory.

The Puppet language provides two high-level language constructs to support the creation of abstractions: the define and the class. A class is appropriate for collections of resources that should only be defined once for any given host. A define is used for resources where it makes sense to have many on a given host. A great example of this breakdown is configuring Apache. You can use a class to install and configure the base Apache service, and you can use defines to set up individual sites, virtual hosts, and modules. (Where these high-level constructs are insufficient, you can
always drop down to Ruby and build your own custom types, which is a topic beyond the scope of this article.)

I usually build abstractions from the top down. As you create a new server role, try to reuse existing components as much as possible. When adding new functionality, consider whether that functionality is applicable to other roles that already exist. If you find a piece of functionality that is useful, create a separate component for it: a class or define in an appropriate module. Then, use that component to build up a new role. Similarly, look for functionality in existing roles that might be useful in the new role you’re creating. This kind of functionality is ripe to be re-factored into a reusable component.

To give a concrete example, at Krux we found ourselves following a common pattern. Most of the Puppet modules we created would recursively copy a directory from the master to a common parent directory on the server being configured. After copying and pasting that file resource a number of times we realized this would be ripe for an abstraction:

The defaults for this define reflect the most common case – we don’t purge existing files, we do recurse, and we set the mode on the files to 0755. These are parameters to the define because there are a few exceptions, or places where we want to purge existing files or use different permissions. We set the common parent directory in our base parameters class (kbase::params) where other global parameters are defined. This relatively simple define allows us to replace a block like this:

And replace it with the much simpler:

If we decide to change the common root directory for these component directories, we can make the change in a single location and be sure that it is updated everywhere, instead of searching and replacing every file resource manually. We can also easily find all of the instances of these component directories if we decide we want to remove them, replace them, or otherwise change the call everywhere.


Like most programming languages, Puppet gives you the power to create abstractions that increase the expressiveness of your code. Building good abstractions enables you to re-use code, improves maintainability, and allows you to avoid having to think of the details of problems you’ve already solved, giving you more time to solve the new problems you encounter.

Here are some Puppet resources from Safari Books Online.

Safari Books Online has the content you need

Pro Puppet is an in-depth guide to installing, using, and developing the popular configuration management tool Puppet. The book is a comprehensive follow-up to the previous title Pulling Strings with Puppet. Puppet provides a way to automate everything from user management to server configuration. You’ll learn how to create Puppet recipes, extend Puppet, and use Facter to gather configuration data from your servers.
Instant Puppet 3 Starter provides you with all the information that you need, from startup to complete confidence in its use. This book will explore and teach the core components of Puppet, consisting of setting up a working client and server and building your first custom module.
Puppet 3 Beginner’s Guide gets you up and running with Puppet straight away, with complete real world examples. Each chapter builds your skills, adding new Puppet features, always with a practical focus. You’ll learn everything you need to manage your whole infrastructure with Puppet.

About the author

PaulLathrop.Headshot Paul Lathrop is a software engineer for Krux Digital, Inc., building infrastructure and back-end services while working to bring down the walls between software and operations engineers. In his prior life as an operations engineer, Paul specialized in building and/or breaking complex systems using Puppet for several companies including Digg, SimpleGeo, and Krux, and can be reached at

Tags: abstractions, class, define, module, Puppet, resource types,

Comments are closed.