Posted on by & filed under Devops, infrastructure.

Creating a Fast Chef Development Environment

I recently had a six-hour flight across the country to visit our Boston office, and thought I’d get some work done using the Internet on the plane. I was trying to debug why a Chef community cookbook (‘application_python’) wasn’t working for me. The airplane wireless was fairly slow, which made pulling down packages and Chef cookbooks from the Internet during  kitchen test iterations too painful to bear.

It was the first time I’ve had Cookbook Karma hit me in the face at 30,000 feet: Why hadn’t I optimized this sooner?

In an effort to get something done, I sought a way to speed things up by reducing use of the network and local CPU, centering around two techniques:

  1. Not re-downloading files
  2. Testing only my last-most Chef code changes (but not things I already knew worked)

The below methods have made my Chef development workflow much more pleasant. I’m about twice as fast as I was before.

Making Test Kitchen Faster

With a little tweaking, Test Kitchen can be made to have a fully-functional Chef Server (via Chef-Zero). This means you can use the knife command in all its glory. This gives you the granularity to make and test only minor incremental changes.

Sample .kitchen.yml File

To get set up, you should make changes to your .kitchen.yml file. If you don’t have a cookbook to test with yet, you can generate a new cookbook with  chef generate cookbook my_app, then replace the contents of the .kitchen.yml file with the text below. If you have an existing cookbook, update parameters in your local .kitchen.yml as necessary to match this:

Connecting Docker with a Docker HTTP Proxy

Uncommenting the “links: squid:squid” and “http(s)_proxy” lines in the above .kitchen.yml will allow networking between your “my_app” Docker container and another Docker container running a Caching HTTP Proxy.

Run this command to launch an off-the-shelf Squid Docker container from Docker Hub:

This does the following:

  1. Runs a Docker container and assigns it the name ‘squid’, so other  docker commands (like ‘stop’ and ‘exec’) can reference that friendly name. This name gets put inside other containers’ /etc/hosts if they have a links: directive
  2. -d flag runs Docker in non-interactive daemon mode
  3. -p exposes port 3128 on the docker container to the host port 3128
  4. -e sets an environment variable. -e CACHE_MAX_SIZE=4000 and -e CACHE_MAX_OBJECT_SIZE=500 set the total cache size to 4000MB and the maximum size for each object to 500MB (default settings are 100MB and 4MB)
  5. ‘sameersbn/squid:latest’ is the resource name. This is its name locally and in Docker Hub. Specifying a name that’s not on your local machine will copy it down automatically.

Set all of your Kitchen Chef containers (or any host) to use this caching proxy while doing local development, and you’ll substantially reduce your network activity.

Set Breakpoints in your Chef Recipes with Chef-Shell

Chef-Shell is a debugging tool that allows you to set breakpoints within Chef recipes, so you can step forward and backward through statements in a chef-client run and inspect various resources (such as node attributes and data bags) as your Chef node converges. The technique is to spin up Chef-Zero (the in-memory Chef Server) from within the Kitchen container, and upload cookbooks to this local server via  knife upload. Then tweak the local client configuration so that you can run chef-shell and/or  chef-client. Both commands will start a run, and execute your node’s run_list

With this technique, you don’t have to use a complete kitchen test run to test smaller sections of code. It removes extras like Foodcritic, Rubocop, and ServerSpec, and allows you to update cookbooks and node data using knife rather than the traditional Test Kitchen way of destroying and re-creating the instance.

In a recipe, you can insert the following:

and Chef-Shell will break at that point allowing you to inspect the state of the local Chef node.

Using Knife inside Test Kitchen

Set up a mounted directory between your workstation and your containers by copying and pasting this block into your terminal:

Inside the chef-shell there’s a variety of commands you can run, including:

Inspecting variables this way is cleaner than peppering your Chef::Log.debug logfile output.

Copying your Working Cookbooks to a Running Test Kitchen Instance

After making changes in a text editor on your host machine, the quickest way I’ve found to get your updated recipes to an already-running Kitchen container is this:

Commit your Docker Images After a Chef-run

Every time you run kitchen destroy, the instance is purged – wiping out all Chef configuration. This is by design, but working around it has some benefits:

  • Saving the states of your containers to a Docker image will update the image in your local repository. Packages and other resources will already be installed the next time you try to install them.
  • You can bake Docker ‘golden images’ with a Chef-run to configure the container, then save the image for later use. Commit these images to your own private Docker Registry, or the public Docker Hub.

To save an image of a desired configuration: commit your image after a full Chef run. Or, set a breakpoint and commit from there:

docker commit ubuntu:14.04 `docker ps -lq`

This command saves a running container as an image for later use. The command arguments are:

  1. ‘ubuntu:14.04’ is the image name. By overwriting the official ‘ubuntu:14.04’ image, Chef’s changes are used the next time this image is run.
  2. docker ps -lq shows the last-run docker container ID. The flag  -l shows only the last-run process and  -q suppresses output headers (“Quiet mode”)


If you’ve ever waited more than a minute to test your Chef code you should benefit from some of the techniques covered above:

  1. Running knife from within a Test Kitchen instance
  2. Installing an easy-to-use HTTP caching proxy for all of your Test Kitchen runs
  3. Using chef-shell and chef-client from within a Test Kitchen container
  4. Uploading your working Chef cookbook changes to an already-running container
  5. Saving a converged container after running Chef on it, and re-using it later

Do you have ideas on how to speed up Chef development? Let me know in the comments!


Comments are closed.