TODO: Come Up With Clever Title

What's a little technical debt between friends?

Understanding Chef Testing Part 1

| Comments

This post is intended to be the first of a multi-part series on testing Chef code. This opening article will be 100% theory, talking about the whys and wherefores of testing when devising Chef-based automation. Subsequent posts will explore some of the different testing frameworks and tools available, along with practical examples of use.

Let’s dive in!

Definition of Terms

Before we begin, it may be useful to review some Chef terminology.

  • Resource: Any statement made in the Chef DSL is called a Resource. Resources are declarative. A full list of the Resources available in Chef is available on the Opscode documentation site. Here’s an example resource:
package "apache2" do
  action :install
  • Provider: A Provider is the code that Chef runs under the hood to turn a Resource into reality. In the example resource above, we never specified to Chef that it should “Fetch a tarball,” “Install an RPM,” or “Run apt-get.” Instead, the Chef interpreter inspects the system and determines which Provider will be most appropriate to satisfy the resource. The Opscode documentation site provides a list of all the providers available to the each resource.

  • Recipe: A recipe is a collection of Chef Resources and Ruby code, evaluated in order, from top-to-bottom. Example recipe (Taken from the zsh cookbook):

package "zsh" do
  action :install

case node['platform_family']
when "debian"
  package "zsh-doc" do
    action :install
when "rhel", "fedora"
  package "zsh-html" do
    action :install
  • Cookbook: A cookbook is a package of Chef code. Cookbooks include recipes, templates, libraries, attributes, and all the other data and logic necessary to interact with some piece of functionality on a server. Typical cookbooks include Java, Nginx, ZSH or Tomcat.

  • Unit Test: Dictionary definitions are lazy, but here’s one anyway. From Wikipedia:

    unit testing is a method by which individual units of source code, sets of one or more computer program modules together with associated control data, usage procedures, and operating procedures, are tested to determine if they are fit for use. Intuitively, one can view a unit as the smallest testable part of an application. (emphasis added)

With all these terms in mind, let’s look at the logic behind Chef testing.

Chef Testing Strategy

When devising a testing strategy for Chef code, there are two important questions to answer:

  1. When to test: At which point in the process of code execution will testing yield the most informative & actionable results about the fitness of “associated control data, usage procedures, and operating procedures?”
  2. What to test: What is the smallest individual unit of source code that will yield informative & actionable results data about fitness for use?

Because Chef is a declarative language, the answers to these questions become inextricable.

When To Test?

There are three points at which one can test Chef code:

  • Signal Input
    • Did I send Chef the correct command?
  • Signal Processing
    • Did Chef interpret and execute my command correctly?
  • Signal Output
    • Did my command, executed by Chef, achieve my desired result?

But at which of these points will we get the most interesting data? The answer to this question lies in…

What To Test?

We know that our unit tests exist to “verify the fitness of individual units of source code.” But in the context of Chef, what is the individual unit of source code? This is where it gets tricky:

Because of Chef’s Resource & Provider model, Chef itself is a testing engine. Either the intent expressed in any given resource is executed, or the Chef run fails!

For example (assuming a clean test environment):

package "apache2" do
  action :install

Because Opscode tests Chef, we know that this resource will work. Either this code will install the apache2 package, or Chef will throw a stack trace. We don’t have to test every resource declaration in our recipes over again!

This reveals two best practices when testing Chef code:

  1. The unit of test is the intent of the recipe. Instead of testing each individual resource in a recipe, test only those resources that best reflect the intent of the recipe.

  2. Because we know that Chef resources will correctly interpret intent, there is no need to test the signal processing phase. Instead of worrying about whether Chef resources will correctly interpret intent, ensure that the intent is correctly expressed, and that the expressed intent achieves the desired goal when processed by Chef.

Next time

We’ll discuss the planet Magrathea, and how a proper understanding of unit testing with declarative languages could have helped the Magratheans save a lot of time and effort.