Chef Blogs

Doing Wrapper Cookbooks Right

Julian Dunn | Posted on | cookbooks
One great thing about the Chef community is how various people have dreamed up ways to use Chef. One of the most popular patterns is the wrapper cookbook, first popularized by Awesome Chef Bryan Berry‘s blog post, How to Write Reusable Chef Cookbooks, Gangnam Style. Bryan’s post is barely a year old, and we’ve all learned a lot from using and writing wrapper cookbooks. In this post I’ll discuss some of the best practices for using wrapper cookbooks, and touch on some caveats as well.

The Origins of Social Coding

In the early days of Chef, forking a cookbook was common. For example, if I wrote a PostgreSQL cookbook and published it, it might be missing some features that you wanted. Or maybe you just didn’t like what that cookbook did. So you forked (made your own copy of) the cookbook, made your modifications, and ran that on your infrastructure.

With the rise of GitHub and the notion of social, collaborative coding via pull requests, forking without contributing back to the canonical source code started to wane. This isn’t simply altruistic behavior, though GitHub definitely encourages altruism through gamification. Contributing changes back means that the forker no longer has to maintain their own copy of the code. They can merely depend on the upstream project owner to maintain the software, with their fixes, and consume that software as an off-the-shelf component.

These are some of the reasons we encourage our customers — startups and enterprises alike — to sign the Opscode Contributor License Agreement (CLA). In addition to reaping the benefits of social coding, it means that companies are not required to support their contributions. Moreover, it means that patent and copyright issues are clarified up-front, thereby encouraging wide re-use of that code, free of legal issues.

Over time, Opscode developed the Community Site, the COOK project, and other tools to encourage social coding of cookbooks. These tools are akin to the Comprehensive Perl Archive Network, Rubygems or Maven Central, but for cookbook components. Just as software developers do not fork and modify an XML library if they want to parse XML, infrastructure automation developers should be able to depend on high-quality, well-maintained, reusable infrastructure-as-code components.

What is a Wrapper Cookbook? Why Might I Use One?

A wrapper cookbook wraps an upstream cookbook to change its behavior without forking it.

There are two main reasons you might want to do this:

  • Codifying the standard settings for your organization or business unit’s use of that cookbook without placing those attributes in a role
  • Modifying the behavior of an upstream cookbook.

Codifying Standards in Your Organization

Suppose I use the community ntp cookbook but I want to enforce a set of timeservers across my infrastructure. Instead of running this cookbook directly, I could create an acmeco-ntp cookbook with the following settings:

acmeco-ntp/attributes/default.rb

default['ntp']['peers'] = ['ntp1.acmeco.com', 'ntp2.acmeco.com']

acmeco-ntp/recipes/default.rb

include_recipe 'ntp'

Now I can simply run recipe[acmeco-ntp] in my infrastructure and the default settings will take effect.

Note that it is not necessary to use normal or override priority here. Dependent cookbooks are loaded first by Chef Client and their attribute files are evaluated before those of the caller.

Modifying Upstream Cookbook Behavior

Sometimes you want to modify the behavior of an upstream cookbook without forking it. For example, let’s take the PostgreSQL community cookbook. It installs whatever PostgreSQL packages come from your operating system distribution. Suppose you want to install version 9.3 of PostgreSQL on an operating system that would not natively provide it (e.g. RedHat Enterprise Linux 6) but those packages can be found in the official PostgreSQL Global Development Group (PGDG) repository.. How would you go about doing that? You could write a wrapper cookbook that set the right attributes:

acmeco-postgresql/attributes/default.rb

default['postgresql']['version'] = '9.3'
default['postgresql']['client']['packages'] = ["postgresql#{node['postgresql']['version'].split('.').join}-devel"]
default['postgresql']['server']['packages'] = ["postgresql#{node['postgresql']['version'].split('.').join}-server"]
default['postgresql']['contrib']['packages'] = ["postgresql#{node['postgresql']['version'].split('.').join}-contrib"]
default['postgresql']['dir'] = "/var/lib/pgsql/#{node['postgresql']['version']}/data"
default['postgresql']['server']['service_name'] = "postgresql-#{node['postgresql']['version']}"

acmeco-postgresql/recipes/default.rb

include_recipe 'postgresql::yum_pgdg_postgresql'
include_recipe 'postgresql::server'

What’s with the repetition of computed attributes in the wrapper? Well, the values for default['postgresql']['client']['packages'] and so on were calculated when the attributes were loaded by the dependency, so to recompute them based on the new value, we need to restate the expressions.

You could do all of this work in roles as well — and if you do, the computed attributes will be correctly resolved without this kind of repetition. This is another reason that roles are still valuable.

You can take this one step further: suppose you wanted to then derive the pg_hba.conf (the database access control file in PostgreSQL) through some external mechanism that isn’t supported in the upstream cookbook. No problem: you can also set an attribute in recipe context, before the include_recipe statements above:

pg_hba_hash = call_some_method_to_get_a_hash()
node.default['postgresql']['pg_hba'] = pg_hba_hash

Again, in recipe context, there is no need to use normal or override priority to achieve the desired effect. Default attributes set in recipe context are #2, and the attribute files are #1:

Advanced Upstream Cookbook Modification, a/k/a [Ab]using the Resource Collection for Fun and Profit

You can also use wrapper cookbooks to manipulate Chef’s Resource Collection. Put simply, the resource collection is the ordered list of resources, from the recipes in your expanded run list, that are to be run on a node. You can manipulate attributes of the resources in the resource collection. One common use case for this is to change the template used by an upstream cookbook to the caller’s cookbook. Again, suppose I’m using the PostgreSQL cookbook but I really hate the sysconfig template that it uses. I can simply make my own template inside the wrapper cookbook:

acmeco-postgresql/templates/pgsql.sysconfig.erb

PGDATA=<%= node['postgresql']['dir'] %>
<% if node['postgresql']['config'].attribute?("port") -%>
PGPORT=<%= node['postgresql']['config']['port'] %>
<% end -%>
PGCHEFS="Ohai" # or whatever changes you want to make

and “rewind” the resource collection definition after that resource has been loaded by recipe[postgresql::server] to change its cookbook attribute:

acmeco-postgresql/recipes/default.rb

include_recipe 'postgresql::yum_pgdg_postgresql'
include_recipe "postgresql::server"

resources("template[/etc/sysconfig/pgsql/#{node['postgresql']['server']['service_name']}]").cookbook 'acmeco-postgresql'

You can play this game with any other parameters to a previously defined resource that you want to change. Because Chef uses a two-phase execution model (compile, then converge), you can manipulate the results of that compilation in many different ways before convergence happens.

Bryan Berry’s Chef Rewind gem will also do this kind of manipulation.

Summary

Over the years, the Chef community has developed a plethora of high-quality, reusable components for infrastructure automation. Therefore, forking a community cookbook  — particularly an actively maintained one — is generally discouraged.

Wrapper cookbooks allow you to modify the behavior of upstream cookbooks without forking them. These modifications can be very straightforward, such as you might do with a role, except that they can contain logic to govern the changes you want to make. Or the modifications can get quite advanced, through altering the resources in the resource collection.

It’s useful to name your wrapper cookbooks with a standard prefix that denotes your organization (e.g. “oc-” is what we use at Opscode). That distinguishes your wrapper from the cookbook you’re wrapping.

Finally, you need not strictly adopt only wrapper cookbooks or only roles. Used effectively, both roles and wrapper cookbooks give you a wealth of tools to model your infrastructure effectively.