Chef 11 In-Depth: Client Improvements
On the client side, Chef 11 development focused on a broad range of
improvements rather than a few major features. We had two sweet new
features from our Co-MVPs, and a handful of improvements from Opscode.
chef-apply
Chef has always been focused on providing a solid way to manage your
infrastructure at scale, which means that we have sometimes overlooked
opportunities to provide tools for quick and easy hacking. For example,
just to get started hacking on a recipe with chef-solo, you need to do
something like this:
echo 'cookbook_path "/tmp/cookbooks"' > /tmp/solo.rb
mkdir -p /tmp/cookbooks/test/recipes/
$EDITOR /tmp/cookbooks/test/recipes/default.rb
chef-solo -c /tmp/solo.rb -o test
In addition to the extra typing, there’s a fair bit of Chef knowledge,
like the structure of cookbooks, configuration parameters, and the
(newish) override run list feature embedded in these commands.
To make all this a lot easier, Chef 11 ships with a chef-apply command
that allows you to execute a single recipe from the command line.
Instead of the above you just need to do:
$EDITOR hack_the_planet.rb
chef-apply hack_the_planet.rb
Beyond being a cool new feature, chef-apply is also a great story
about the awesomeness of open source. chef-apply started life as a
quick hack that I posted to twitter.
I’d intended on cleaning it up someday, but life intervened, and the
code went nowhere. Then Bryan Berry
stepped in and did all the dirty work required to make it a part of core
Chef. It’s stuff like this that makes working on open source software
pretty much the best job ever. Thanks, Bryan!
Formatted Output By Default
When we developed why run mode for Chef 10.14, we realized we needed
Chef’s output to be more structured so users could easily scan the
information it generated. Since it’s such a big change to your
day-to-day interaction with Chef, we wanted to give everyone a while to
try out the formatted output and offer feedback before making it the
default. In Chef 11, we’ve made the switch: whenever running Chef in an
interactive terminal, you’ll get formatted output by default. We’re
confident that this will be a huge improvement in your experience when
running chef to check out changes you’ve made to your cookbooks.
Contrarily, the logger output is still better suited for offline
analysis or for integrating with log storage tools, so when you’re
piping Chef’s output to another program (like your system logger), Chef
will switch back to the logger output. If you want to override the
automatic output selection, there are two new command line flags:
--force-formatter: Formatters all the time--force-logger: Logger all the time
Note that the logger is not completely disabled when formatters are
enabled, because some kinds of messages, such as deprecation warnings,
are not yet integrated with output formatters. What we’ve done instead
is add a special log level, “auto”, and made this the default. When an
interactive terminal is detected, “auto” is equivalent to “warn,” and
when one is not detected, it’s equivalent to “info.” If you’re noticing
that you get both logger and formatter output, check your log_level
setting in your config file. It’s likely that nodes originally
bootrapped using knife bootstrap and upgraded from 10.x will need a
configuration update, since the bootstrap in 10.x configures the log
level to info by default.
In Chef 11, just as in 10.14+, you can change the formatter using the
-F FORMATTER option. You can also write your own output formatter if
you don’t like the default ones. For a fun example, have a look at
Andrea Campi’s nyan cat formatter.
Template Partials
If you’re using Chef to manage applications with a huge single
configuration file, you’ve probably wished you could break out the one
or two sections you care about and manage those individually. Or as
Bryan Berry put it in his feature request:
Templating such a large file is a mega PITA. JBoss AS 7 is nicely
componentized so that you can easily turn on and off modules. This
process would be a billion times easier and more readable if I could
include fragments within an erubis template.
Andrea Campi whipped up a pull request
and after a little design back-and-forth we now have this handy feature
in Chef. To use a partial from another template, all you need to do is:
<%= render "partial.conf.erb" %>
The render method takes some additional options that allow you to set
variables within the partial, lookup partials from disk, or use partials
from a different cookbook. Rather than explain all the features here,
I’ve created a sample cookbook
that shows off the major features.
Run Locking
Chef client and solo now use filesystem-based locking to ensure that
only one instance of Chef is actively configuring the system at a time.
This will prevent unexpected issues that could occur if, say, you run
Chef daemonized but SSH into a box to test cookbook changes or push an
urgent update.
Note that the locking mechanism uses flock. This means that if Chef
dies unexpectedly, the lock will automatically be released, so you don’t
need to deal with stale lockfiles blocking Chef runs. By default, the
lockfile is located at $file_cache_path/chef-client-running.pid. Some
filesystems (most notably NFS) don’t support flock, so you’ll need to
manually configure the lockfile location if you’re using such a
filesystem. You also need to customize the lockfile location if you’re
running multiple configurations on a single host. The config setting for
this is lockfile:
# In client.rb or solo.rb
lockfile "/not-nfs/chef-lockfile.pid"
Delayed Notifications Run on Failure
You’re probably familiar with this scenario:
- Chef reconfigures a service
- A delayed notification to restart the service is queued
- An unrelated resource fails and halts the Chef run
- Subsequent Chef runs don’t restart the service because it hasn’t been reconfigured during that run.
We originally felt this was the best design choice because restarting a
partially configured service could cause it to fail when it wouldn’t
have otherwise. In retrospect, we’ve found that it’s preferable to run
the delayed notifications on failure to ensure that all configuration
changes are applied, so Chef 11 changes this behavior: when a Chef run
fails, all queued notifications are run. If any of those fail, Chef
continues to run the remaining notifications and presents all failures
at the end.
Get It Now
Before upgrading head over to the breaking changes page
so you’re aware of the changes in this release. When you’re ready to
upgrade, hit up the Chef install page
to get Chef 11 and whip up some awesome.
