Chef 11 In-Depth: Attributes Changes

Chef 11 In-Depth: Attributes Rewrite

Chef’s attributes system is frequently cited by power users as one of their favorite features. Chef users love having the flexibility to tune their applications based on a node’s role or environment. Since we first introduced the current attributes system in Chef 0.8, however, we’ve found a few ways to make attributes work even better, but to do so we needed to make some incompatible changes.

Read Vs. Write

The primary change in the revamped attributes implementation is that read and write have been separated syntactically. When node attributes were updated to support multiple precedence levels in Chef 0.8, an important design constraint was maintaining compatibility with the prior API. In Chef 0.7, for example, you would set attributes like this:

node["attribute"] = "value"

Chef 0.8 introduced the precedence-based API we’re familiar with today:

node.default["attribute"] = "default value"
node.set["attribute"] = "use this value"
node.override["attribute"] = "No, I really mean this"

Supporting both APIs was very difficult and required a lot of complexity in the implementation, and as a result attributes would sometimes exhibit surprising behaviors. To fix these bugs, we needed to simplify the code, and that meant dropping support for the Chef 0.7-style API. Now you must specify which precedence level you want to write to when setting attributes; accessing attributes on the node directly is only for reading. When reading attributes, a merged view of the components is generated. The merged attributes are made read-only, because if you were able to write to them, any changes would be lost the next time the attributes were regenerated. In practice, you’ll see something like this if you try to write to the read-only merged attributes:

node[:foo] = "oh no"
> Chef::Exceptions::ImmutableAttributeModification 
> Node attributes are read-only when you do not specify which precedence level to set.
> To set an attribute use code like `node.default["key"] = "value"'

Stricter API

We also took the opportunity to make the attributes API more strict. The new implementation still allows you to interact with attributes using Hash-like syntax with either String or Symbol keys, or with method calls via method_missing. When using the method_missing API, you now can only set values using setter syntax. For example, node.default.an_attribute("value") is no longer valid; you need to use code like node.default.an_attribute = "value" instead. The reasoning for this is best explained by example.

Attribute Mishaps

Lamont Granquist > this probably is an issue though:
Lamont Granquist >
Recipe Compile Error in /var/chef/cache/cookbooks/REDACTED/recipes/default.rb

Undefined node attribute or method `has_key' on `node'

Dan DeLeo > @Lamont did `has_key` work before? should be `has_key?` with the question mark, no?

Steven Danna > has_key probably worked by mistake?
Steven Danna > do you have a "has_key" attribute?

Lamont Granquist > hah

Lamont Granquist >
% knife node show i-d2da7caa -l -a has_key
has_key:  [REDACTED]

Steven Danna > @Lamont lolol

And the best part?

$ git blame FILE
b32f1ac5 (NAME REDACTED 2011-04-11 17:38:04 -0700 104) 

Role and Environment Attributes Visible Everywhere

One problem we noticed users running into was that Chef would give unexpected results when you used logic to compute an attribute in an attributes file. For example, suppose we want to set a virtual server’s host based on two other attributes. Even simple code like this wouldn’t work as expected:

node.default[“server_name”] = node[“app_name”] + “-” + node[“app_environment”]

This didn’t work because Chef internally managed the precedence of role attributes vs. attribute files by waiting to apply role attributes until after the attributes files were evaluated. If you had set app_name or app_environment in a role (or environment), you’d get the wrong result. In Chef 11, the attributes implementation now maintains role and attribute file-sourced attributes separately, so role and environment attributes are readable while attribute files are evaluated.

Ordered Evaluation of Attributes Files

In addition to setting attributes in roles, another common way to customize attribute values is by designing cookbooks to work together. Suppose you have the MySQL cookbook from the community site and a “mysql-with-our-tweaks” cookbook that modifies the other cookbook’s behavior for your needs. The natural place to tweak attributes with your site-specific modifications is in the attributes file of your “tweaks” cookbook. In Chef 10 and earlier, this could be frustrating because Chef would load attributes files in essentially random order, so you had to manually track dependencies between attributes files using the include_attribute directive. In Chef 11, we’ve fixed this. Attributes (and other cookbook components, aside from recipes) are now evaluated in order based on your run list and cookbooks’ dependencies; all of a cookbook’s dependencies appear before it in the final sort order, but the overall order is otherwise controlled by the run list.

Chef solo users should be aware that this feature is driven by dependencies specified in cookbook metadata. This means that chef-solo will now only load cookbooks that are directly in the run_list or reachable via the dependency chain. Additionally, chef-solo users will now see errors when a nonexistent cookbook is specified as a dependency.

Debugging Attributes

One nice side-effect of this change is that you can more easily debug where attribute values are getting set. To get a list of all the values set for a given key, use debug_value(:foo, :bar). For example:

# Role File:
default_attributes "test" => {"source" => "role default"}
override_attributes "test" => {"source" => "role override"}

# Attributes File:
default[:test][:source]  = "attributes default"
set[:test][:source]      = "attributes normal"
override[:test][:source] = "attributes override"

# Recipe:
require 'pp'
pp node.debug_value(:test, :source)

# Output:
[["set_unless_enabled?", false],
 ["default", "attributes default"],
 ["env_default", :not_present],
 ["role_default", "role default"],
 ["force_default", :not_present],
 ["normal", "attributes normal"],
 ["override", "attributes override"],
 ["role_override", "role override"],
 ["env_override", :not_present],
 ["force_override", :not_present],
 ["automatic", :not_present]]

Force Default and Override

An important consequence of splitting the environment and role attributes is that attributes you set in recipe files no longer overwrite values set in roles or environments. For most users, this isn’t a problem becuase they only set attributes in recipes to work around these problems. Some users, however, have adopted a workflow that heavily relies on cookbooks for specific applications setting different values than what the role or environment specifies. For these users, the above changes were a step backward. To mitigate this problem, we’ve added force_default and force_override attribute levels. These are available in attributes files, too, so you can keep all of your attribute logic in one place.

# attributes/default.rb
default["attribute"] = "a value set from a role will replace me"
force_default["attribute"] = "I will crush you, role attribute!"
# default! is an alias for force_default
default!["attribute"] = "The '!' means I win!"


Breaking changes are never easy, and we didn’t make these choices lightly. We believe we’ve done a good job of keeping the API and behavior mostly intact while removing the unexpected “gotchas” from the previous design. We think you’ll find the new attributes system will work just the way you’d expect with less opportunity for hidden mistakes to go unnoticed, and make Chef even more powerful.

Posted in release