Integrating Chef Analytics with Splunk

Ohai Chefs,

One of the exciting features in Chef Analytics 1.1.2 that is the ability to link Chef Analytics to Splunk. This features give you the ability to extract meaningful insights about your Chef infrastructure if you are using Splunk.

We have also created a basic Splunk App for Chef Analytics that showcases a few things you can do. Here are some screenshots from this app:

Nodes Activity Dashboard Screenshot Server Activity Dashboard Screenshot

To give this a try, check out the instructions on https://github.com/chef/analytics-splunk-app and create an issue here if you run into any problems.

Validatorless Bootstraps

Starting with the Chef 12.2.0 Client there is no longer any need to use the validation key to provision new chef nodes with knife. Furthermore, all that needs to be done to take advantage of this feature is to delete your validation keys and optionally remove the validator configuration from your knife.rb file.

Instead of shipping a validation key up to the newly provisioned node and having the node use the validation key to authorize itself to provision a new client and node, the knife bootstrap command will use the user’s key to create a client key for the node, use the client key to create a node object, and then ship the client key up to the node.

Configuration Details

Starting with Chef 12.2.0 existing Users will begin seeing a new banner on new knife bootstraps:

Doing old-style registration with the validation key at /home/lamont/.chef/myorg-validator.pem...
Delete your validation key in order to use your user credentials instead

In order to use the new behavior it is as simple as deleting the validator key:

rm -f /home/lamont/.chef/myorg-validator.pem

The existing validation_client_name and validation_key parameters in the knife.rb file can also be deleted. Note that the default value of the validation_key is “/etc/chef/validation.pem” and if that file happens to exist on the workstation or server that it will attempt to be used after removing the validation_key setting. That file should either also be deleted, or else the validation_key should be set to something like “/nonexist” to disable it.

Provisioning Details

The new output of knife bootstrap when not using a validation key will look similar to:

desktop% knife bootstrap 10.1.1.1 -N foo01.acme.org \
   -E dev -r 'role[base] -j '{ "foo": "bar" }' \
   --ssh-user vagrant --sudo
Node foo01.acme.org exists, overwrite it? (Y/N)
Client foo01.acme.org exists, overwrite it? (Y/N) 
Creating new client for foo01.acme.org
Creating new node for foo01.acme.org
Connecting to 10.1.1.1
10.1.1.1 Starting first Chef Client run...
[....etc...]

What you can see here is that if the node and client already exist that knife bootstrap will prompt to overwrite them. The ‘-y’ command line flag can be used to skip the prompts and answer ‘y’ to both questions. The new client is created first and the new node is then created with the client key.

Behind the scenes the ‘-r’ and ‘-E’ and ‘-j’ flags to knife bootstrap are already applied to the new node which gets created — so the object in the database will have its run_list, environment and initial ‘normal’ json attributes saved. This avoids the edge condition where for some reason if the initial chef-client run fails the node is never saved and it ‘forgets’ its own run_list and environment. Since the node is saved with that correct data before provisioning starts on the host, the run_list will still be correct even if the initial chef-client run fails for some reason.

Summary

The validatorless bootstrap changes to Chef 12.2.0 achieve a few key things:

  • No more need for the validation key (fewer things, reduced fussiness)
  • Ability to eliminate shared access (and ultimately have better auditing around provisioning actions)
  • Eliminating the first-run failure edge conditions where a node forgets its run_list, environment or attributes
  • Gracefully handling the situation where an old Client key or Node object exist in the database

ChefConf 2015: DevOps, Velocity, and Community

ChefConf 2015 welcomed 1500 attendees and 42 sponsors over our four day event. If you weren’t able to attend or just want to relive it all over again, here are a few of the highlights:

Keynotes:

Read more ›

Standard Bank: Our DevOps Journey Blog Posts

For those of you who would like to read Standard Bank: Our DevOps Journey in its entirety, here are the blog posts.

Standard Bank: Our DevOps Journey – The Final Chapter (Part 6)

This is the sixth and final entry in our ongoing, bi-weekly series examining our customer Standard Bank’s DevOps journey. You can read the first entry here, the second entry here, the third entry here, the fourth entry here, and the fifth entry here. Continue below for part six.

On February 11, the Chop Chop team went live with their prepaid feature. Currently, it’s available internally, on the Standard Bank network, but anyone who’s a part of that network can use it. The next day, there was a large, internal IT conference at Standard Bank where executives discussed what they want to do in the coming year. Dawie Olivier gave a presentation, showing how the app was promoted through the different quality gates into production, and how a person could log on and use it.

The Numbers

The Chop Chop team gathered some metrics to demonstrate what they’ve accomplished with Chef and DevOps.
  • Time to build the stack: 26 minutes to build for production nodes (2 web servers, 2 app servers, with end to end deployment and testing)
  • Number of automated tests embedded into Bamboo:
o   Test Environment – Total 209 Tests
  • 31 Infrastructure
  • 178 Application – already existed before Chop Chop
o   Production Environment – Total 39 Tests
  • 31 Infrastructure
  • 8 Application – written by Chop Chop
  • Number of cookbooks – 12 (5 custom, 7 community)
  • Time to market for pilot Internet Bank Refresh deployment – 12 weeks
  • Number of catalogued automated services: 3
o   Redhat Linux VM

o   Enterprise Application Platform (EAP)/JBoss on Redhat

o   Apache on Redhat

Read more ›

Hosted Chef oc-id Partial Failure

On Thursday, March 26th Hosted Chef experienced a degradation in service where logging into oc-id, Hosted Chef’s identify service, periodically failed. This failure meant that it was difficult to login into Supermarket, Hosted Chef’s profile page, and oc-id itself, since each of these systems rely on oc-id for their authentication tokens. During this degradation all other systems functioned normally and there was no impact to any Chef client runs that use Hosted Chef. Users that had already successfully logged into one of these systems also saw no impact, as this degradation only impacted new logins.

I’m sorry this impacted your ability to login to Hosted Chef and other services. In this post I’m going to talk about why this degradation occurred and what steps Chef is taking to ensure this does not happen again.

Read more ›

Chef Audit Mode: CIS Benchmarks

Today we’ve released an initial version of audit-cis. This is an “audit mode only” cookbook that runs on a node to check for compliance with The Center for Internet Security (CIS) benchmark for a specific platform. This release targets CentOS 7, CIS Benchmark version 1.0.0. Read more ›

Guest Post: Microsoft Announces Nano Server for Modern Apps and Cloud

This is a repost of a blog published today by our friends at Microsoft, including Jeffrey Snover, Distinguished Engineer and Lead Architect, Andrew Mason, Principal PM Manager, and Alan Back, Principal SWE Manager, who we have been working with extensively to ensure a best-in-class Windows experience with Chef. You can read the original post here.

Today we announced new container technologies as well as Nano Server, a purpose-built operating system designed to run born-in-the-cloud applications and containers. As customers adopt modern applications and next-generation cloud technologies, they need an OS that delivers speed, agility and lower resource consumption.

Nano Server is a deeply refactored version of Windows Server with a small footprint and remotely managed installation, optimized for the cloud and a DevOps workflow. It is designed for fewer patch and update events, faster restarts, better resource utilization and tighter security. Informed directly by our learnings from building and managing some of the world’s largest hyperscale cloud environments, and available in the next version of Windows Server, Nano Server focuses on two scenarios:

Read more ›

#HugOps and Community Post-ChefConf

This post originally appeared on Medium.

ChefConf 2015 is a wrap, and most of the community has returned home. For those of you not familiar with the conference, it’s the gathering of hundreds of Chef community members. We get together to learn about the latest and greatest in the industry (both the hows and the whys), as well as exchange ideas, brainstorm solutions, and give hugs, which has become the calling card of the DevOps community, and the Chef community in particular. Read more ›

Guest post: From zero to Chef and from Chef to the cloud

This is a guest post by Marco Meinardi from our friends at Flexiant.

Let’s say you need to deliver and operate an application in the cloud, at scale and velocity. Chef is of course essential for providing the automation that enables you to do this. There are other aspects that need to be addressed as well, though. You’ll need to create, adapt and integrate the infrastructure resources that the application will be running on; you’ll need to make sure Chef can operate on these infrastructure resources. And once your application goes live, you’ll need to continuously monitor its performance along with the underlying infrastructure. I’m sure you know what I’m talking about, and I’m sure you’re already using a set of tools to address it. Many cloud providers try to give you the required toolset, however, many times these are not packaged up to provide a seamless, integrated, automated experience through your entire delivery and operation workflows.

Read more ›