We are proud to announce some major improvements recently implemented in InSpec. Jerry Aldrich and I, two members of Chef’s InSpec Engineering team, have added two features which considerably improve performance when used with the Windows platform.
Backend Caching Improvements
First, we have added backend caching for commands. This change enables InSpec to cache the result and output of any commands run on the target device during the InSpec run. When a command needs to be executed multiple times for multiple tests, this change will ensure InSpec only runs the command once, resulting in a huge time savings for slower-running commands.
This has huge performance gains for Windows as some commands used by Windows-specific resources can take significant time to run, such as the wmi resource. And if your profile checks multiple WMI objects, the time savings generated with caching enabled can be impressive! For example, With the backend caching changes we are seeing a 95% speed improvement across all Windows CIS profiles. This takes the average runtime of ~7 minutes down to about 20 seconds.
To ensure we don’t break existing profiles which may run commands multiple times and expect different output each time, this new caching feature is currently disabled by default. To enable it, use the --backend-cache option with inspec exec. Backend caching will be enabled by default beginning with InSpec 2.0. The latest versions of the audit cookbook and Chef Automate already have the backend cache feature enabled by default.
While performance issues with Windows was the primary driver for this new feature, we are seeing ~30% speed improvement across all Linux CIS profiles as well.
Backend caching is available in InSpec 1.47.0 and later.
Windows Local Execution Improvements
Second, we have enhanced how Windows commands are executed when InSpec is run locally (such as running inspec exec in a local PowerShell session, or via the audit cookbook). Previously, it would spawn a unique PowerShell session for each command which adds a significant amount of time to the execution of each command. This pain is easily felt the more resources and commands are used in a profile.
This new enhancement enables InSpec to create a single PowerShell session and communicate with it using a named pipe. This drastically improves the local execution time on Windows. For example, the Level 1 CIS Windows 2012R2 Member Server profile runtime decreased from approximately 20 minutes to approximately 1 minute.
This feature was introduced in version 0.31.0 of Train, InSpec’s underlying transport library. Since these changes to InSpec local execution are invisible to the end user, it enabled by default for both InSpec CLI and the audit cookbook when Train 0.31.0 is used.
Overall Impact
When the results of these two new enhancements are combined, the impact is quite staggering. For example, our tests show the average execution time Level 1 CIS Windows 2012R2 Member Server profile is now 11 seconds. That is a 99% decrease in execution time! Mind: blown.
The table below shows a sampling of some of the improved execution times for some fairly complex profiles using these new performance improvements:
| Profile | Before | After | Improvement |
|---|---|---|---|
| cis-windows2012-level1-domaincontroller-2.0.0-4 | 442s | 17s | 96.02% |
| cis-windows7-level1-3.0.0-4 | 64s | 11s | 82.12% |
| cis-windows8_1-level1-2.2.0-4 | 353s | 32s | 90.8% |
| cis-windows10-1511-level1-1.1.0-4 | 516s | 88s | 82.89% |
Join the Conversation
The InSpec team is working hard to ensure that your experience is as delightful as it can be. If you’d like to chat about these changes, or anything else InSpec related, join us in the #inspec channel on the Chef Community Slack.
