Creating a local Chef repository with Pulp

It’s very common these days to leave part, or even your whole farm, on an air gapped environment. It will definitely make it harder for malicious visitors to gain unauthorized access, but you’ll have some other things to worry about regarding the management of that infrastructure.

Chef is one of the most popular orchestration tools around, and if you need to bootstrap servers on an air gapped environment, it’s important that you keep a local repository at least with the products your company uses.

Chef’s rpm packages are made using omnibus. Unfortunately, that means every single dependency of that software will be declared in the package, making the repository’s yum metadata HUGE (currently it has about 2.1 GB).

Here is a simple way to create a local copy of the packages, and then create a local repository using Pulp, the best tool around for creating and managing repositories:

  • Create a local user, let’s say yum with all the defaults and add it to your /etc/sudoers file and login using that user
  • And then:
$ sudo yum install wget createrepo
$ sudo setenforce 0 # I won't cover SELinux permissions here
$ chmod o+x . # You'll need execute permission in your home directory to serve http

You’ll have to create the directories where your repo will be located, and run the wonderful wget tool for mirroring the packages:

$ mkdir -p repos/chef ; cd repos/chef
$ wget -r -N -np -nH --cut-dirs=3  https://packages.chef.io/repos/yum/stable/el/7/x86_64/

Brace yourself because there’s a LOT of packages to download and it will take a while. The good thing here is that this command will only download new/changed files, so it’s safe to add it to your crontab and let it run every night. Be aware that Chef imposes a transfer limit per IP, so if you abuse you’ll be temporarily blacklisted. Use wget to fetch the packages to a single central location, and redistribute from there internally as needed.

Once done, you’ll need to run:

$ createrepo /home/$USER/repos/chef/el/7/x86_64

Now follow Pulp’s documentation to successfully install it, and then change the verify_ssl parameter to false on /etc/pulp/server.conf. Run the following commands so Pulp creates your repository:

$ pulp-admin login -u admin -p admin
$ pulp-admin rpm repo create --repo-id=chef --serve-http=true --serve-https=true \
 --relative-url=chef --description 'Chef Software Repo' \
 --display-name 'Chef Software Repo' --feed file:///home/$USER/repos/chef/el/7/x86_64/
$ pulp-admin rpm repo sync run --repo-id=chef # This will probably take a while
$ pulp-admin rpm repo sync schedules create -s 2017-04-12T12:00:00Z/P1DT  --repo-id chef

Now you can access your own local Chef repository at:

http://ip-address/pulp/repos/

That’s it! Hope this helps you.

Thiago Vinhas

Former Chef Employee

Join Automate for Good hackathon and be eligible to win $60000 in prizes
chefconf-banner
image_334

FOLLOW @CHEF