"Immediately we were able to see a lot of success with Chef and Chef. For our datacenter move, we faced the challenge of transitioning from an extremely heterogeneous physical infrastructure to a highly virtualized environment. Now, we spend more time on innovation. We can spend time thinking about how to do things better and more efficiently, with less time spent on interrupt-driven incident mitigation." Ryan Creasey, Principal Systems Engineer
IGN is a leading Internet media and services provider focused on the video game and entertainment enthusiast markets. Collectively, IGN's properties reach more than 70 million unique users worldwide (February 2011, comScore). Its network of video game-related properties (IGN.com, 1Up. com, GameSpy, FilePlanet, TeamXbox and others) is one of the web's leading video game information destinations; IGN also owns and operates leading men's lifestyle site, AskMen, and entertainment site, UGO.com. With an imminent data consolidation and migration to a single, large scale datacenter in Las Vegas on the horizon, IGN was faced with challenges including the immediate need for better configuration management, and the requirement to rapidly virtualize that hardware into a virtual configuration.
After considering several other industry solutions, IGN deployed their datacenter with Chef. The company has experienced significant success with Chef because it is implemented using a Ruby DSL, which allowed better integration of both centers, superior configuration management, OS flexibility, and easier virtualization and automation.
From its corporate headquarters in San Francisco and offices throughout North America, Europe and Australia, IGN Entertainment has become a leading Internet media and services provider focused on the video game and entertainment enthusiast markets.
As the company grew, IGN decided to consolidate its datacenter in Southern California with its new datacenter in Las Vegas, which immediately presented problems. Its legacy center was a traditional, old-school datacenter with no coherent architecture, comprised of a collage of different technologies based on both Linux and Windows infrastructures.
To affect the move to a single datacenter, IGN would need to move from physical to virtual platforms. It would need to take old hardware from its SoCal datacenter and rapidly provision it in a virtual configuration, for which management became a prime directive.
"We needed to do things in a reliable and idempotent fashion," says Ryan Creasey, Principal Systems Engineer at IGN. "We had basically done everything with homegrown scripts. We had entertained the idea of CFEngine and Puppet, but it didn’t facilitate what we wanted to get out of it."
Prior to Chef, IGN had written a significant portion of its libraries to partially automate some of its infrastructure in the same Ruby-based DSL that Chef utilized for the implementation of its Chef engine.
"We went with Chef primarily because it was implemented with Ruby DSL," says Creasey. "That would allow us to take previous libraries and integrate them into what we wanted to do with recipes and cookbooks that Chef would provide. We had a lot of success off the bat with Chef."
Though IGN initially envisioned running its own in-house Chef server, Chef and its expansive community drew them to Chef. "All of the deployments in Vegas were done using Chef," adds Creasey. "We didn’t have to worry about the infrastructure side and were in a better position to concentrate more on doing, and not how it gets done. It also allowed us to have a distributed automation framework that could be easily accessible from any environment: datacenter, cloud or local workstation."
Today, IGN’s Las Vegas infrastructure is comprised of a highly virtualized blade environment. Exploiting Chef’s ability for automation and configuration management, "We spend more time now on innovation, than being mired down with fighting fires all the time," Creasey says. "We can spend more time thinking about doing things better, more efficiently."
With a Chef-based architecture, IGN has also streamlined the continuous integration of their engineering team’s applications by configuring auto-deployments into a stage environment based off a production stack. "We take the run list for a production node and have our continuous integration agents build an entire node from scratch, along with a particular application build," Creasey says. "We can now guarantee that stage box would look like a production box. Providing that degree of quality control was simply not possible before without systems architecture automation."
"We’re using an open-source project called Vagrant," says Creasey, "which utilizes Chef to build Sun virtual-based boxes on an engineer’s local workstation. Chef was able to provide us a whole new degree of ownership for our systems. Treating infrastructure as code, you can take software development concepts and apply them to how the lifecycle of server configuration. We can take the proposed change to a server configuration through development, stage and production, and know that all of our bases have been covered. We provision a node with Vagrant on an engineer’s local workstation based off a production run list, and make changes locally. Then, we push the changed recipes to a stage environment and verify that nothing breaks when we’ve integrated that change in with other nodes. Pushing that change into production becomes much less stressful and can easily be done on a regular release schedule."
Overall, deploying with Chef alleviates unnecessary complexity and significantly reduces man-hours for maintenance and scalability. "I shudder to think how long it would have taken to migrate if we didn’t have a systems automation framework," concludes Creasey. "We did it quickly with Chef, and saved a lot of time and man-hours in the process."
With Chef, IGN was able to streamline its datacenter deployments and provide an agile solution for its scalability. When the need arises to add web servers to the pool to meet customer demand, it can be accomplished via hands-off automation. "We see peaks of traffic particularly when new games are announced or there are video game expos, like E3," says Creasey. "During these times our traffic will dramatically peak. We used to have to combat that by hand-building web servers to deal with that capacity a month or so in advance. With Chef we’ll be able to organically grow that pool in an automated way. For example, when the application servers are at 80 percent of some arbitrary performance metric; we can have some process spin up additional nodes with Chef, configure them with the right run list, add it to the farm and do it in minutes. Chef is exceeding our expectations. The product has been great."