Kraft

Serving Up Application And Infrastructure Visibility For Kraft Foods

Kraft Foods is a powerhouse food and beverage company that is home to some of the world’s iconic brands including Planters, Philadelphia Cream Cheese, Kool-Aid, Oscar Mayer, Cracker Barrel, Maxwell House, Jell-O, and of course, Kraft Macaroni and Cheese. The company boasts annual sales of $18+ billion, and its brands are found in 98% of North American households. Kraft’s flagship consumer website, KraftRecipes.com, attracts 40,967,289 during their peak months, and during the holiday season, their typical visitor tally jumps by 30 to 40 percent. High availability and performance are top priorities, especially during peak holiday seasons.

Challenge: How many people are coming for dinner?

Anytime a website gets a major overhaul, whether it’s a technology migration or a complete redesign, issues are going to pop up. It’s inevitable when such a large amount of code and so many interdependencies are in play. So when the Canadian Kraft recipes site was being rebuilt during Q3-Q4 of 2013, the team knew there would be at least one big issue that would need to be solved. The new site was incapable of handling the amount of traffic Kraft projected post-launch. And the busy holiday season was coming up fast.

The website team included several vendors, which definitely complicated matters. Fortunately, one of those vendors was already using AppDynamics to monitor load and performance tests, and recommended that  Kraft reach out to AppDynamics. By now it was Friday, December 13th, and all the players were sitting around the same table. And that’s where they spent the rest of the weekend.

AppDynamics quickly deployed its agents, which automatically mapped Kraft’s infrastructure and applications. Soon, AppDynamics was capturing business transactions, which quickly began to identify core issues with code, stored procedures (database queries), and bottlenecks within the infrastructure. By Sunday night, after an entire weekend of performance and load testing, it was obvious that despite fixing numerous code defects and improving website configurations, the site could not handle the 2,600 business transactions per second, which was one of Kraft’s targeted KPIs.

AppDynamics easily drew out where the problem areas were top-to-bottom, both from a code perspective and an infrastructure perspective. Without AppDynamics, we would not have been able to identify the critical issues and get them fixed…

“There was no way we could refute the results AppDynamics brought to the table,” said Shawn Griffin, Kraft’s technical project manager, Information Systems. “It put an end to the finger-pointing. AppDynamics easily drew out where the problem areas were top-to-bottom, both from a code perspective and an infrastructure perspective. Without AppDynamics, we would not have been able to identify the critical issues and get them fixed, and meet the site launch deadline.”

Beyond helping to solve the immediate issues, AppDynamics identified for Kraft issues that needed to be addressed from an infrastructure perspective to ensure scalability and future growth of the platform, as well as a need to provide better governance over introducing new code provided by partners. AppDynamics demonstrated such superior value that Kraft signed them in 2014 as their number one application support partner, displacing SolarWinds and Keynote.

“The three products we’re leveraging from AppDynamics are .NET modules, the DB module, and the end-user monitoring tool,” Griffin says. “These three tools enable us to do what these other two server monitoring tools, SolarWinds and Keynote, could do, but better. What AppDynamics can do, those two cannot do real-time.”

These three tools [.NET, database, and end-user monitoring] enable us to do what these other two server monitoring tools, SolarWinds and Keynote, could do, but better. What AppDynamics can do, those two cannot do real-time.

Turkey Day Trepidation

That was 2013. Fast-forward to 2014, this time literally the 12th hour, 9:00 a.m. on Wednesday, the day before Thanksgiving. Under the crush of Thanksgiving cooks looking for ideas, recipes, and coupons, the Kraft Recipes U.S. site starts to falter.

“People were looking for their recipes and they were visiting every single one of our sites — Cool Whip, Jell-O, you name it. They were there,” Griffin recalls. “Our infrastructure was not handling the added heavy volume of traffic. The AppDynamics dashboard was showing page-load times of up to forty seconds for some business transactions. Then the Kraft Recipes site went down. We were literally getting ‘page not found’ errors. It was bad, and it was expensive. We have a monetized site and couldn’t serve web pages to our customers.”

Once again, Kraft reached out to AppDynamics, and immediately put into place additional monitoring agents for additional databases and SharePoint servers. Before long, AppDynamics identified several problem areas that were resulting in extremely slow transactions, including a seemingly innocent stored procedure that was the majority cause, resulting in deadlocks on the database.

Once AppDynamics pointed out the bottlenecks, Kraft was able to fix the problem code and also work with Rackspace to scale out their environment to handle the traffic that was hitting the site. The fixes were in place by the end of the day, so Kraft was able to handle the traffic on Thanksgiving day and maintain high availability through the Christmas season.

“Without AppDynamics, you’re just throwing darts in the dark.” 

Without AppDynamics, you’re just throwing darts in the dark.

Benefits: “Exceptional visibility into every corner of the application environment”

Griffin’s concluding statement points out the overarching benefit that AppDynamics was able to bring to Kraft: Exceptional visibility into every corner of the application environment. It enables Kraft to see when its vendors need to step up. It helps Kraft to identify architectural issues that impact overall performance. It takes away any ambiguity about root cause and what fix is needed.
AppDynamics .NET, database, and end-user monitoring agents give the Kraft teams exceptional visibility and accelerated troubleshooting every day, all year long. Since the incidents described above, AppDynamics has rolled out its Virtual War Room functionality, which supports Kraft’s multi-vendor structure, enabling efficient, real-time collaboration among the various players on the team. Everyone on the team can chat, make system changes in real time, and view and annotate events on second-interval charts, enabling everyone to collaborate in a view that unites their troubleshooting efforts and provides common context and common data to all.

AppDynamics enables us to capture, in real time, how changes in the environment, or new sites being introduced to the environment, are affecting the entire ecosystem.

“Kraft now uses AppDynamics in each and every one of our load tests. It is part of our strategy from a support standpoint,” Griffin says. “We have a strategy with continuous integration, and AppDynamics enables us to capture, in real time, how changes in the environment, or new sites being introduced to the environment, are affecting the entire ecosystem.”