The website team included several vendors, which definitely complicated matters. Fortunately, one of those vendors was already using AppDynamics to monitor load and performance tests, and recommended that Kraft reach out to AppDynamics. By now it was Friday, December 13th, and all the players were sitting around the same table. And that’s where they spent the rest of the weekend.AppDynamics quickly deployed its agents, which automatically mapped Kraft’s infrastructure and applications. Soon, AppDynamics was capturing business transactions, which quickly began to identify core issues with code, stored procedures (database queries), and bottlenecks within the infrastructure. By Sunday night, after an entire weekend of performance and load testing, it was obvious that despite fixing numerous code defects and improving website configurations, the site could not handle the 2,600 business transactions per second, which was one of Kraft’s targeted KPIs.
Beyond helping to solve the immediate issues, AppDynamics identified for Kraft issues that needed to be addressed from an infrastructure perspective to ensure scalability and future growth of the platform, as well as a need to provide better governance over introducing new code provided by partners. AppDynamics demonstrated such superior value that Kraft signed them in 2014 as their number one application support partner, displacing SolarWinds and Keynote.“The three products we’re leveraging from AppDynamics are .NET modules, the DB module, and the end-user monitoring tool,” Griffin says. “These three tools enable us to do what these other two server monitoring tools, SolarWinds and Keynote, could do, but better. What AppDynamics can do, those two cannot do real-time.”
That was 2013. Fast-forward to 2014, this time literally the 12th hour, 9:00 a.m. on Wednesday, the day before Thanksgiving. Under the crush of Thanksgiving cooks looking for ideas, recipes, and coupons, the Kraft Recipes U.S. site starts to falter.
“People were looking for their recipes and they were visiting every single one of our sites — Cool Whip, Jell-O, you name it. They were there,” Griffin recalls. “Our infrastructure was not handling the added heavy volume of traffic. The AppDynamics dashboard was showing page-load times of up to forty seconds for some business transactions. Then the Kraft Recipes site went down. We were literally getting ‘page not found’ errors. It was bad, and it was expensive. We have a monetized site and couldn’t serve web pages to our customers.”
Once again, Kraft reached out to AppDynamics, and immediately put into place additional monitoring agents for additional databases and SharePoint servers. Before long, AppDynamics identified several problem areas that were resulting in extremely slow transactions, including a seemingly innocent stored procedure that was the majority cause, resulting in deadlocks on the database.
Once AppDynamics pointed out the bottlenecks, Kraft was able to fix the problem code and also work with Rackspace to scale out their environment to handle the traffic that was hitting the site. The fixes were in place by the end of the day, so Kraft was able to handle the traffic on Thanksgiving day and maintain high availability through the Christmas season.
“Without AppDynamics, you’re just throwing darts in the dark.”