Mohegan Sun
Mohegan Sun Takes No Chances with Its Casino Apps
Key Benefits
Reduced troubleshooting time by 42%
Bird's-eye view of the business transactions and errors
Slow transactions increased in performance by 90%
Challenge: Managing applications in a highly complex, widely distributed environment
As Lead Application Architect for Mohegan Sun, George Lee is responsible for application development and platform integration across Mohegan Sun properties. With multiple heterogeneous systems to tie together into a responsive, coherent platform that encompasses back-of-house systems, front-house systems, point-of-sale, and customer-facing websites for both the Connecticut and Pennsylvania properties, Lee and his team have their jobs cut out for them.
According to Lee, quickly diagnosing and troubleshooting issues related to distributed systems is key. “With service-oriented architecture, you have a lot of different components, and you're only as good as your weakest link,” he says. “And whether it's a database or a faulty connection from one state to another or one piece of code to another, it can cause problems for our customers. AppDynamics lets us turn around answers fast, and alert IT folks when things aren't working correctly.”
Mohegan Sun uses the Adobe CQ AEM platform for web development and for hosting content. It connects to a Java-based Mule Enterprise Service Bus (ESB) that enables customers to log in and book hotel rooms or purchase tickets to shows. From there, a load balancer sprays traffic to the ESB farm where it’s transformed, remediated, and routed to the back-end systems.
Lee and his colleagues host domain-level services on a WebSphere application server that’s running on the IBM i operating environment. In turn, the IBM i environment hosts key gaming application systems that manage the slot machine floor and table games. Through the ESB, WebSphere also connects to Agilysys LMS, which is Mohegan Sun’s property management suite that lets guests book a room, check in, and communicate with the front desk.
As you might expect, Mohegan Sun also depends on multiple SQL Server and DB2 databases as well as internally developed applications that use WebSphere MQ components and JMS to communicate with each other. Breaking down the company’s software development, Lee says it’s about 60% Java, 30% XML, and 10% .NET.
“When we peeled the covers off the different products, AppDynamics was the one with the intelligence to find the business transactions automatically, which was huge.”
George LeeLead Application Architect @Mohegan Sun
About Mohegan Sun
A world at play and a world to its own, Mohegan Sun, created in 1996 by the Mohegan Tribe of Connecticut, is one of the world's most incredible destinations with some of New England's finest dining, hotel accommodations, retail shopping, live entertainment, and sporting events. Boasting three world-class casinos, Mohegan Sun is also host to the Kids Quest/Cyber Quest family entertainment facility, a luxurious day spa, convention center and meeting facility, a state-of-the-art Poker Room as well as three major entertainment venues with seating from 350 to 10,000. People from across the globe come to see live concerts with some of today's top headliners, along with major sporting events including Mohegan Sun's very own WNBA team, the Connecticut Sun.
Solution: Quick identification of issues, enabling rapid IT response
For George Lee, minimizing turnaround times is a must. Being able to address a system failure right away, long before it becomes an issue for the customer, is the very definition of success. Mohegan Sun had a monitoring product in place for infrastructure but it only monitored infrastructure and not applications. So Lee and his colleagues started their search for the best application-monitoring technology for their needs by installing trial editions of various products, “kicked the tires,” and discovered the intelligent automation of the AppDynamics application performance monitoring (APM) solution.
George Lee tells the story: “Normally when a guest experiences a performance issue, it gets escalated to the help desk, and it goes to support for troubleshooting. That's the alert piece of it. Then once IT is alerted, the staff greps through the log files from the web tier, the ESB tier, and the database tier to find the issue. Depending on the complexity of the issue, this can take a long time. On the other hand, with the alerting, tracing, and environment fully configured with the AppDynamics APM solution, you could turn it around in 5 or 10 minutes.”
One of the things Mohegan Sun wanted to avoid was any product that would require staffing up. “We looked at APM product selection from the perspective of what we could acquire without a ton of configuration, maintenance, and manual labor,” says Lee. “When we peeled the covers off the different products, AppDynamics fit the bill, and it was the one with the intelligence to find the business transactions automatically, which was huge.”
“On day one, within an hour of going live with AppDynamics, we were able to track and fix problems before the customers even noticed them.”
George LeeLead Application Architect @Mohegan Sun
Mohegan Sun’s timing for selecting and installing the AppDynamics APM solution couldn’t have been better. The company had just completed a system-of-records upgrade and a loyalty program migration. Lee and his team were also releasing 40 applications into production. So, they were in the midst of a testing marathon. As any developer knows, you can’t catch everything right out of the gate. However, the AppDynamics APM enabled the team to quickly identify critical code-level issues. “On day one, within an hour of going live with AppDynamics, we were able to track and fix problems before customers even noticed them,” Lee explains.
As to day-to-day operations, the AppDynamics APM solution has had a big impact as well, helping to tune Mohegan Sun’s WebSphere application server environment. Lee says that, even with IBM tools and best practices, it's still difficult to get all the statistics needed to optimize WebSphere. But with the perspective provided by AppDynamics monitoring, he can see what's going on with the servers, find out how quickly garbage collection is running, and appropriately size and tune the environment.
Speaking of garbage collection and slow transactions, JVM (Java virtual machine) pauses are one of Lee’s pet peeves. For Mohegan Sun, AppDynamics APM minimized them in a couple of hours. “That compares to what probably would have taken a week without it. Conservatively, slow transactions have increased in performance by 90%,” says Lee. Additionally, the team can reduce logging by 70%, as AppDynamics provides the data they need without wasting time in logs.
One more way that AppDynamics APM helps Mohegan Sun manage applications and business transaction speeds is by enabling IT to easily trace and troubleshoot transaction slowdowns or failures that originate with third-party vendors. With agents installed on vendor servers, the Mohegan Sun team gains insights into its performance through the snapshot execution waterfall view in AppDynamics to see where any trouble spots are located. AppDynamics automatically discovers and monitors application code running on Tomcat and provides deep visibility into how it executes through the JVM in production, as well as real-time alerting.
“If I had to choose one benefit as ‘the big one,’” says Lee, “I’d say it’s being able to look at all of the business transactions and their associated issues on one screen, one dashboard, because you're able to look at your whole environment holistically. Having a bird's-eye view that lets you see the errors in your environment—there’s nothing better.”
“Having a bird's-eye view that lets you see the errors in your environment—there’s nothing better.”
George Lee, Lead Application Architect, Mohegan Sun
