As Lead Application Architect for Mohegan Sun, George Lee is responsible for application development and platform integration across Mohegan Sun properties. With multiple heterogeneous systems to tie together into a responsive, coherent platform that encompasses back-of-house systems, front-house systems, point-of-sale, and customer-facing websites for both the Connecticut and Pennsylvania properties, Lee and his team have their jobs cut out for them.
According to Lee, quickly diagnosing and troubleshooting issues related to distributed systems is key. “With service-oriented architecture, you have a lot of different components, and you're only as good as your weakest link,” he says. “And whether it's a database or a faulty connection from one state to another or one piece of code to another, it can cause problems for our customers. AppDynamics lets us turn around answers fast, and alert IT folks when things aren't working correctly.”
Mohegan Sun uses the Adobe CQ AEM platform for web development and for hosting content. It connects to a Java-based Mule Enterprise Service Bus (ESB) that enables customers to log in and book hotel rooms or purchase tickets to shows. From there, a load balancer sprays traffic to the ESB farm where it’s transformed, remediated, and routed to the back-end systems.
Lee and his colleagues host domain-level services on a WebSphere application server that’s running on the IBM i operating environment. In turn, the IBM i environment hosts key gaming application systems that manage the slot machine floor and table games. Through the ESB, WebSphere also connects to Agilysys LMS, which is Mohegan Sun’s property management suite that lets guests book a room, check in, and communicate with the front desk.As you might expect, Mohegan Sun also depends on multiple SQL Server and DB2 databases as well as internally developed applications that use WebSphere MQ components and JMS to communicate with each other. Breaking down the company’s software development, Lee says it’s about 60% Java, 30% XML, and 10% .NET.
For George Lee, minimizing turnaround times is a must. Being able to address a system failure right away, long before it becomes an issue for the customer, is the very definition of success. Mohegan Sun had a monitoring product in place for infrastructure but it only monitored infrastructure and not applications. So Lee and his colleagues started their search for the best application-monitoring technology for their needs by installing trial editions of various products, “kicked the tires,” and discovered the intelligent automation of the AppDynamics APM solution.George Lee tells the story: “Normally when a guest experiences a performance issue, it gets escalated to the help desk, and it goes to support for troubleshooting. That's the alert piece of it. Then once IT is alerted, the staff greps through the log files from the web tier, the ESB tier, and the database tier to find the issue. Depending on the complexity of the issue, this can take a long time. On the other hand, with the alerting, tracing, and environment fully configured with the AppDynamics APM solution, you could turn it around in 5 or 10 minutes.”
Mohegan Sun’s timing for selecting and installing the AppDynamics APM solution couldn’t have been better. The company had just completed a system-of-records upgrade and a loyalty program migration. Lee and his team were also releasing 40 applications into production. So, they were in the midst of a testing marathon. As any developer knows, you can’t catch everything right out of the gate. However, the AppDynamics APM enabled the team to quickly identify critical code-level issues. “On day one, within an hour of going live with AppDynamics, we were able to track and fix problems before customers even noticed them,” Lee explains.As to day-to-day operations, the AppDynamics APM solution has had a big impact as well, helping to tune Mohegan Sun’s WebSphere application server environment. Lee says that, even with IBM tools and best practices, it's still difficult to get all the statistics needed to optimize WebSphere. But with the perspective provided by AppDynamics monitoring, he can see what's going on with the servers, find out how quickly garbage collection is running, and appropriately size and tune the environment.
One more way that AppDynamics APM helps Mohegan Sun manage applications and business transaction speeds is by enabling IT to easily trace and troubleshoot transaction slowdowns or failures that originate with third-party vendors. With agents installed on vendor servers, the Mohegan Sun team gains insights into its performance through the snapshot execution waterfall view in AppDynamics to see where any trouble spots are located. AppDynamics automatically discovers and monitors application code running on Tomcat and provides deep visibility into how it executes through the JVM in production, as well as real-time alerting.