Making Healthcare.gov scale isn’t a quick fix

image_pdfimage_print

Health Insurance Marketplace - Please wait healthcare.govLike many of you, I’ve been following the news about the Obamacare situation. I’m not going to blame the President or his administration for the performance issues that Healthcare.gov are experiencing. I’m not going to blame the contractors that did the integration work. Instead I’m going to blame the colleges and universities around the world who teach computer science and programming.

I became a Bachelor of Computer Science many moons ago. While I wasn’t in a bar drinking Guinness, I studied the application development lifecycle from a bible-like book called “Software Engineering,” which detailed all the steps I needed to write modern applications. It covered everything from use cases to requirements to development, testing and maintenance. I graduated from college and started as a Java developer for a well-known U.S.-based systems integrator. I soon realized I had learned very little about developing applications that would work in the real world. Making an application function or work per spec is easy; making it perform and scale is an entirely different skillset.

downloadBut surely my “Software Engineering” book had a section called “Non-Functional Requirements” which included references to Performance and Scalability? Yes, it did, but no lecturer deemed those sections critical to my learning. Not once did someone tell me, “By the way, here are the top 25 things that will make your application run like a dog with 3 legs.” Building efficient code, designing scalable architectures and dealing with over 100,000 transactions per minute was a massive gap in my education. That’s why most developers today ignore the performance and scalability non-functional requirements, and leave them to QA teams to deal with once the application has been designed and developed. Simply put, performance is an afterthought.

I’ve spoken at a lot of conferences in the past two years for AppDynamics. In nearly all of them I’ve asked the audience, “How many of you performance test your code?” Usually one or two people raise their hands. This isn’t good enough, and it’s evidence of why websites and applications frequently slow down and break. Performance and Scalability are not something you can just “bolt on” to an application – these non-functional requirements must be designed from the start in any application architecture. Any experienced developer or architect knows this. The secret is that requirements will always change throughout the development lifecycle, especially now that agile is the de facto development process. The secret is to design an architecture that can scale and perform through change without needing to rewrite large numbers of components or services. Service Orientated Architectures (SOA) allowed developers to do this, because it provides the abstraction needed so new application services can be plugged in, and scaled horizontally when required. You then add virtualization and cloud into the mix and you’ve got on-demand computing power for those SOA services.

A key problem with SOA is that it creates service fragmentation, distribution and complexity. Developers and architects no longer manage just a few servers – they manage hundreds or thousands of servers that are shared and have lots of dependencies. If you truly want to test these huge SOA environments for performance and scalability you have to reproduce them in test, and that’s not easy. Imagine the Obama HealthCare.gov team trying to build a test environment and simulate millions of users across hundreds of different internal and remote third party services – an impossible task. Turns out it was an impossible task, their team only tested the websites health and performance a week before go live. Now, imagine if your application looked like this:

productionCraziness

 Screenshot above is from an AppDynamics Customer that leveraged SOA design principles. Healthcare.gov could be equally complex to manage and test.

Building fast and scalable applications is not easy. The answer isn’t simply to do more testing before you deploy your application in production. The answer is to design performance and scalability into your architecture right from the start. So yes, performance testing is better than no performance testing, but when HealthCare.gov has millions of concurrent users you really need an architecture that was built to scale from its inception.

Application performance monitoring like AppDynamics can definitely help identify the key architecture bottlenecks of HealthCare.gov in production, but the reality is the government is going to need a team of smart, dedicated architects and developers to make the required architecture changes and fixes. Unfortunately this isn’t something that can be achieved in hours or days.

If anyone needs to learn a lesson it should the colleges and universities around the world who educate students on how to build applications, so when students like me graduate, they are aware of what it takes to build scalable applications in the real world. Non-functional requirements like performance and scalability are just as important as the functional requirements we’ve all come to understand and practice in application development.

Healthcare.gov is just another example of what happens when non functional requirements are ignored.

Steve.

Copyright © 2014 AppDynamics. All rights Reserved.