Challenge: Limited visibility into production
Alan Novitskiy, Performance Lead at OpenTable, was responsible for ensuring high performance and throughput of OpenTable applications. “I work to detect bottlenecks, report them to engineers and make sure that we optimize the sections of our applications that are responsible for a good dining experience,” he said.
“Our applications really need to be fast because our major competitor is the phone,” Novitskiy said. Because online reservations should be a more convenient experience, OpenTable ran the risk of losing business if diners had to revert to phone reservations due to application performance problems.
However, even with rigorous pre-production testing, Novitskiy and his team could not always predict problems that would arise in the production environment. “We performed thorough code analysis and rigorous benchmarking in pre-production environments, but when we released code into the wild all kinds of new variables were introduced, and we didn't have full visibility into what those different variables were,” said Novitskiy.