Proactive is better than reactive

While the test unfolds, dozens of Hallmark employees pore over the data to get a head start on fixing anomalies well in advance of the real deal.

"If we notice that the load coming from the Des Moines [Iowa] data center is having a problem, we can drill into the analytics to see what's going on with the databases right at that millisecond," Lounibos said. "It could be a load balancer that's configured wrong or a switch setting that needs to be toggled a different way. Or, if it's a mobile environment, what kind of phones were they using? By being able to drill into the information, they're able to isolate potential problems."

Davis agreed, saying that the CloudTest platform's reporting tools enable Hallmark to correlate server statistics — for example, memory, CPU and caching — back to the actual testing event, which is quite helpful.

"SOASTA helps us know whether we're good at the hardware layer or the memory layer, or if there's some slowdown on the database," she said. "It helps us pinpoint where we need to tune."

The test — generally conducted for two to three weeks prior to the big event — provides a real-world picture that simply can't be produced in a laboratory, according to Lounibos.

'You can run these tests in minutes and get a really deep analysis of where you might have bottlenecks or problems that will cause latency or even [cause] the site to crash under stress," he said. "So, cloud computing has changed the way mobile applications and Web applications are forever tested, because it's fast, it scales and it's affordable."

Indeed, SOASTA has access to more than 500,000 cloud servers in 71 locations worldwide, provisioned by 17 different cloud service providers, including Amazon, IBM, HP, Microsoft and Rackspace, according to Lounibos.

"When Hallmark utilizes the SOASTA test cloud, the servers are made available on a per-hour basis to us, and they're really affordable," he said. "Hallmark only needs those servers for 3 or 4 weeks prior to Valentine's Day, so why would they buy a bunch of them and stick them in a back room somewhere, when they won't use them 11 months out of the year?"

Davis agreed that the flexibility and scalability provided by the SOASTA platform — used by Hallmark for the past 5 years — are attractive features.

"Before cloud computing, you couldn't just add a lot of servers for your peak and then turn them back off — you had to build for peak," Davis said. "The nice thing about cloud computing is that you can use them when you need them and turn them off when you don't. Having a smaller baseline footprint is a lot more cost-effective for us — we don't need a huge server farm with everything built out."