Many people today are too focused just on preventing service downtime in their Q&A efforts. It is still not so uncommon to find managers, or even testers, who think that actual downtime (as in 100% service interruption) is the only thing that will noticeably hurt business.
Well, lately there has been a lot of data piling up that disputes this, and shows us that performance is pretty important too. A slow site means lost revenue due to users disappearing to other sites. As quality of Internet services in general is slowly rising, Internet users are getting less and less tolerant of slow sites and services.
Velocity 09
At this year's Velocity conference, Shopzilla, Google and Bing all reported findings that show how slowness can affect business. Shopzilla (presentation video, slides) had made a major performance redesign of their site, that had improved page load times from about 7 seconds to about 2 seconds. This had resulted in a 25% increase in pageviews and a 7-12% increase in revenue.
Bing and Google (presentation video, slides) experimented the other way - they introduced delay to see what would happen. Bing found that slowing down searches by 2 seconds means 4.3% lower revenues, while Google saw a 400ms added search delay result in a drop in traffic that was just below 1%.
There are two interesting things to note about the Google findings. The first is that they noted that traffic remained lower even after the delay was removed, which implies that some users who experience bad performance might go away never to come back.
Earlier studies
Another interesting thing to note is that Google ran a similar experiment around 1999/2000, recounted by Google's Marissa Mayer first at the Seattle conference on scalability 2007 (video), and then also on Velocity 09 (video). She let a test group get 30 search results per page instead of the normal 10 in a Google search. Those 20 extra search results meant that the page loaded in 0.9 seconds instead of the usual 0.5 (i.e. also about a 400ms slowdown). This resulted in an astounding 25% decrease in usage of the service.
It is not clear why the decrease was so dramatic then, while the 400ms delay added in the later experiment only resulted in a <1% dropoff. It might have been that the earlier result was also affected by things such as lower user satisfaction with the page layout/usability due to the extra information on the page, but one thing that just struck me was that another factor that may have contributed is brand recognition. Google wasn't very well known in 1999/2000, and users may have less patience when testing a new and unknown search engine, than when using a service that they have already been using daily for years and have gotten to know and trust. If that is in any way true, it means that new brands and services have even more to gain from testing and optimizing performance.
But wait, there is more!
A recent experiment was made by Strangeloop networks, where they did some A/B testing and offered some website visitors an unoptimized version of the website, while others got an optimized version. The result they found was that the optimized version of the site exhibited 16% higher conversion ratio and 5.5% higher average order value, than the unoptimized version of the site. Here is the article
Finally, I want to recommend this nice article: how slow websites impacts visitors and sales by the hosting provider Peer1. The article is full of statistics and references and argues, among other things, that users are more impatient and want faster sites today than they did 10 years ago. It suggests a mathematical model for calculating visitor loss as a function of page load time. Read it.
