Black Friday as a critical holiday shopping season delivers a spike in traffic that can cause massive issues. Having been aware of the consequence of failure, eCommerce companies allocate many hours to testing, tuning, and optimizing to assure that their platform works smoothly during the high-load time and provide a good user experience for their client.
Despite IT teams’ efforts to mitigate the issues happening during Black Friday or Cyber Monday, thoroughly preventing performance failure is almost inevitable due to third-party vendors or external resources. Therefore, we were curious about the state of the Top 10 e-commerce websites and conducted a detailed performance benchmark amongst these players.
Research Questions
How does a web performance benchmark of the Top 10 e-commerce websites look like?
How many are impacted by performance issues?
What are their top performance challenges?
Benchmarking Approach
Synthetic monitoring of their websites during a week
Test execution every 60 minutes from 3 locations
Used Dynatrace Synthetic Monitoring
Sample Monitoring Script
{
"version": "1.0",
"requests": [
{
"description": "alibaba.com/",
"url": "http://www.alibaba.com/",
"method": "GET",
"validation": {
"rules": [
{
"value": ">=400",
"passIfFound": false,
"type": "httpStatusesList"
}
]
},
"configuration": {
"acceptAnyCertificate": true,
"followRedirects": true
}
}
]
}
There are some difficulties to conduct this performance benchmark, such as
Lack of access to all backend layer
Hundreds of devices and browsers are in use
Customers are connected using all kinds of network speed and geographical connections
To lessen the complexity and provide a meaningful comparison, we used a SaaS-based synthetic monitoring approach. We use the start page web URL of the top 10 e-commerce websites and execute them every 60 minutes for seven days on the following locations.
Beijing
Iowa
Los Angeles
Although various metrics impact website metrics and performance, we considered the following metrics to prevent redundant complexity
Page load time
Time to first byte
Page size
How to compare these metrics?
Calculate the average Page Load Time, Time to First Byte, and Page Size for each website during the seven days monitoring periods.
Normalized results and expressed the average values as percentages to make a more meaningful comparison.
Sample calculation
Average response times
Website 1: 5 sec Page Load Time
Website 2: 6 sec Page Load Time
Website 3: 7 sec Page Load Time
Benchmark calculation
Website 1: 100 / summary (Website 1, Website 2, Website 3) * 5 = 28 %
Website 2: 100 / summary (Website 1, Website 2, Website 3) * 6 = 33 %
Website 3: 100 / summary (Website 1, Website 2, Website 3) * 7 = 39 %
Page Load Time Benchmark: (28 + 33 + 39 / 3) = 33%
Results
Page Load Times of Website 1 is better than the Benchmark.
Page Load Times of Website 3 is 4% above the Benchmark.
The results of our E-commerce Website Performance Benchmark
Our performance comparison demonstrated that 70 percent of e-commerce websites are facing performance problems.
As can be seen from the chart, 30 % of e-commerce websites suffer from higher page load times.
Another major issue is a high average value in time to first byte, which can be a symptom of slow backends. The chart indicates that 40 percent of the compared companies confront this issue.
Sixty percent of e-commerce websites are facing problems owing to high page size. Consequently, customers face a website forcing them to download massive data which in turn leads to performance slowdowns. Low network speed can also make such not optimized web page size and more serious issue.
Conclusion
There is no denying that the competition for gaining a vast number of customers and becoming a best-in-class user experience is still ongoing in the e-commerce companies, especially during the biggest shopping day, Black Friday. Fast-loading, responsive, and beautiful design are essential factors to keep current customers and attract new ones.
Performing modern continuous performance tests and comparing your website with your competitors regularly give you a chance to upgrade your production environment before increasing demand raises serious problems.
Comments