Meaningless Performance Metrics

As a performance engineer, I have witnessed many presentations of impressive numbers. A closer look often revealed they have zero impact:)

The top 5 meaningless metrics include:

1. Average Response Time

2. CPU Utilization

3. Number of Dashboards

4. Number of Performance Tests Executed

5. Total Alerts Generated

Why are these metrics not only meaningless but also misleading?

Let’s start with average response times. They tell a shiny story that is always too good to be true because one bad experience can ruin many good ones. Averages are skewed by minimum and maximum values. When a large proportion of unimportant requests are fast, but the crucial ones are slow, you will see acceptable average response times, and this is dangerous.

CPU utilization is worse. While infrastructure teams use it for their capacity planning, as a performance engineer, you need to be very mindful when presenting it. High CPU utilization is not always a bad thing as long as the user experience is ok. Similarly, low CPU utilization can be either good or bad. When CPU load is low and user experience is bad, it’s even worse. Rarely, CPU utilization tells you what matters.

Next, the number of Dashboards. Why do teams believe the more dashboards, the merrier? In fact, more dashboards don’t equal more insights. Data on glass is useless based on my experience. What we need is to navigate to the root from a Dashboard. Impressive, well-designed dashboards can be eye-catchers, but the real value lies in how effective they are in troubleshooting sessions.

Running dozens of performance tests does not guarantee better performance of a new release. In fact, the right test can produce much better results than hundreds of poorly designed runs. What matters is how your test mimics production workload, user, and data volumes.

And the last meaningless performance metrics on my top 5 list is the number of alerts generated. In fact, for this metric, while large numbers may sound impressive, they always indicate that the alerting configuration has failed to meet its most important goal. By escalating what matters and suppressing the noise, teams have more time for optimization and innovation instead of chasing meaningless noise.

Don’t fall into the trap of shiny metrics. Always ask yourself whether these shiny graphs and numbers create a real impact. In many cases, you will learn that they are mislearning or not supporting the mission.

Happy Performance Engineering!

#Performance #Metrics #PerformanceEngineering