Updated: May 2
I always prefer to take a holistic view in performance engineering. Others, who believe that agile-performance testing is the holy grail, often go down a single track that leads them to a dead end. They realize that some requirements can’t be validated within a CICD-testing train after all, After spending good money on agile-performance testing, they see a gap that stops them from reaching excellent user experience. In this post, I’ll share some pros and cons of agile-performance testing. It could be an eye-opener that will reveal how a holistic approach to performance engineering is the best way to go.
Pros of agile-performance engineering
You find issues earlier. Shift left testing is a popular practice. From C-Level to developer, everyone is familiar with the idea behind agile, and finding defects earlier in the SDLC certainly does enable your teams to reduce costs. Performance defects are often related to failures in application design or implementation, but changing both can be a massive programming task for your teams. Early identification enables you to avoid such issues and frees your resources for the development of new features.
Test coverage is king. The more functions your unit tests cover, the lower the chances are that those core features will contain errors. As usual, there’s no need to reinvent the wheel—just reuse existing unit tests for continuous performance validation. You should execute such unit-level baseline tests on every new build. The result is a continuous baseline that allows quick problem detection.
In the world of agile-performance testing, load and performance testing specialists attend regular scrum meetings. This close collaboration among team members makes performance a part of development-team thinking, as well as reducing the risk of running into serious design pitfalls.
Cons of agile-performance testing
“Agility” in this context means to move forward quickly while making continuous improvements at the same time. It allows your teams to push more releases to production in shorter time-frames, which increases the pressure on all the engineers participating in the value stream. Quick development, short testing cycles, immediate feedback and quick deployment to production is the credo. The nature of performance engineering means it can’t be fully automated because different tests are required to validate complex application landscapes. Performance-testing teams are often forced to work extra hours to run the many tests needed, and to deliver their findings within hours. I’ve seen the efforts of performance-testing teams increased by factor 3 after switching to agile-development processes.
You can certainly automate the test execution and reporting, and make these part of your build pipeline. The effort of doing this should not be underestimated, though. Increased test coverage is not for free. Reducing the risks of running into performance slowdowns requires some forward-thinking IT leaders committed to investing in the tooling and training required to transform the mindset of all the engineers involved.
The impact on my day-to-day activities
While I’ve been working in agile-performance testing teams, I’ve not only increased my application knowledge, but also learned how customers use our IT services. Such insights are highly beneficial for my performance testing and monitoring tasks. Admittedly, my simultaneous involvement in many different agile teams makes my task list quite long. Some of the tests can’t be executed during regular working hours because the environments are blocked for something else. We’ll soon need more engineers to meet the increasing demand for performance engineering. I always think twice before deciding which test should be executed.
In short, agile performance testing increases the need for automation, experience and good tooling. Don’t start your agile transformation journey until these are all in good shape.
Feel free to contact me or one of my team with any questions concerning performance testing or monitoring in an agile world
Happy performance engineering!