top of page
Writer's pictureJosef Mayrhofer

How to create a performance testing pipeline

Updated: Apr 15, 2022

Moving fast from an initial design to production requires flexible concepts such as agile or scrum. This approach enables teams to release a new feature to production within a matter of weeks. Obviously, the teams need a highly automated quality-assurance stream. Here, I’ll give you some ideas on how to use fully automated load and performance testing and how to integrate it into your development pipeline.

ContPerfTestMon (1)

Continuous performance testing

The process of starting tests and watching test executions is really not effective. Successful performance engineering is highly automated—it allows your engineers to focus entirely on performance analysis and optimization.

Continuous performance testing involves:

  1. adding performance stories to your sprints

  2. ensuring a performance baseline

  3. setting thresholds

  4. scheduling test runs

  5. adding more complex integration and end-to-end tests

For reliability., use existing unit tests or implement  either API or service-based test scripts. Because interfaces and services don’t change too often, such scripts can be adjusted within minutes.

Avoid simulating high-load volumes. Ideally, you should configure goal-based performance—this uses the same data and reproduces the same number of users and requests in each test run. This process gives you an accurate baseline that helps you quickly identify any response-time deviations.

You’ll find that relying entirely on API level tests can lead to blind spots. The orchestration of your services to end-to-end business cases is also something that performance engineers need to validate. Use business-case level performance testing  as a final quality gate for performance before you release the new product.

Continuous performance monitoring

As already mentioned, automation is key to the success of your development pipeline. Monitoring plays a crucial role in this picture by providing quick insights and preventing problems from recurring. The most important steps in your monitoring pipeline are to:

  1. use transaction tracing solutions

  2. capture errors, exceptions, service response times and more KPIs

  3. set thresholds and [or?] use automatic baselines

  4. automate problem analysis

  5. automate reporting

Don’t restrict access to such performance-monitoring data. By sharing your monitoring results with the entire value stream, you’ll identify issues quickly. Modern solutions such as Dynatrace have powerful problem analysis and reporting capabilities that simplify the entire bottleneck identification process.

Information radiators

Engineers tend to overlook the importance of sharing results with their peers and managers. Charting key performance indicators on monitoring screens or intranet pages will prevent any blindspots. Although engineers can generate some valuable results, they don’t always present their findings in the most effective way. A nicely crafted performance cockpit solves this blindspot.

Try to get monitoring screens or devices to visualize the health status of your pipeline. Response times, baselines, error rates, test coverage are among the key performance indicators your developer, tester and operational teams need to know about. These indicators let them view the current state and enable proactive intervention if something has been broken.

Success criteria

The list below will help you make your performance and monitoring pipeline a big success. Don’t risk too much, but consider an incremental rollout process. Make a plan, get the approval from your leads, then implement these tests within your next five sprints:

  1. showcase

  2. baseline

  3. remove the noise

  4. visualization

  5. performance gate

Start your engines and make performance testing and performance monitoring a fundamental part of your value stream. Contact us today for all queries concerning performance-engineering, testing and monitoring.

Keep doing the good things! Happy Performance Engineering!

14 views0 comments

Recent Posts

See All

Comments


bottom of page