How to implant Performance in your SDLC – Part I

User expectations for speed have risen dramatically over the last 15 years. And research now shows that more than 50 percent of users abandon websites that fail to load within 3 seconds. The result? Customers simply spend their money using more reliable websites. In other words, your business revenue will soon plummet if you can’t deliver fast response times.

Amazon has found out that a 100 ms push in load times results in an increase of 10 percent in revenue. Although you’re probably not competing with any big players like Amazon, unreliable applications will still have a strong impact on your organization. This situation will usually result in:

1. longer processing times

2. more troubleshooting

3. an unreasonable challenge to the employees’ attention span

4. cuts in productivity

5. less time for innovation

6. massive design rework

Deploying to production quickly is also a priority, but at the same time, it’s not a good idea to be in too much of a hurry. Customers expect websites to always be up and to load at lightning speed. All these factors now make IT leaders integrate performance optimization into their development pipeline.

Performance in the Software Development Lifecycle

For many years, performance was just an afterthought and developers focused on implementing new features instead. The tester executed functional tests and then compared the actual results with those expected. But after the operational teams had deployed the new release to production, massive performance problems often appeared once the user community started to use the new product. This scenario is quite common, in fact. Nearly every week we see another story in the news about an unreliable website or service. Not only will customers stop using that company’s services and an expensive rework will be required, but a performance disaster is also very damaging to a company’s reputation, especially when it’s publicized.

In response, forward-thinking organizations have introduced new approaches such as shift left. But what does this actually mean, and how is it related to performance? In accordance with the defect-cost theory, this term refers to the point in time when you detect and, fix issues. Shift-left testing can reveal functional defects and even more importantly, can bring to light any of the performance flaws that often lead to a complex design rework.

It’s clear that massive design or code changes require some development time and introduce additional risks. Businesses want to avoid last minute changes and massive reworks. One successful strategy for minimizing the risk of last minute changes is to identify problem spots early in the development life cycle. This practice means that key metrics are validated immediately after the new build has been created. Specified thresholds help to determine whether to push the new build back to development or proceed with the testing stages. The key steps of such a fool-proof development pipeline that follows the shift left idea are:

1. automated build creation

2. automated deployment of new build on staging environment

3. automated functional, security and performance tests

4. automated update of quality gate

5. automated deployment to the system-integration stage one the quality gate is passed

6. automated production, user and transaction-volume testing

Although we know that early performance validation plays a major role, there’s still a missing piece in our puzzle. Performance engineering is all about simulating a realistic load and comparing performance metrics with the given requirements. Without specified requirements, how will you know if your performance test was a success? In the next section, I’ll outline how to derive meaningful performance requirements.

Nonfunctional requirements

If you can’t measure it, you can’t manage it. Just think about the meaning behind those words and you’ll realize how well this definition fits with our performance-engineering business. Too many projects neglect these nonfunctional requirements. They’re fully tied up with implementation and with testing the functionality and features. They recognize their mistake after deploying to production as soon as all the functional quality gates have been passed. Then the big nonfunctional issues appear. Users get frustrated because applications are too slow, support teams run around in endless troubleshooting, and business teams feel very disappointed with the low quality of their new system.

Meaningful nonfunctional requirements, on the other hand, help you to avoid such disasters. First and foremost—before your teams start with the application design—requirement experts need to document the “what´s” and “how´s” of the new system. And from the performance engineer’s perspective, the “how´s” are extremely important because those details form the input for your performance validation.

Here’s a sample of a performance requirement:

The page-load times for 90 percent, of all our CRM application must be under of 3 seconds for the average business day. This means user volumes of 500 concurrent users that execute 2000 CRM use cases per hour.

Page load times must be less than 5 seconds for 95 percent of our CRM applications on peak business days. These peak days involve user volumes of 700 concurrent users executing 3000 CRM use cases per hour.

System-resource utilization of all the servers involved must be under 90 percent both for average and peak business day user volumes.

Response times of up to 30 percent longer are acceptable in Asia and the U.S.

Best practices

1. Don’t start performance validation without having performance requirements

2. Current and future user and transaction volumes must be specified

3. Use percentiles (90th or 95th) for response times requirements

4. Note the relevance of your users’ geographical locations

5. Make nonfunctional requirements part of the contracts with your vendors

No matter whether your applications are in-house developed, or a supplier delivers the new IT services, nonfunctional requirements must be a fundamental part of your agreement with those parties. Developers, infrastructure teams, testing specialists and vendors have to build and test the new system to reflect those requirements. It’s a good idea to review all the nonfunctional requirements with the mentioned teams.

In my next blog post, I’ll share practical guidelines on how to kick-start a load- and performance-testing project, mentioning all the pitfalls you need to avoid. Feel free to make any comments or raise any questions. I’m always happy to help.

Keep doing the good work! Happy performance engineering!