Are you testing the right things? How do you ensure that the configuration of your load and performance testing environment is similar to your production? There are thousands of properties and config settings, and they all might have a positive or negative impact on the reliability and performance of your application under test.
I assume that everyone working as a performance engineer has faced similar situations. You've prepared your load tests for weeks, executed the tests, collected the results, and crafted an excellent report. During the presentation of your findings, a decision-maker asked you:
"Are you sure that you've tested the latest, production-like configuration?"
Some of you might create a few screenshots showing the tested version but is this enough evidence? From my perspective, you can't control complex environments manually. However, I understand that nobody will take your load and performance testing results for granted without facts about the detailed environment configuration.
Automated change detection and comparison
Our business applications are getting increasingly complex and dynamic. For example, think about Microservices running in Docker containers on Kubernetes platforms. Such services may auto-scale depending on the user and transaction volume. Applications consist of hundreds of such services deployed on a multi-cloud backend system.
Your only chance to keep track of the configuration during your load and performance test is automated asset management. However, everyone hates error-prone manual CMDB (configuration management database) updates because we all know that this is never the whole truth.
Thanks to recent developments, we use tools such as Versio.io to enable automated configuration management of everything our business applications are built. Automation comes to the rescue, stores all changes securely, and allows reporting of the current state and comparison of entire environments.
The screenshot below highlights a automated environment comparisons
- We compared Dev and Prod database schema
- We identified differences in database Indexes
- Delta index history visualizes the development of this issue
Such a comparison detects that something changed, but you can also figure out its values before and after such changes.
Use Cases for automated environment comparison
#Migration to a new environment
Automated comparison of your servers, configurations, databases and more elements can save a lot of time and prevent you from running into outages. We all know that a single configuration mistake can result in severe performance problems.
Have you ever searched for a needle in a haystack? Finding configuration mistakes in complex environments is time-consuming. So instead, use the automated way and release your engineers from frustrating tasks.
#Application topology mapping
What are the dependencies of our services? Are there any external interfaces involved, and what technologies are in use? Such questions are difficult to answer, and without automated topology mapping and change detection, you won't have the whole truth.
#Continous change detection
Everything from the installed application version, its configuration, infrastructure, and database is changing all the time. Therefore, you should not run your tests before validating if the correct format is available. Creating a configuration baseline and comparing your actual config helps identify unexpected changes.
#Validation of security policies
How do you validate the proper implementation of security policies, such as blocking specific ports? You can do it manually, but this is very time-consuming. A better way is to use the automated environment configuration feature to check any misconfigurations.
Your Benefits?
Eliminate manual work
Everyone trusts your testing results
Reduce the number of re-tests
Simplify application environment management
Detect and eliminate configuration problems faster
Your Takeaway?
Automated change detection and environment comparison are crucial for your load and performance testing activities. They save time, provide transparency, and ensure that you run the tests on the correct configuration.
We are a Versio.io partner and are happy to support your automated change detection and environment comparison projects.
Keep up the great work! Happy Performance Engineering!