Universities across the globe are trying to attract the best talents, and the digitalization of their services is a must-have because our digitized students expect outstanding user experience and are not used to slow or not responsive websites.
I bet you would also not apply to a University that provides a very outdated, slow, and not reliable booking portal. Many Universities realized that the user experience of their websites is crucial for their business and make load and performance testing part of their development streams.
Toolbox
LoadRunner 10.000 User License
Chrome Browser development Tools
System Monitoring Tools
SAP Monitoring Tools
System under Test
DB2 Database
SAP Application Server
SAP Web Server
3 x Security Proxy
OWASP Security Scan rules
Loadbalancer
Performance Requirement
Up to 28.000 Students
15-minute booking event duration
Each student will take up to 10 minutes for the booking
Check Booking Performance
Check the stability of Queuing System and find an optimal configuration
Ensure response times are within industry standards
Load Testing approach
We transformed all performance requirements into a performance testing plan to ensure everyone was on the same page. It's crucial to review such documents with all stakeholders, including infrastructure, application, and business teams.
Our load testing use case was complex and a very data-driven exercise. For example, students used several thousand modules during their event booking. When complexity is high, we always simplify things and try to run smaller, less complex experiments in the first place. The benefit is that we better understand the entire system, run smaller tests earlier, and reduce the load testing defect costs.
Step 1: Simplified Booking Load Tests
We had three booking types and several thousand different modules in this application under test. Each module resulted in checks in the backend, and students had only access to specific modules. Therefore, we focused on the most complex booking type and a set of similar modules only to simplify things.
Thanks to this approach, the LoadRunner script implementation and test data extraction were more straightforward. We've parametrized all dynamic requests and executed several load tests within a few days.
Surprisingly our simplified load tests already pointed out several performance bottlenecks in this SAP Module booking application, such as
Long-running DB queries
CPU pressure on the DB2 layer
Out of file handles on SAP Webserver
CPU pressure on Proxies
OWASP scan rules issues
After several iterations, we solved these bottlenecks and were ready for the more complex booking load tests. The benefits of this exercise were that we identified critical performance issues much earlier, and engineers had more time to work on remediation tasks.
Step 2: Complex Booking Load Tests
Our team learned a lot during the initial phase of this load testing project. Not only from a technical perspective but also the business rules behind such booking systems, the architecture, and how to extract and reset meaningful load testing data. After this in-depth understanding, we implement the complex data-driven load tests to support several thousand different booking modules.
The input data set consisted of 10.000 students and between 3 and 10 actual bookings of each student. The LoadRunner script implementation took approximately one week to deal with these dynamics. We've created LoadRunner functions for login, search, mark, book, and logout. Our book function was one of the most complex because it had to recognize the event type and handle several different booking types.
The load test execution started with 500 users, followed by 2500, 5000, and 10.000 user tests. After every test run, we executed SAP reports to clean up all bookings and bring back the data to the desired state.
Thanks to the initial tests, our second round of load tests identified fewer performance problems as expected, such as
Long-running SQL queries
Infrastructure sizing problems
Caching misconfigurations
Functional issues in the booking logic
Load testing is often tight to response time validation. Still, it addresses many more issues because it is the only test to ensure that the application fulfills your requirements under production-like conditions.
Step 3: Queue based Booking Load Tests
User behavior varies and depending on the type of application we are dealing with, it can be extreme, such as thousands of users pressing the login button simultaneously. To protect backend systems, so-called Queuing Systems can help reduce the pressure on business applications. Such a Queue acts as a load reduction gateway. It is a pass-through component to the main entry point, such as a login dialog, and pauses the login procedure for a specific time if the concurrency in the backend system is too high.
Crafting load tests for such Queue based systems involves a deep understanding of its actual integration. For example, in our project, it was part of the login form, and if an immediate pass-through was blocked, the system returned a referrer URL and forwarded the request to the waiting Queue. Scripting such behavior was a challenge, but after a detailed analysis, we solved the dynamics.
Our load tests on the Queue based event booking system pointed several problems out
The watchdog was not working as expected.
The automated throughput limitation was not working.
The static threshold was too high.
Your Takeaway
Don't underestimate the importance of user experience to the success of your business. You can't see speed and throughput during your development activities, and even functional testing will not give you insights. Remember always: Performance is our most important feature because nobody will use your product if it's not there.
We are here to take care of your booking system performance optimization. No matter how complex your booking portal is, we automate it, simulate real production scenarios, and ensure that your customers will feel the best user experience.
Keep up the great work! Happy Performance Engineering!