Defining and Analyzing Service Level Agreements (SLAs) in Performator

When testing an application, someone from either the IT team or the business side often has expectations about the system's performance under test. Depending on where you work and your perspective, these expectations may be called Key Performance Indicators (KPIs), Service Level Objectives (SLOs), or Service Level Agreements (SLAs). The Performator team chose to use Service Level Agreements as it seems to be the most widely understood.

In this blog post, we will explain how to analyze and define SLAs for your metrics.

Defining SLA using HSR

Performator uses the Hierarchical Statistics Report (HSR) engine to define its SLAs. You can define an SLA and reuse it on multiple start()-end()-blocks, as seen below:

private static final HSRSLA MY_SLA = 
			new HSRSLA(HSRMetric.p90, Operator.LTE, 100); // 90th percentile <= 100ms
HSR.start("MyMetric", MY_SLA);
	// Code to measure
HSR.end();
HSR.start("MyOtherMetric", MY_SLA);
	// More code to measure
HSR.end();

You can also define multiple conditions for an SLA. The example below checks for the 90th percentile and failure rate:

private static final HSRSLA MY_SLA = 
        new HSRSLA(HSRMetric.p90, Operator.LTE, 100) // 90th percentile <= 100ms
              .and(HSRMetric.failrate, Operator.LT, 10); // failrate <= 10%

Defining SLA on HTTP Requests and SQLs

The PFRHttp Plugin provides you with methods to add an SLA to your HTTP request. Below is an example of both methods; Be aware that you can only add one SLA per request.

PFRHttp.create("API Request", url+"/api") 
    .sla(MY_SLA)	// add a reusable SLA 
    .sla(HSRMetric.avg, Operator.LTE, 100)	// add an SLA directly
    .send();

The same works with the PFRDB interface used to test SQL statements:

private static PFRDB db = 
    PFRDB.initDBInterfacePostgres("localhost", 5432, "dbname", "user", "pwd");
db.create()
  .sla(MY_SLA)	// add a reusable SLA 
  .sla(HSRMetric.p90, Operator.LTE, 100)
  .query("SELECT id, time, endtime from hsr_tests LIMIT 100")
  .close();

Analyzing SLAs

The evaluation of SLAs will be reported in the following two fields:

• ok_sla:  Contains the values for the evaluation of Service Level Agreements (SLA). Either 1 for true if the SLA was met, or 0 for false if the SLA was not met.

  • Shown on the UI as OK and Not OK.

• nok_sla: The same as above, but in 0/1 format.

  
  

The most efficient way to analyze SLA evaluations is through the HTML Report. It shows two tables, OK and NOK (Not OK), as well as a bar graph that shows the behavior of the SLA over time.

Real-Time Analysis of SLAs

To analayze SLA's while your test is running, report the metrics to a database and use the dashboard template for the Engineered Monitoring Platform (EMP).

The SLAs are fetched from the database in real time, allowing you to observe what is happening while your test is still running:

Conclusion

Performator makes it simple to define SLAs on your metrics. Its reports visualize how each SLA was evaluated, and if you store your metrics in a database during a test, you can also evaluate SLAs in real time.

Learning how to use Performator can take some getting used to. If you would like to learn more about using Performator for your needs, contact Performetriks and let our experts do the work for you. We would be happy to assist you.

Happy Performance Engineering! #Performator #LoadTesting #Performance