Define objectives

Types of objectives

A load test can be used to test an application robustness and performance, as well as its hardware and bandwidth capacities. The requirements should be clearly defined in order to set precise objectives for the test. These objectives may be of different types:

• Check an application stability. The number of simultaneous users of the application must be known already. The test should answer the following questions:
  • Is the server capable of handling a certain number of simultaneous users?
  • Is the average response time for pages acceptable under this set load?
  • Does the server revert to normal behavior after a load peak?

  This type of test is called a load test. The simulated load remains constant over a long period.

  The meaning of acceptable response time must be defined beforehand as the notion can differ depending on the application tested and the type of page. A simple page should display faster than a page full of search results, which takes time and for which the user is prepared to wait.

  The CPU and memory usage should be closely monitored throughout the test. The server is considered overloaded if these usage figures regularly exceed 90%.

  It is equally important to test the server capacity to recover after a load peak. Many applications are subjected to load peaks, for example an intranet at 8.30 a.m., resulting in a temporary stress on the application. It is necessary, therefore, to check that no damage is done to the system as a result of these peaks. This type of test can also pinpoint memory leakage or resource management problems.

• Establish the number of simultaneous users an application can handle. This test can be used to validate a hardware configuration or targets set for application visits. The test should answer the following questions:
  • How many users can the application handle while maintaining an acceptable response time?
  • What is the load threshold above which the server begins to generate errors and/or refuse connections?
  • Does the server remain functional under high load or does it crash?

  This type of test is called a stress test. The load is progressively increased to discover the different breaking points.

  Use a scenario with a ramp-up load, starting from normal and up to the maximum predicted limit, and Monitor the response times and error rates. A sudden change indicates that a threshold has been passed.

• Validate performance variations after an application or infrastructure update. The test should answer the following questions:
  • Have the implemented upgrades resulted in real performance gains?
  • Which pages have seen a loss in performance?

With NeoLoad, a performance comparison may be made between two sets of test results. For more information, see Compare test results.

NeoLoad also allows you to filter test results using different criteria. For more information, see Filter test results.

Define criteria for success or failure

Defining the success or failure criteria is a prerequisite to any test. Before testing an application, set the acceptable levels for robustness and performance.

In most cases, these criteria are defined in terms of response time, error rate or number of requests per second:

• Define an average response time per page (may be different from one page to another).
• Define a maximum response time per page (may be different from one page to another).
• Define a maximum error rate.

With NeoLoad, some of these criteria may be checked using request validations. For more information, see Validation.