Certain points in the load test require particular attention if informative results are to be obtained. Simulating users does not consist of merely playing back requests to the server. Even if most of the dynamic parameters are created by NeoLoad (cookies, sessionID), some do need special attention.
Define the number of Virtual Users
The number of Virtual Users must be close to the number of real users once the application is in production, with a realistic think time (Think Time) applied between pages. Avoid testing with less Virtual Users with a minimized think time. It could be assumed that the result would be the same, as the number of requests played per second is identical. However, this is not the case, for the following reasons:
The memory burden on the server will be different: Each user session uses a certain amount of memory. If the number of user sessions is underestimated, the server will be running under more favorable conditions than in real-life and the results will be distorted.
The number of sockets open simultaneously on the server will be different. An underestimation of user numbers means the maximum threshold for open server sockets cannot be tested.
The resource pools (JDBC connection pools) will not be operating under realistic conditions. An inappropriate pool size setting might not be detected during the test.
Define several types of Virtual Users
Not all users use a web application in the same way. Define a Virtual User for each user profile: simple browsing, browsing with modifications, system administrator...
Group these user types into a Population. Populations allow a user ratio to be maintained when the load varies. For example, a ratio of 90% standard users and 10% system administrators may be selected.
Use different user accounts and values
Use variables to dynamically modify key values such as user account logins or certain form parameters (such as productID in an e-business application). The main idea of this is to bypass the use of the various server caches, for the following reasons:
Playing the same requests with the same values produces an unrealistically high performance, due to the use of various caches: preloading into memory cache, connection pools, system swap...
On the other hand, completely disabling the caches (when available) will produce an unrealistically poor performance.
Test load balancers
Load balancers can use the client IP address to balance the load over several servers. If all Virtual Users are using the same IP address (default setting), the total load will be sent to a single server.
To obtain a realistic load, use NeoLoad IP spoofing function. For more information, see Configure IP Spoofing.
Simulate actual user bandwidth
A user connecting at 100 Mbps through a local network and a user connecting with a dial-up modem at 56kbps do not load the server and bandwidth in the same way.
Use the Population bandwidth setting to realistically simulate the users' connection speeds. For more information, see Wan emulation.
Tips
After a re-start, do not hesitate to warm up the server with a few calls before generating a sudden, high load which, in addition to being unrealistic, may cause the server to crash. Sending a short, light load beforehand allows certain resources, such as connection pools or thread pools, to be pre-allocated. NeoLoad allows the user to gradually spread the initial load using a start-up policy. For example, you can choose to start your 1000 Virtual Users over a two-minute period. For more information, see Population advanced parameters.
Run the test for a significant length of time in order to iron out any outliers.
Make sure the Load Generators are not overloaded; CPU and memory usage are displayed in real time throughout the test. Various settings also may be changed, such as the number of sockets a machine may simultaneously open to the server, or the memory allocated to the Load Generator. For more information, see Troubleshooting.