Your company is rolling out its new web application. This web application is well designed and attractive. It has been well tested by your quality assurance (QA)group. You have spent a lot of money marketing your new website. The website now goes live. Very soon you realize that your new web application is unable to scale to your typical production load and the response times and performance reach completely unacceptable levels. Your web logs indicate that most of your website visitors simply leave the new website before buying anything since its response times are very poor. You start to panic, and one of your friends tells you that the reason your application does not perform adequately is that you do not have ample hardware. Hearing this, you decide to add more server hardware. While this has a small effect on the website response time, it is still woefully inadequate. You soon realize that problem does not really lie with the hardware but rather with the web application itself. It appears that your web application was not well optimized for the typical peak loads. It becomes increasingly evident that most of your marketing dollars were wasted.
Enter the world of performance and load testing. In order to ensure that your application scales as the load increases, it is necessary to perform tests where your application is tested against a simulated load that closely resembles the actual load that it will be subjected to when it is exposed to the outside world.
It is important to distinguish between functional or regression testing and load or performance testing. Functional and regression testing is used to automate a large number of scenarios to ensure that your website works as intended. Load testing on the other hand gauges how well your website performs when it is subjected to a large load, such as a large number of simultaneous users.
Load and performance testing require the use of automated testing tools. It is impractical if not ridiculous to try to simulate a load of 200 concurrent users by having a group of 200 folks sit on 200 machines and try to perform operations at the same time. Before you can begin any kind of load and performance testing you will need to identify the test scenarios you need to automate. A load testing tool will typically record web requests and responses based on user interactions with a website. As you perform various operations on your website or application, the tool records all the web transactions that take place. When you finish recording, it generates an automated script. Alternatively you could use the tool to manually create the script. Typically testers will perform a combination of the two. They will use the recording mechanism to generate the skeletal foundations of their scripts and then manually modify the scripts to take into account specialized scenarios. The load testing tool should also allow the tester to simulate constrained bandwidth situations. This means for example that it would accurately model users who would use the application on a slow modem connection. It should also allow the tester to drive the script using random data from large external data sets.
Load testing scripts can be either graphical or textual in nature. Good load and performance testing tools do not require the testers to be programmers. Testers that are not familiar with programming will work with scripts that can be graphically manipulated. Graphical scripts will typically show all recorded steps in the form of a hierarchical structure and testers would modify any of the steps in order to modify the recorded script. Testers who are proficient in programming may want to manipulate their scripts by editing the code. These testers would work with code written in a well known programming language such as JavaScript, VBScript or C.
Once your script has been created, you may wish to put specialized checks in place. These checks would typically be related specifically to the application under test. For example you may wish to check that the response returned by a certain web request meets certain textual criteria. This would involve the placement of a text checkpoint. A text checkpoint can verify that a given text segment is present or is not present in a specified portion of the web page. Once you have finished the creation of the script you would typically test the script on your own machine as a single user to ensure that the test script runs as intended.
You are now ready to perform a performance based load test. A good load testing tool will allow the tester to fine tune the execution of the test. This means that it will allow the tester to choose the number of concurrent virtual users, how the script ramps up, how it ramps down and for how long it runs. It should also allow the tester to create various groups of concurrent virtual users that have their own ramp up, steady state and ramp down characteristics. A good testing tool will allow the user to distribute this load over several machines since a single machine may be only able to scale up to a few hundred users.
To understand the behavior of the loaded web application, it is also important for the load testing tool to enable the tester to track the performance characteristics of external components such as operating systems, web servers, databases etc. This allows the user to see how the performance of his application correlates with the performance characteristics of the external component. This kind of analysis will allow the tester to pinpoint the root cause of performance bottlenecks fairly easily.
During test execution the tester should be able to view the performance graphs in real time for performance metrics such as the transaction response time, HTTP responses per second grouped by HTTP code (e.g. 200, 404, 500 etc), passed transactions per second, failed transactions per second, total transactions per second, hits per second, pages downloaded per second etc. The tester should also be able to simultaneously view the performance characteristics of the external components described above. For an operating system this could be something like the % processor time, for a database it could be the number of writes per second. At the end of the test, the tester would typically be able to view and save this data as a report for further analysis.
Load and performance testing allow you to simulate the behavior of your application under a typical production environment. This will allow you to plan your hardware deployment strategy effectively and ensure that your application will deliver the expected performance characteristics. Rolling out a web application without testing its performance characteristics under expected production loads would resemble crossing a road blindfolded. Load testing is an essential part of the development cycle of a web application and should never be overlooked.