Monday, January 20, 2014

Load Test and Benchmark your web application using Siege

Developers are always worried about how their web application will perform to the real world users. Once the site is rolled out; it may come down due to sudden surge in its use, depicting DoS (Denial of Service) attack. Wouldn’t it be nice to test and benchmark your web application in development and before moving to pre-production or production servers. ‘Siege‘ can be the answer for this.
Siege is a command line utility which exposes the performance related issues of web based applications. Siege simulates the given number of users who are calling the url; within a given period of time. For example; Siege can tell – “How will be the performance of the server; if it is called by 100 users within a period of 10 second”.
Following are the features of Siege.
Siege is an open source load testing and benchmarking utility.
Siege can be used for HTTP/HTTPS and basic authentication based sites.
Siege creates real-world simulation of how the application will perform for given number of users over a given period of time.
Siege supports GET, POST HTTP methods, Cookies and transaction logging.
Siege can be used for extracting and checking the HTTP headers.
Multiple urls can be tested simultaneously with Siege.
Installing Siege
Siege can be installed on Linux (or other *nix), Solaris, Mac systems. Currently there is no installer for Windows (as its written on GNU/Linux) but Windows based server urls can be tested.
Installing on Linux
You can download latest package distribution from its developer site.
1
2
3
4
5
6
7
8
9
//Untar the downloaded package using command line
$ tar xvzf siege-latest.tar.gz
  
//Configure it
$ ./configure
  
//Now, compile and install
$ make
$ make install
Running Siege
The url can be tested using following command.
1
siege -c5 -d10 -r1 -v http://www.yoursite.com
Siege utility can have following options.
-cNUM : is to simulate the number of concurrent users (NUM count can be changed).
-dNUM : is the duration in second to run the test. (NUM is in seconds)
-rNUM : is the number of times the test can be repeated.
-v : is the verbose mode. Transaction information is printed on the screen.
-i : is the for internet. The requests are fired randomly between the number given in -d option.
-f : is used if multiple urls are tested from file
Siege can be used to check the header of the site or webpage using following command.
1
siege -g http://www.yoursite.com
Multiple urls can be tested by creating a file with the multiple urls separated by a line. The command is
1
siege -d10 -c50 -i -f /path_to_file/testfile.txt
Result
The result is printed as shown below. The url in the command is masked. You can replace it with the url to be tested.

The various parameters in the output can be analysed to check the performance of the server against the values provided. At start, keep the values of -c and -d options low and then increment them linearly, observing the ‘Failed Transactions’ count. The maximum values used till the ‘Failed Transactions’ changes to non-zero can be treated as the threshold values for the application and the server.
Do not keep the values for -c,-d or -r options high in the beginning; this may flood the server with too many requests; resulting in its failure.

Apachetop

As a webmaster, I’ve often wanted to be able to see real-time hits as they arrive. Sure, Google Analytics is a wonderful package for looking at trends over time, but there’s a delay of a few hours there, and you really can’t see data like requests per second or total bytes.
This is where the apachetop utility comes in. It’s a very simple command line utility that you can use to monitor traffic real-time. It accomplishes this by parsing the apache logfiles and displaying meaningful output to the screen.
Using Apachetop
Once you’ve installed the utility (instructions below), you can launch it by simply running apachetop from the command line. Since apachetop sometimes defaults to the wrong directory for the logfiles, you can pass in the -f parameter to specify the location of the logfile. This is also helpful when you have many virtual hosts on the same box.
apachetop -f /var/www/vhosts/howtogeek.com/statistics/logs/access_log
This is what you’ll see after a few requests have come in:

Monitoring Timeframe
The first thing to note is that the default time range for data shown is 30 seconds, so don’t expect the total counts to continue to climb forever. You can change this by passing in a few different arguments.
apachetop -H hits  (Will display stats on the last x number of hits)
apachetop -T secs  (Will display stats on the last x number of seconds)
I’ve been using a range of 5-10 minutes in my testing, and it really shows some useful feedback. There’s other options you can try out as well.

Filters
The next thing to note is that you can filter what gets shown in the view. To access the filters, use the f key, and you should see a small line pop up.
Hit the a key to add a filter and the line should switch. Now you can choose to filter by URL, referrer, or host.
I’m going to choose URL by hitting the u key. The filter dialog will show up near the bottom:
Since all of my articles are under the subdirectory /howto/, I’m going to enter that. Now apachetop will only show the hits relevant to hits to the articles, instead of every hit for every image.

Viewing Request Details
If you use the up/down keys, you’ll notice the cursor move up and down to allow you to select a request. (notice the * char)
If you hit the Right arrow key, you’ll be taken to the details page for that request. From here you can see the actual hosts hitting your site, as well as the referrers. I’m not going to show the hosts, since I don’t want to give out user’s IP address, but you can see the referrer here:
To go back to the list, just use the Left arrow key.

Switch Between Hosts, Referrers and URLs
If you use the d key, you can easily switch between the different views.
For instance, here I can see what traffic StumbleUpon is sending me, and then I can use the details view(right arrow) to see the exact articles that are getting hit from stumbleupon.

Help
At any point you can hit the ? or the h keys to take you to the help screen, which will give you a quick view of all the options.
I find the sort by very useful.

Installing on Ubuntu
sudo apt-get install apachetop

Installing from Source on CentOS
yum install readline-devel
yum install ncurses-devel
tar xvzf apachetop-0.12.6.tar.gz
cd apachetop-0.12.6
./configure
make
The binary can be found in src/apachetop, and you can copy it anywhere you’d like.

Installing from Source on Ubuntu
sudo apt-get install ncurses-dev
sudo apt-get install libreadline5-dev
tar xvzf apachetop-0.12.6.tar.gz
cd apachetop-0.12.6
./configure
make
 The binary can be found in src/apachetop, and you can copy it anywhere you’d like.