As I alluded to in my excuses for not posting in ages blog post the other day I am working on a new Magento hosting review for a company called MageMojo. In doing so I’m trying out the performance testing tool called Siege. Siege was mentioned several times at the Magento Developers Paradise, so I have been wanting to try it ever since.
This is a little article that summarizes the steps I took to do the testing, in case you want to test your own Magento stores.
Get a Server
Being all the way over in New Zealand pretty much means performance testing Magento from here is out of the question. I fired up an Amazon EC2 small instance for the purposes of testing for a few hours, 20c – I’ve said it once, I’ll say it again, commodity computing is the future . I won’t bore you with a tutorial on starting an EC2 server, there are enough perfectly good guides already online.
You could optionally use a server you already have, or run this from your own computer at the office or home. It would arguably be a fairer test of latency, not running the tests in a production data centre!
Definitely do not run it from the server that is running Magento, that would be a poor test; a) because there would be no latency and b) because the server would be using resources to run the tests, and serve the webstore, that’s not very fair on it!
wget http://www.joedog.org/pub/siege/siege-latest.tar.gz tar xvzf siege-latest.tar.gz cd siege-2.71/ # if you're on a fresh ubuntu, get this package first sudo apt-get install build-essential ./configure make && sudo make install
Now you can begin a siege on your store, but first, an important warning…
Too many requests on your production server could easily make it unusable for other visitors, so be careful. Start small, with low numbers of concurrent users for short durations and keep an eye on your servers memory usage. You can halt an in-progress siege with
ctrl+c at any time.
Oh and I’m sure the people who read my blog don’t need to be told this but; if you use Siege maliciously to _performance test_ other people’s websites (from multiple Amazon EC2 instances) that’s not performance testing anymore – that’s a denial of service attack, I strongly advise against it!
Run a Basic Siege Test
Now you can run your first Siege test. In this test we just hit one URL with some number of concurrent users (10) for some amount of time (1 minute):
siege -c10 -t1M http://www.google.com
Run a More Complex Test against Magento
First grab a bunch of categories and products from the sitemap.xml (you have a sitemap, right?) and put them into a urls.txt file.
#categories curl http://yourmagentostore.com/sitemap.xml | sed 's/\<url\>/\<url\>\n/g' | grep 0.5 | sed 's/.*loc>\(.*\)<\/loc.*/\1/g' > urls.txt #products curl http://yourmagentostore.com/sitemap.xml | sed 's/\<url\>/\<url\>\n/g' | grep 1.0 | sed 's/.*loc>\(.*\)<\/loc.*/\1/g' >>urls.txt
And then run the test like this for 30 seconds with 25 concurrent users, adjust to suit:
siege -i -c25 -t30s -f urls.txt
Note: I bet a 100 million dollars there is a more elegant way to grab the urls from a sitemap. Someone post one in the comments so I can buy an island.
Looking at Results
Siege results look something like this:
Transactions: 149 hits Availability: 100.00 % Elapsed time: 13.87 secs Data transferred: 1.26 MB Response time: 0.37 secs Transaction rate: 10.74 trans/sec Throughput: 0.09 MB/sec Concurrency: 4.02 Successful transactions: 149 Failed transactions: 0 Longest transaction: 0.42 Shortest transaction: 0.36
The most import numbers to look for are
Transaction rate and
Response time they will give you an indication of how quickly your server is responding and how many connections it can handle per second.
I’d be very interested to hear from anyone with tips and ideas for performance testing, likewise if you try this and have any trouble feel free to ask for help.
Edit: Updated Siege URL and version, thanks to Eric@MageMojo for letting me know.