hits
Cherokee 0.8.1 benchmark
Today, after releasing Cherokee 0.8.1 I decided it was time to test again how it was doing about performance. In the last months we have got a great new I/O cache layer and a couple of structural changes that were supposed to have a positive impact on the general server performance, although nobody checked whether those changes were as effective as we aimed them to be.
So, once again, here I am bringing good news. We have done it. Cherokee is actually the fastest web server among a set of the most common servers nowadays: Apache, Cherokee, Lighttpd and Nginx!!
The benchmark consisted on half a million requests of a 1.7KiB static file, with 20 concurrent clients, using a 1Gbit/s local network. The results (fastest to slowest) were:
Cherokee:
Server Software: Cherokee/0.8.1 Server Hostname: 10.0.0.102 Server Port: 80 Document Path: /index.html Document Length: 1795 bytes Concurrency Level: 20 Time taken for tests: 17.819725 seconds Complete requests: 500000 Failed requests: 0 Write errors: 0 Keep-Alive requests: 500000 Total transferred: 999007442 bytes HTML transferred: 897506630 bytes Requests per second: 28058.79 [#/sec] (mean) Time per request: 0.713 [ms] (mean) Time per request: 0.036 [ms] (mean, across all concurrent requests) Transfer rate: 54747.93 [Kbytes/sec] received
Lighttpd:
Server Software: lighttpd/1.4.19 Server Hostname: 10.0.0.102 Server Port: 80 Document Path: /index.html Document Length: 1795 bytes Concurrency Level: 20 Time taken for tests: 21.248000 seconds Complete requests: 500000 Failed requests: 0 Write errors: 0 Keep-Alive requests: 470598 Total transferred: 991856958 bytes HTML transferred: 897503590 bytes Requests per second: 23531.63 [#/sec] (mean) Time per request: 0.850 [ms] (mean) Time per request: 0.042 [ms] (mean, across all concurrent requests) Transfer rate: 45585.94 [Kbytes/sec] received
NginX:
Server Software: nginx/0.5.33 Server Hostname: 10.0.0.102 Server Port: 80 Document Path: /index.html Document Length: 1795 bytes Concurrency Level: 20 Time taken for tests: 23.741872 seconds Complete requests: 500000 Failed requests: 0 Write errors: 0 Keep-Alive requests: 500000 Total transferred: 1006000217 bytes HTML transferred: 897500000 bytes Requests per second: 21059.84 [#/sec] (mean) Time per request: 0.950 [ms] (mean) Time per request: 0.047 [ms] (mean, across all concurrent requests) Transfer rate: 41379.30 [Kbytes/sec] received
Apache2.2:
Server Software: Apache/2.2.8 Server Hostname: 10.0.0.102 Server Port: 80 Document Path: /index.html Document Length: 1795 bytes Concurrency Level: 20 Time taken for tests: 35.438605 seconds Complete requests: 500000 Failed requests: 0 Write errors: 0 Keep-Alive requests: 495064 Total transferred: 1043777896 bytes HTML transferred: 897500000 bytes Requests per second: 14108.91 [#/sec] (mean) Time per request: 1.418 [ms] (mean) Time per request: 0.071 [ms] (mean, across all concurrent requests) Transfer rate: 28762.81 [Kbytes/sec] received
For the record: I did my best configuring all the servers in the very same way. In all the cases I removed unnecessary rules that could have slowed down the server (checks for htpasswd files and so on). And all the binaries came from the Debian repository, except for Cherokee 0.8.1 that hasn't been packaged yet.
Anyway, this benchmark has been just a quick test. It is not certainly representing the result that these servers would have handling real traffic though. So, in the following days I will try to do a new a more accurate benchmark with static and dynamic content, compression, redirections, etc. I'm pretty sure the results will be even better.
Comments
Hi, I read another benchmark with an older version of cherokee. How does the 0.8 perform with file more than 20 KB?? Perhaps 50 KB
No. Unless you pack into a space shuttle and shoot it into orbit.
@ak I agree it would be the best thing to make a platform for testing where everything is know. I want to implement such thing for a very long time now; so I'll try to free up some time for it.
What about memory usage?
What about memory usage?
Can you re-run the tests with -n 100, -n 200 etc?
Without a detailed descripton of each of the webservers' setup the results are pretty worthless. Also, you haven't checked out gatling, which is in my experience the webserver that scales best when delivering static files.
Very interesting, Cherokee beats lighttpd and nginx by a significant margin. As a fellow developer who's interested in performance, I'd really like to know how you managed to make Cherokee so fast. Would you consider blogging about the optimization techniques that you've used?
dagi3d: Of course it does (check out the documentation, there are even cookbooks covering those topics). Besides, a few months ago someone published a few benchmarks where Cherokee was faster and more stable than nginx serving FastCGI and SCGI content. :)
can cherokee run ruby on rails applications(maybe through fastcgi or working as proxy server for mongrel instances) ? it could probably be an alternative for nginx...
Are there any plans to have Cherokee included in the Fedora repositories?
That's ab for sure. I thought that Lighty 1.4.19 was the latest stable release. I don't follow NginX so I'm not sure about the release. I must admit I've heard nice things about it, though. Anyway, I'd love to see the benchmarks with dynamic content. Especially how they cope with extremely high loads and PHP's inherent bugs. This certainly seems to be getting interesting :)
what did you use to test? ab , httperf ? always interested to find a good test client.
What about more recent versions of nginx and lighttpd?
Hi, It would be interesting to add Yaws to your next benchmark. Richard
