6 Apr
2011
94314
hits
hits
Linux Format benchmarks Cherokee
I have written about the performance of Cherokee a few times already. Cherokee is quite fast and efficient, I suppose you are aware of that by now. However, I wanted to write these few lines to let you know about an article titled 'Cherokee: Fast' that the Linux Format magazine published last month (March 2011, issue 142, pages 96-99).
The article includes an independent benchmark between Apache 2.2, Cherokee 1.0.15, Lighttpd 1.4.26 and Nginx 0.7.65. Truth be said, even though it is a very well written article, you shouldn't expect an extensive, in-depth benchmark. Still, the benchmark results are still fairly representative IMHO.
All in all, Cherokee was the fastest one, outperforming the rest of the servers. Check out one of the resulting graphs as an example:
Cherokee (green), Nginx (red), Lighttpd (yellow) and Apache (blue)
The article also mentions Cherokee's administration and monitoring graphical interface and some of the many advantages of not having to deal with plain text configuration files any longer.
By the way, I do know that publishing this reference will probably raise the same questions and comments as the rest of the post I wrote on the performance of Cherokee.. so, as a preemptive measure I will answer a couple of the most usual questions in advance:
- Yes, there are more web servers. Obviously I did not write the article so I couldn't tell you for sure. I suppose they were not included either because of issues with their license, the project's maturity status, or simply because of its lack of user base.
- Yes, of course it's possible to write a tiny program to serve static content faster than any of the benchmarked servers. Bear in mind, though, that the article tested fully functional Web Servers able to run and interact with the wide variety of the technology on a regular Web server box: PHP, Java, Python, Ruby, MySQL and LDAP servers, Audio/Video streaming, and so on and so forth.
So, having said that, I'd just like to finish this quick post by letting you know that there are a couple of compelling new features on Cherokee that I'll try to merge into our Trunk branch within the next few weeks. It's pretty exciting because both of them are new concepts that have not been implemented on a Web Server before... You know, at the end of the day, the performance of Cherokee is not something we have focused on, but one of the consequences of thinking out of the box.
Posted by Alvaro Lopez Ortega
Comments
Jones Lee on Wed Apr 6 10:41:40 2011
1627
Such a bug gap in concurrency performance, I gave Cherokee a try long time ago and was quite impressed, yet I could not settle with Cherokee because of its poor support for RoR. I might be wrong now, so please let me know how well Cherokee support RoR3, I'd definitely give another round ;)
Alvaro on Wed Apr 6 10:55:10 2011
1628
@Jones Lee: Technically speaking, there are plenty of options. You can choose between using FastCGI, SCGI, uWSGI and a Reverse HTTP Proxy to connect Cherokee with your RoR projects. However, Cherokee ships a wizard to set up RoR projects so you don't even have to worry about that (if you don't want to). Give it a try! -- BTW, if something didn't work as you expected, I'd appreciate if you dropped me a line. Cheers!
Michael Alan Dorman on Wed Apr 6 14:26:53 2011
1629
Unfortunately, a little research suggests this comparison was spectacularly poorly done---even leaving aside the question of whether testing transfer of a 100-byte static file could ever be representative of anything in this day and age. In short, they seemed to have chosen old versions of everything but Cherokee. In some cases, perversely old versions. Cherokee 1.0.15 was released 2010-12-29, making it the newest server tested. On 2010-12-29, the latest stable version of Apache was 2.2.17, released 2010-10-19, more than two months before the version of Cherokee they used. 2.2.14 was released 2009-10-05, more than 14 months before the version of Cherokee they used.. On 2010-12-29, the latest stable version of Lighttpd was 1.4.28, released on 2010-10-22, more than two months before the version of Cherokee they used. 1.4.26 was released 2010-02-07, more than 10 months before the version of Cherokee they used. On 2010-12-29, the latest stable version of nginx was 0.8.54, released 2010-12-14, (one day) more than two weeks before the version of Cherokee they used. 0.7.65 was released 2010-02-01, more than 10 months before the version of Cherokee they used. Arguably, nginx got screwed worst, since we're talking about a major version difference---the 0.7 line having been in more-or-less maintenance mode since 2009-06---but still, this comparison seems to privelege Cherokee against *all* the other servers. Alvaro, I understand that it is nice to see your software perform well against its competition, but I would encourage you to dissociate yourself from this comparison, or at least take it upon yourself to point out that there were some things that may have left your competitors at a disadvantage.
Alvaro on Wed Apr 6 14:43:13 2011
1630
@Michael: Agreed. You did read my comments about it not being an "extensive, in-depth benchmark" and the results being just "fairly representative" but not scientifically proven, right?. I tried to give it the importance that it should be given to an informal benchmark at the end of an article of a divulgation magazine, and I believe that's what I did.
Jones Lee on Wed Apr 6 14:45:13 2011
1631
@Alvaro: thanks. I do not have much success with RoR3 and FCGI manually or with the wizard. The official guide doc is quite obsolete and I could not find much help around. Can you please point me to a right place to seek answer?
Michael Alan Dorman on Wed Apr 6 15:30:11 2011
1632
@Alvaro: Yes, I read your post. I understood your comment about it not being an "extensive, in-depth benchmark" to be in reference to it being about serving a 100 byte static file, and that seems like a perfectly reasonable caveat on your part. I probably shouldn't have made reference to that in my response, since it wasn't really pertinent to my main issue with the article. I apologize for the lack of clarity. That aside, I do not feel that a "well written article" would compare a new version of one piece of software against several older pieces of software, or, having done so, whether the article in question can be portrayed as "fairly representative" of the pieces of software in question. Since you reiterate your statements in your response to me, I must assume that you do. I am disappointed.
Alvaro on Wed Apr 6 15:50:37 2011
1633
@Michael: I did not write the article, and thus I did not pick the versions of the software used on the benchmark. Do not blame me for that. Would it have been better to use the very latest version of each one of them? Of course, no doubt about it! Would the results have changed significantly? Quite frankly, I do not believe so.
Davidlohr Bueso on Wed Apr 6 17:39:57 2011
1634
It's unfortunate that the monkey HTTP server is not included in these comparisons, although we have a different target (embedded systems), it is nice to see how it compares to standard severs nowadays.
Luzem on Wed Apr 6 19:10:24 2011
1635
Why Cherokee number of request is no lineal like the others?, I don't understand why why the number of request/seconds variations the others appears to have a lineal progress
Alvaro on Wed Apr 6 23:43:00 2011
1636
@Luzem: The architecture and implementation are different, and thus the behavior difference.
Andreas on Thu Apr 7 01:14:40 2011
1637
I'd like to know which part of Cherokee's architecture and design makes it so fast (in this test). Maybe some rules for writing IO-heavy programs can be derived from that information. I assume that the design of the other webservers hasn't changed, so the relative performance (in this test) is still similar.
packet7 on Thu Apr 7 09:23:27 2011
1638
try to make benchmark with the latest version of nginx. The version you have tested is 1 year old.
Jones Lee on Fri Apr 8 10:34:57 2011
1639
@alo: I got the vhost up and running with using HTTP Reverse Proxy way, however I think cherokee has issue with spawning the 3 default ror interpreter sources. The 'rails' did not spawn and I have to end up running the spawing command manually in my terminal. You might have any idea?
Alvaro on Fri Apr 8 16:10:58 2011
1640
@Jones: Could you please log a bug report with your configuration and an example I could use to reproduce the problem. I'm not aware of anything that could cause that issue, but it's worth checking anyway.
Jones Lee on Sun Apr 10 14:38:09 2011
1643
@alo: Issue 1177 lodged. Cheers
jell on Tue Apr 12 17:51:14 2011
1646
apache2 with which mpm? event? worker? prefork?
Nelson on Sun Apr 17 23:34:58 2011
1654
I still am to see an independent and in-depth test using the latest version of the major webservers.
Tallin on Sat Apr 30 11:12:56 2011
1669
Glad I've fnilaly found something I agree with!
