hits
Front-Line Cache
There are many reasons by which Web infrastructure can perform poorly or even degrade its performance over the time. Among the most common you can find misconfigurations, infrastructures based on good ol' servers that dries the hardware resources, incorrect provisioning policies, and a whole lot of human errors (Amazon and VMWare's Cloud Foundry outages are perfect examples of the latest group).
Fortunately, there are quite a few ways to improve the global performance of Web infrastructures as well. Some of them relies on the change of some of the software pieces involved, while others require a design change of the Web infrastructure itself.
It's been a few months since I had an idea about how to improve the global performance of many of the Web deployments I was working with. An idea directly related with one of the most common architectures that medium and big Webs use nowadays.
These days, it's really usual to find a Proxy-cache server right in front of the Web servers of any organization. Its purpose is to store local copies of the dynamically generated responses, so a concrete request is only processed by the Web Server a single time, and thus some of the most time-expensive operations are skipped. Despite the simplicity of the architecture, it is very effective. Actually, if you are interested on Cherokee, odds are you knew about this a long, long time ago.
Due the effectiveness of the scheme, it is usual to deploy Cherokee along with either Varnish or Squid in order to improve the performance of the Web. There are pros and cons about this scheme, though. On the one hand, it decreases the system(s) load, and improves response time whenever a cache hit happens. On the other hand, it introduces some latency to the system; bear in mind that the communication between the Proxy-cache server and the Web server takes time, and that's ultimately a latency increase that the Web site user will suffer.
From an architectural point of view, the scheme wasn't optimal either. Actually, there is a very simple question I'd like you to answer: Why would you want to keep two separate servers working on the same service? Is there something you get from it? It is neither safer, nor faster.. so, why would you do it then?
The answer is clear to me. You have been doing it, simply because you had no other choice. There was no way to enjoy the advantages of the scheme without paying the price of deploying a sub-optimal architecture.
Today, I'm delighted to introduce Front-Line Cache, a brand new take on early caching of Web content. Front-Line Cache is a new mechanism in Cherokee that implements the cache functionality you'd expect from a Proxy-cache server, but within the Web Server itself.
Advantages? Many. Please, allow me to list a few:
- Resource optimization: A Front-Line Cache enabled version of Cherokee uses far less resources (CPU, memory and disk access) than the tandem of a Proxy-cache server and a Web server.
- You get all the benefits of using a Proxy-cache, but you don't pay the price of an increase of the latency. Since there is no communication between the two software pieces, there is no additional delay.
- As with the rest of the pieces of the Cherokee stack, there is no need to deal with text based, error prone, configuration files. You will enjoy a nice configuration interface where you used to have to deal with two different configuration files, with two different, non-standard grammars.
- You will enjoy a unified log of the transactions of your Web infrastructure where you used to have two different files, one from each of the two different servers.
- Enjoy the same set of tools and goodies of Cherokee, including: Live monitoring of traffic, Remote administration and tweaking, Live server status reports, etc.
Please, check the following captures as an example of the performance improvements that can be achieved by using Cherokee's Front-Line Cache. Both images were captures running MediaWiki under Cherokee trunk (the upcoming Cherokee 1.2.3 version) and php-fpm 5.3. Despite the lower load of the server, the response time was also significantly lower: 204ms using Front-Line Cache, against 1280ms without it -- or, which is the same, the response was delivery took barely 16% of the time that it'd have required without Front-Line Cache.
Having said all this, I'd also like to clarify that the Front-Line Cache mechanism is still an experimental feature that will be shipped with Cherokee 1.2.3 by the first time. So, even though it's been stable for all our testing cycles, I'd recommend you to use it with caution until its 'Experimental' status is removed.
Comments
Where does the Front-Line Cache store its contents? Is it a file- or memory-based cache? How about using memcached (I know, it's another server process...) as storage system? I guess, Nginx has the ability to do this. I think Cherokee is a great product, but how does the Front-Line Cache scale? If one server isn't enough, would you add more web-servers with Front-Line Cache? Kind regards, Ralf
Do you have some numbers comparing front-line cache against the setup mentioned (proxy-cache server and webserver in different machines)? Thanks.
@Ralf Geschke FL cache is internal part of cherokee, if you want to scale it, just scale the cherokee ;-) It's not separated thing. It's a part of cherokee ;-)
Hopefully, for the first public stable release, you'll publish some benchmarks. As per now, this is really interesting and i can't wait to hear more about it. Keep it up!
I guess the biggest consideration I'd have is whether this feature will be sufficiently customisable like other caching proxies. For instance, options for setting what (regex, filetypes, etc) and against what conditions to cache (cookies, auth, etc) are crucial as certain responses from application servers cannot be cached (and shouldn't be -- such as authenticated responses for instance). Looking forward to seeing a comparison between this and something like Varnish in front. Having all the config in one place would certainly be a god-send.
Looking forward to the new cache release! This was a RFE I submitted :) @David, there is some functionality to enhance the caching with Cherokee's rules. Nothing specific within the caching configuration, but rather the default rule types already in Cherokee can be made to work in conjunction with the new cache. From what I've seen in betas it is good enough to cover most folks needs. Certainly will be expanded in the future as people use it and define good needs.
Really good news!!! From now on I will be able to convince my project manager to use the one and only one fastest solution for serving web content.Adding the cherokee market and easy management to the equation and we will be proud cherokee users within a month.
@Pigmej: Yes, FL cache is part of Cherokee, but somewhere it has to store its contents. ;-) Just an example - we're using a multi-level cache system, based on memcached and file storage. Memcached is configured with 2 x 4 GB and in the file system are currently 58 GB cached data (per webserver). I'm in doubt because of the concurrency of resources - FL Cache or webserver. How do they behave in a critical manner? So I'm really curious to test the release of FL Cache. :-) Kind regards, Ralf
Geez, that's unbelviebale. Kudos and such.
