Faster WP Super Cache with NginX

on December 17, 2009 in Projects with 9 comments by

Burnout!A while ago my blog started to act up by randomly showing translated pages in place of the desired language. The culprit was a WordPress caching plugin (Hyper Cache) that started to misbehave with the latest upgrade. I promptly disabled it and went on a search for a replacement.

As you may have read in one of my previous blog entries, specifically “NginX and Apache, but no memcached”, I prefer to use NginX as the front-end serving static files, and Apache as a back-end dealing with the dynamic pages. So it would be ideal if NginX could serve up static WordPress files, which is exactly what I am doing now with the help of WP Super Cache.

WP Super Cache is a rather popular plugin and converts the output generated by WordPress into a static HTML file. Installation is quick and painless, and automates a few tasks for you (or lets you know what needs to be changed on your website).

To make it work with a NginX front-end and Apache back-end, a few additional changes need to be made, primarily to the NginX configuration.

WordPress / Apache Changes

Firstly, enable  WP Super Cache into the “full on” setting, meaning it generates a “cache” and a “super cache”. WP Super Cache will then ask you to change the .htaccess file, by adding a few Rewrite Rules. In this case, we do not need those entries and we can reduce it to just the following:

# BEGIN WPSuperCache
### WARNING: This is handled by NginX!
# END WPSuperCache

This will keep the plugin’s setting page from complaining about the need to configure .htaccess.

I have also enabled the “Super Cache Compression” option, however this is not mandatory. I choose to do this to relieve NginX from a few CPU cycles (as the HTML is already compressed then).

NginX Changes

Next the NginX website configuration needs to be modified. And a few of these modifications will depend on your website or needs. In my case, I  am using a separate template for the iPhone, Android and other smart phones and do not wish to use the cache for these. So I have the following entry in my NginX that will detect mobile phones:

server {

... (snip!) ...

        # Check if it's a mobile phone
        set $mobile "";
        if ($http_user_agent ~* "(2.0 MMP|240x320|400X240|AvantGo|BlackBerry|Blazer|Cellphone|Danger|DoCoMo|Elaine/3.0|EudoraWeb|Googlebot-Mobile|hiptop|IEMobile|KYOCERA/WX310K|LG/U990|MIDP-2.|MMEF20|MOT-V|NetFront|Newt|Nintendo Wii|Nitro|Nokia|Opera Mini|Palm|PlayStation Portable|portalmmm|Proxinet|ProxiNet|SHARP-TQ-GX10|SHG-i900|Small|SonyEricsson|Symbian OS|SymbianOS|TS21i-10|UP.Browser|UP.Link|webOS|Windows CE|WinWAP|YahooSeeker/M1A1-R2D2|NF-Browser|iPhone|iPod|Android|BlackBerry9530|G-TU915 Obigo|LGE VX|webOS|Nokia5800)" ) {
                set $mobile "M";
        }

... (snip!) ...

This will set the variable $mobile to “M” if  the user-agent is from a mobile web browser. If you don’t need or want this, then you can simply leave it out of your NginX configuration.

We also need to modify the common location section. There are a number of ways to approach this, but I choose to use a number of “if” statements to build a variable $wpsc_flags. Depending on the list of “if” statements NginX will output the static cached file from WP Super Cache or continue to the Apache back-end (which will handle the rest).

Unfortunately NginX does not provide support for nested “if” statements or booleans, so this is a slightly dirty hack:

       location / {
                # If it's a POST request, send it directly backend:
                if ($request_method = POST) {
                        access_log      off;
                        proxy_pass      http://apache_backend;

                        break;
                }

                # Is it a mobile user?
                set $wpsc_flags "${mobile}";

                # Is the user logged in?
                if ($http_cookie ~* "(comment_author_|wordpress_logged_in_|wp-postpass_)" ) {
                        set $wpsc_flags "${wpsc_flags}C";
                }

                # Do we have query arguments?
                if ($is_args) {
                        set $wpsc_flags "${wpsc_flags}Q";
                }

                # Does the (gzip) Super Cache exist?
                if (-f $document_root/wp-content/cache/supercache/$host/$uri/index.html.gz) {
                        # The file exists in the WP Super Cache
                        set $wpsc_flags "${wpsc_flags}F";
                }

                # If the following flags are set (in this order) we use the cached version:
                if ($wpsc_flags = "F") {
                        expires         1h;
                        rewrite ^(.*)$ /wp-content/cache/supercache/$host/$uri/index.html.gz break;
                }

                # Or else it goes to the backend:
                access_log      off;
                proxy_pass      http://apache_backend;
        }

By reading the comments it should be clear what this does, but I’ll clarify further just in case.

A HTTP POST should not be cached, and so it will be sent directly to the Apache back-end server(s) and stops doing any other checking.

In the next step, it creates a $wpsc_flags variable based on the $mobile variable we created earlier. If you do not use the $mobile variable, then you can simply replace it as following:

set $wpsc_flags "";

Then it will add flags to $wpsc_flags depending on whether the current user is logged in, if the request contains arguments (ie.: “someurl.com?this=is&an=argument”). This will prevent an editor or commenter from being presented cached pages, which can be bothersome.

Now, as I had turned on the “Super Cache Compression” in the WordPress plugin, I am looking for files that end with “.html.gz”. If you are not using the compression option, simply remove the “.gz” from the “if (-f” statement as well as the “rewrite” statement.

Also note that it will only display the cached data if, and only if, $wpcs_flags is set to “F”. If you were logged in, this variable would actually be “CF” (in this order!) and so it will continue to the portion where everything is sent to the Apache back-end.

Speed Improvements

A while ago I had posted a message on a forum that showed how a WordPress cache could improve responsiveness. The (truncated) results of a simple “ab -n 1000 -c 50 http://<website>” (ApacheBench) showed the following back then:

Requests per second:    1753.08 [#/sec] (mean)
Time per request:       28.521 [ms] (mean)
Time per request:       0.570 [ms] (mean, across all concurrent requests)

The cached responses were generated by Hyper Cache at the Apache back-end, then forwarded to NginX to be delivered to the user (“ab” in this case).

With the new setup, where NginX is responsible for the delivery of a cached response opposed to Apache / WordPress:

Requests per second:    4712.65 [#/sec] (mean)
Time per request:       10.610 [ms] (mean)
Time per request:       0.212 [ms] (mean, across all concurrent requests)

Although these are not conclusive, lab-certified benchmarks, the crude test does show a rather impressive improvement. The time for each request has been reduced by more than half. So it’s well worth the effort.

 

Photo “Ashley Force” by Ford Racing (CC-BY)

9 comments

  1. posted on Jun 18, 2014 at 11:35 AM  |  reply

    thanks!

  2. posted on Apr 20, 2010 at 4:32 AM  |  reply

    Thank you this nice post.

  3. posted on Dec 17, 2009 at 9:08 PM  |  reply

    Personally I like to use apache disk caching now (but using tmpfs).> Document Length: 21558 bytes> Requests per second: 7926.71 [#/sec] (mean)You can expect around 20,000/sec for small file sizes. I just have an include file that detects high load that any of my ‘hostees’ can use… then activates caching accordingly. Of course they can just send http headers that do caching all the time if they want to. http://light.myxcp.net/sys/cache.phpUsing core 2 duo @ 2.8Ghz with 2GB ram and mpm-itk (for shared hosting).

    Apache mem caching didn’t work with php as mpm-itk handled each request in such a way that the cache was always empty.

    • posted on Dec 17, 2009 at 10:43 PM  |  reply

      tmpfs and Apache makes for a huge improvement in performance. The thing was that I wanted to avoid Apache altogether if I didn’t need it to generate dynamic pages, because NginX can serve static faster files than Apache at all times. The drawback is that the back-end has to be on the same server or you need an NFS (GlusterFS in my case) for the website’s files (and that’s a bottleneck in itself). On the positive side and as an extra bonus, if Apache dies for some reason, the WordPress website will still work provided that the requested page is cached :)I just ran another test on this particular article, since I had upgraded NginX to 0.8.30:Server Software: nginx/0.8.30Server Hostname: http://www.myatus.co.ukServer Port: 80Document Path: /2009/12/17/faster-wp-super-cache-with-nginx/Document Length: 47608 bytesConcurrency Level: 50Time taken for tests: 0.293 secondsComplete requests: 1000Failed requests: 0Write errors: 0Total transferred: 48886893 bytesHTML transferred: 48530977 bytesRequests per second: 3407.25 [#/sec] (mean)Time per request: 14.675 [ms] (mean)Time per request: 0.293 [ms] (mean, across all concurrent requests)Transfer rate: 162665.78 [Kbytes/sec] receivedI did disable the delivery of the “.html.gz” files because I happened to run into a problem last night (and will investigate that further), so the size is a bit larger than before. Still I’m pleased with a 1.2 Gbps transfer rate and the 0.293 ms mean time per request.

      • posted on Dec 18, 2009 at 5:34 AM  |  reply

        Problem with nginx for shared hosting…. permissions problems? No way to track bandwidth on a per user basis etc etc etc.

        • posted on Dec 18, 2009 at 4:35 PM  |  reply

          Permission problems? How so? As for tracking bandwidth, simplest is to use a log analyzer (NginX can spit things out in the same format Apache uses, so that helps).

          • posted on Dec 18, 2009 at 5:06 PM  |  

            Well the nginx requests will come through as 127.0.0.1.

            Additionally. Nginx would have to run as a user that could read files from all user accounts. Hence the permission problem… using nginx would be less secure.

            Also I have systems in place that measure actual # of bytes transferred per user that won’t work right with nginx.

          • posted on Dec 19, 2009 at 10:31 AM  |  

            Ah, I see. NginX isn’t a solution for everyone, true :)

            For the 127.0.0.1 issue you can use the rpaf module for Apache. Then configure the rpaf.conf file accordingly. I’ll write up a little how-to, since someone asked how exactly to create this NginX/Apache setup.

            As for user permissions, I’m not sure how you’ve got things setup, but the only point where NginX would need access to the file directly is to retrieve a cached version. This is read-only inside a directory specifically specified for the host (specified internally, so one cannot override it and you can even add extra safeguards internally). If the file doesn’t exist or is not supposed to be read from cache, everything gets handed over to Apache and PHP, which may have additional security features regarding read/write operations.

          • posted on Dec 19, 2009 at 11:37 PM  |  

            I thought you also used nginx to serve all static files? That is where the permissions thing came from. Anyway for now apache disk caching will do me fine. I do have lighttpd running with some ‘shared content’ (famfam.. etc) that hostees can use.

            As for 127.0.0.1 … with varnish I just modified the logging things .. and for PHP just had it switch around using auto_prepend_file (not ideal I know).

            Can I suggest that you not indent each new comment.. or at least not so much.. because the comment boxes are getting thinner and thinner ..

Join the discussion

Your email address will not be published. Required fields are marked *