Varnish HTTP accelerator

varnish http accelerator

Varnish, as most of Linux administrators are already known to, is an HTTP accelerator and it does it’s job pretty well. So well, that most of the high traffic websites uses it for easing up load on the web server. Basically what it does is, it sits in front of a web server looking out for requests from end users, when a request is received, it looks up for any cached version of same page and if found, it serves it from there without even consulting the web server at the backend.

Port 80 basically is the port number reserved for the web server, but as we are using HTTP accelerator, Varnish sits on port 80 intercepting all the incoming traffic. It caches the pages and serves them from itself reducing load on the web server, which also increases the speed to an unbelievable level. Settings Varnish is quite easy (we will look into the configuration shortly) which requires novice to medium level of expertise. The more you want to explore Varnish, the more it gets complicated. That is the beauty of Varnish, even new server administrators can set it up in few minutes and the way it is built up, basic setup should suffice for most needs. Let’s dive into the basic setup, I am going to illustrate it on a Linux box (CentOS) but you can us any other flavor as per your choice:

1. Install Varnish 4.0

# rpm --nosignature -i https://repo.varnish-cache.org/redhat/varnish-4.0.el6.rpm
# yum install varnish

2. Make changes in configuration file to make Varnish run on port number 80 along with some other changes:

# vim /etc/sysconfig/varnish
VARNISH_VCL_CONF=/etc/varnish/default.vcl

VARNISH_LISTEN_PORT=80
VARNISH_ADMIN_LISTEN_ADDRESS=127.0.0.1
VARNISH_ADMIN_LISTEN_PORT=6082
VARNISH_MIN_THREADS=50
VARNISH_MAX_THREADS=1000
VARNISH_THREAD_TIMEOUT=120
VARNISH_STORAGE_SIZE=512M
VARNISH_STORAGE="malloc,${VARNISH_STORAGE_SIZE}"
VARNISH_TTL=120

DAEMON_OPTS="-a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} 
-f ${VARNISH_VCL_CONF} 
-T ${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} 
-t ${VARNISH_TTL} 
-p thread_pool_min=${VARNISH_MIN_THREADS} 
-p thread_pool_max=${VARNISH_MAX_THREADS} 
-p thread_pool_timeout=${VARNISH_THREAD_TIMEOUT} 
-u varnish -g varnish 
# -S ${VARNISH_SECRET_FILE} 
-s ${VARNISH_STORAGE}"

Explanations:

1. VARNISH_VCL_CONF= Defines the location of varnish configuration file which will be loaded at starting the service.

2. VARNISH_LISTEN_PORT= Varnish will start listening on this port after a service restart. Make sure you do not have any web service running on that port because two services trying to run on the same port will clash. For now, make your web service to run on port 8080 instead.

3. VARNISH_STORAGE_SIZE= Define the size of storage that will be used by Varnish to store the cached content.

4. VARNISH_STORAGE= This option defines what type of storage you want to use, here I have selected to use RAM instead of a file to store the cached contents because reading from RAM is pretty fast as compared to a file on hard disk. For more information, have a look here.

5. DAEMON_OPTS= Here all the above will be used, make sure there is only one instance of “Daemon Opts” in the varnish file.

VCL file:

Varnish’s default configuration file is well documented (if you want to perform some advanced tasks using varnish) but at this basic level, we should simply tell varnish where to look for backend web server. This is where VCL’s come in handy. Open up the configuration file and uncomment/make changes as mentioned below:


# vim /etc/varnish/default.vcl

backend default {
.host = "127.0.0.1";
.port = "8080";
}

That is it! now restart the Varnish service (/etc/init.d/varnish start), Once done, you have a basic setup of varnish that is listening connections on port 80 and forwarding non cached content to backend web server which is listening on 8080.

Varnish Examples:

1. You have a specific file that you do not want to cache and that request should goto the web server everytime a user requests it. Default ‘VCL’ file makes it pretty easy if you know regex. I am going to bypass a file called “xmlrpc.php” such that every time there is request for this file, it will be sent to the web server, no questions asked 😉


# vim /etc/varnish/default.vcl

sub vcl_recv {
if ( req.method == "POST" && req.url ~ "^/xmlrpc.php$" ) {
return (pass);
}

2. You want to restrict access to some login page on your website only to a single IP address. Well, let’s see how to achieve this. Here I will be blocking access to my blog’s admin page such that it can only be accessed from a single IP address that I will define in /etc/varnish/default.vcl:

</pre>
vim /etc/varnish/default.vcl

acl admin_whitelist {
"localhost";
"127.0.0.1";
"X.X.X.X";
"XX.XX.XX.XX";
}

sub vcl_recv {
if ((( req.url ~ "^/wp-admin/$" ) || ( req.url ~ "^/login$" ) || ( req.url ~ "^/wp-login.php$" )) && !( client.ip ~ admin_whitelist )) {
return (synth(403," "));
}

Above code in default.vcl will make sure that when ever a request is made to admin page, it will return a 403 Forbidden error message if the IP address is not mentioned in acl admin_whitelist { block. “X.X.X.X” and “XX.XX.XX.XX” are the external IP addresses from where you want to allow access. Now if you impletement it, you will see the error messge but you will also see another contents such as Guru Meditation, Varnish server, XID etc. which makes the page kind of look ugly. We want a simple, clean page displaying the error only and nothing else.

Well, default.vcl is here to rescue again. Let’s edit the same file again:


# vim /etc/varnish/default.vcl

sub vcl_synth {
set resp.http.Content-Type = "text/html; charset=utf-8";
if (resp.status == 403) {
synthetic( {"
<html>
<head><title>FORBIDDEN!</title></head>
<body>
<h2>Access denied!</h2>
</body>
</html>
"} );
}
else {
synthetic( {"
<html>
<head><title>ERROR!</title></head>
<body>
<h3>Something went wrong, please contact administrator at: <a href="mailto:admin@example.com">admin@example.com</a></h3>
</body>
</html>
"} );
}
return (deliver);
}

Restart the Varnish service and bingo! you have custom error page for varnish. You can add any code there to execute when a 403 error is encountered or any other error. Just make sure to add another block, for example, you want to show a custom page for “500 Internal Server” error, varnish is the way to go.

There are much more advanced implementations regarding varnish which you can create simply by using one configuration file, default.vcl. Google is an awsome place to start and if you want to get your hands really dirty with varnish, then head to the official varnish guide. It has ample amount of samples which you can use/implement. Official varnish guide can be found here. Just make sure to choose the guide which matches your version of varnish. There are a lot of changes in the syntax for Varnish 3 and Varnish 4.

I hope this has been informative for you. Adios till next time!

Leave a Reply

Your email address will not be published. Required fields are marked *