The Goal of SocialTech is to Broadcast You as a Personal Stimuli Frequency

[ NOTE: I am now using Nginx for everything, i.e. not using Varnish anymore,
and getting the same or better results. ]

I run
Varnish
here on the site, with
Nginx
as the backend. I’ve written before about my overall setup,
and how to improve site performance, so I won’t go into it here.

Here I want to cover a subtlety of putting varnish in front of nginx (or any
other web server, really) with respect to redirects. Redirects occur when
the client makes a request to the server that cannot (or should not) be
served in the manner asked, e.g. asking for /resource instead of /resource/.
The trailing slash matters, as those to URLs look the same to humans but are
not the same to the web server.

Web servers such as Apache and Nginx handle this naturally by sending
clients a redirect for the latter when you ask for the former, i.e. you ask
for /resource, you get sent back a redirect (HTTP code 301 or 301) saying
that you should be asking for /resource/.

Simple enough. This all happens transparently to the client, as the initial
request, the redirect response, and the second request happen so quickly
that most people don’t even notice the turnaround.

But things can get a bit weird when you have two web daemons fielding
requests, like when you have Varnish in front of Nginx. The problem is that
the two daemons have to be listening on different ports. Varnish and Nginx
can’t both be on the same exact IP address and port combination (TCP/IP
stacks don’t like that), so you have to perform some sort of networking
trickery to get them to work.

One common way of doing it is to have Varnish listen out front on port 80,
and to have Nginx listen in back on 8080, or 81, or whatever. Then you tell
Varnish what port Nginx is on and everything works great.

Until someone asks for /resource.

When that happens, Varnish asks Nginx for the resource and gets a redirect,
only the redirect isn’t to a usable URL — it’s to the backend URL. So
instead of being sent to site.com/resource/ like you would be if Varnish
weren’t there, you get sent to site.com:8080/resource/, which your users
shouldn’t be able to connect to (you do have a firewall on your server,
right?).

Get a weekly breakdown of what’s happening in security and tech—and why it matters.

But there’s a simple solution.

TL;DR: Have Varnish listen on 1.2.3.4:80 (your external IP), and have
Nginx listen on localhost:80. This way both daemons are convinced they’re
authoritative, so when Nginx responds with redirects the won’t be mangled
with some alternative port in the URL that external users can’t reach.

Here’s what the localhost config looks like in Nginx:

    server {
        listen  localhost:80;
        server_name  localhost;
    ...

Now all your redirects should look as if Nginx is sitting in front, and you
should be good to go. Hope this helps.

[ Note: Don’t try to handle this in Varnish itself; this is a backend issue
and should be handled by properly situating your daemons, not by rewrite
hand-waving on the acceleration server. ]

::