Using the Mythic Beasts IPv4 -> IPv6 Proxy for Websites on a v6 only Pi and getting the right REMOTE_ADDR
So, more because I was intrigued than anything else, I've got a pi3 from Mythic Beasts, they're supplied with IPv6 only connectivity and the file storage is NFS over a private v4 network. The proxy will happily redirect requests to either http or https to the Pi, but this results (without turning on the Proxy Protocol) with getting remote addresses in your logs of the proxy servers, which is not entirely useful.
I've cheated a bit, because the turning on of ProxyProtocol for the hostedpi.com addresses is currently not exposed to customers (it's on the list!), to do it without access to Mythic's backends use your own domainname (I've also got https://pi3.sommitrealweird.co.uk/ mapped to this Pi).
So, first step first, we get our RPi and we make sure that we can login to it via ssh (I'm nearly always on a v6 connection anyways, so this was a simple case of sshing to the v6 address of the Pi). I then installed haproxy and apache2 on the Pi and went about configuring them, with apache2 I changed it to listen to localhost only and on ports 8080 and 4443, I hadn't at this point enabled the ssl module so, really, the change for 4443 didn't kick in. Here's my /etc/apache2/ports.conf file:
# If you just change the port or add more ports here, you will likely also # have to change the VirtualHost statement in # /etc/apache2/sites-enabled/000-default.conf Listen [::1]:8080 <IfModule ssl_module> Listen [::1]:4443 </IfModule> <IfModule mod_gnutls.c> Listen [::1]:4443 </IfModule> # vim: syntax=apache ts=4 sw=4 sts=4 sr noet
I then edited /etc/apache2/sites-available/000-default.conf to change the VirtualHost line to [::1]:8080.
So, with that in place, now we deploy haproxy infront of it, the basic /etc/haproxy/haproxy.cfg config is:
global log /dev/log local0 log /dev/log local1 notice chroot /var/lib/haproxy stats socket /run/haproxy/admin.sock mode 660 level admin stats timeout 30s user haproxy group haproxy daemon # Default SSL material locations ca-base /etc/ssl/certs crt-base /etc/ssl/private # Default ciphers to use on SSL-enabled listening sockets. # For more information, see ciphers(1SSL). This list is from: # https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/ ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS ssl-default-bind-options no-sslv3 defaults log global mode http option httplog option dontlognull timeout connect 5000 timeout client 50000 timeout server 50000 errorfile 400 /etc/haproxy/errors/400.http errorfile 403 /etc/haproxy/errors/403.http errorfile 408 /etc/haproxy/errors/408.http errorfile 500 /etc/haproxy/errors/500.http errorfile 502 /etc/haproxy/errors/502.http errorfile 503 /etc/haproxy/errors/503.http errorfile 504 /etc/haproxy/errors/504.http frontend any_http option httplog option forwardfor acl is_from_proxy src 2a00:1098:0:82:1000:3b:1:1 2a00:1098:0:80:1000:3b:1:1 tcp-request connection expect-proxy layer4 if is_from_proxy bind :::80 default_backend any_http backend any_http server apache2 ::1:8080
Obviously after that you then do:
systemctl restart apache2 systemctl restart haproxy
Now you have a proxy protocol'd setup from the proxy servers, and you can still talk directly to the Pi over ipv6, you're not yet logging the right remote ips, but we're a step closer. Next enable mod_remoteip in apache2:
a2enmod remoteip
And add a file, /etc/apache2/conf-available/remoteip-logformats.conf containing:
LogFormat "%v:%p %a %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" remoteip_vhost_combined
And edit the /etc/apache2/sites-available/000-default.conf to change the CustomLog line to use remoteip_vhost_combined rather than combined as the LogFormat and add the relevant RemoteIP settings:
RemoteIPHeader X-Forwarded-For RemoteIPTrustedProxy ::1 CustomLog ${APACHE_LOG_DIR}/access.log remoteip_vhost_combined
Now, enable the config and restart apache2:
a2enconf remoteip-logformats systemctl restart apache2
Now you'll get the right remote ip in the logs (cool, huh!), and, better still, the environment that gets pushed through to cgi scripts/php/whatever is now also correct.
So, you can now happily visit http://www.<your-pi-name>.hostedpi.com/, e.g. http://www.srwpi.hostedpi.com/.
Next up, you'll want something like dehydrated - I grabbed the packaged version from debian's jessie-backports repository - so that you can make yourself some nice shiny SSL certificates (why wouldn't you, after all!), once you've got dehydrated installed, you'll probably want to tweak it a bit, I have some magic extra files that I use, I also suggest getting the dehydrated-apache2 package, which just makes it all much easier too.
/etc/dehydrated/conf.d/mail.sh:
CONTACT_EMAIL="my@email.address"
/etc/dehydrated/conf.d/domainconfig.sh:
DOMAINS_D="/etc/dehydrated/domains.d"
/etc/dehydrated/domains.d/srwpi.hostedpi.com:
HOOK="/etc/dehydrated/hooks/srwpi"
/etc/dehydrated/hooks/srwpi:
#!/bin/sh action="$1" domain="$2" case $action in deploy_cert) privkey="$3" cert="$4" fullchain="$5" chain="$6" cat "$privkey" "$fullchain" > /etc/ssl/private/srwpi.pem chmod 640 /etc/ssl/private/srwpi.pem ;; *) ;; esac
/etc/dehydrated/hooks/srwpi has the execute bit set (chmod +x /etc/dehydrated/hooks/srwpi), and is really only there so that the certificate can be used easily in haproxy.
And finally the file /etc/dehydrated/domains.txt:
www.srwpi.hostedpi.com srwpi.hostedpi.com
Obviously, use your own pi name in there, or better yet, one of your own domain names that you've mapped to the proxies.
Run dehydrated in cron mode (it's noisy, but meh...):
dehydrated -c
That s then generated you some shiny certificates (hopefully). For now, I'll just tell you how to do it through the /etc/apache2/sites-available/default-ssl.conf file, just edit that file and change the SSLCertificateFile and SSLCertificateKeyFile to point to /var/lib/dehydrated/certs/www.srwpi.hostedpi.com/fullchain.pem and /var/llib/dehydrated/certs/ww.srwpi.hostedpi.com/privkey.pem files, do the edit for the CustomLog as you did for the other default site, and change the VirtualHost to be [::1]:443 and enable the site:
a2ensite default-ssl a2enmod ssl
And restart apache2:
systemctl restart apache2
Now time to add some bits to haproxy.cfg, usefully this is only a tiny tiny bit of extra config:
frontend any_https option httplog option forwardfor acl is_from_proxy src 2a00:1098:0:82:1000:3b:1:1 2a00:1098:0:80:1000:3b:1:1 tcp-request connection expect-proxy layer4 if is_from_proxy bind :::443 ssl crt /etc/ssl/private/srwpi.pem default_backend any_https backend any_https server apache2 ::1:4443 ssl ca-file /etc/ssl/certs/ca-certificates.crt
Restart haproxy:
systemctl restart haproxy
And we're all done! REMOTE_ADDR will appear as the correct remote address in the logs, and in the environment.