Setting up GitLab as a docker container behind a secure reverse proxy on CentOS 7.x

Last weekend I decided to install GitLab for Mallorn Computing. I’d been meaning to do it for ages, but I never really got around to it. Surprisingly, I couldn’t find much about the process online, so I thought I’d document and share it.

I dutifully downloaded the installation package to a fairly empty server, ran it, and immediately freaked out. My server wasn’t just going to run GitLab; it was going to have Apache on it, other Redis databases, MariaDB, etc. GitLab didn’t seem to play nice with other applications and took over all kinds of ports. The install eventually failed when it realized it couldn’t bind to port 80 since Apache was already installed, and I was left with a mess (hint: the command to clean things up is gitlab-ctl cleanse).

I really didn’t want to set up a new dedicated server, so I hoped there was a Docker container I could install instead. Thankfully, a quick Google search showed there was.

The first thing I did was to install the Docker container:

docker run --detach --hostname git.mallorn.com --publish 4443:4443 --publish 2211:2211 --name gitlab --restart always gitlab/gitlab-ee:latest

Then I edited the GitLab config file and tried a bunch of settings for a few hours. Ultimately, I figured out which ones worked. Running docker exec -it gitlab vi /etc/gitlab/gitlab.rb opened a vi session editing the configuration file for GitLab. I had to change the variables below to the values you see.

   
   external_url 'https://git.mallorn.com:4443'
   nginx['ssl_client_certificate'] = "/etc/gitlab/trusted-certs/cert.pem"
   nginx['ssl_certificate'] = "/etc/gitlab/trusted-certs/fullchain.pem"
   nginx['ssl_certificate_key'] = "/etc/gitlab/trusted-certs/privkey.pem"
   gitlab_rails['trusted_proxies'] = ["172.16.1.10"]
   nginx['proxy_set_headers'] = {
     "X-Forwarded-Proto" => "https",
     "X-Forwarded-Ssl" => "on"
   }

Note that trusted_proxies is my internal network; set it to the IP of the interface on your proxy server that will talk to your GitLab installation.

Once that was done, I had to edit the ssh configuration file to tell it to listen on port 2211 (I now realize I could have just changed my startup parameters when I started the container, but too late now): I had to run docker exec -it gitlab vi /assets/sshd_config and change the port to 2211, then save the file. The last step was to restart the sshd process with docker exec -it gitlab /etc/init.d/sshd restart.

At this point I needed to import my SSL certificates because I had told GitLab to use certs that didn’t exist yet. I used letsencrypt to generate a certificate, then copied it into my Docker container:


   cd /etc/letsencrypt/live/git.mallorn.com
   docker cp -a -L cert.pem gitlab:/etc/gitlab/trusted-certs
   docker cp -a -L privkey.pem gitlab:/etc/gitlab/trusted-certs
   docker cp -a -L fullchain.pem gitlab:/etc/gitlab/trusted-certs

Your directory will be different; change into the one for your server. After this is done, restart your container by running sudo docker restart gitlab.

The last step was to create /etc/httpd/conf.d/git.mallorn.com.conf:


   <VirtualHost *:80 *:443>

      ServerName git.mallorn.com
      RewriteCond %{HTTPS} !=on
      RewriteRule ^/?(.*) https://%{SERVER_NAME}/$1 [R,L]
           
      SSLEngine on
      SSLHonorCipherOrder on
      SSLProtocol all -SSLv2 -SSLv3 -TLSv1 -TLSv1.1
      SSLCipherSuite "EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA!RC4:EECDH:!RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS"
      SSLCompression off
      SSLSessionTickets off
      SSLCertificateFile /etc/letsencrypt/live/git.mallorn.com/cert.pem
      SSLCertificateKeyFile /etc/letsencrypt/live/git.mallorn.com/privkey.pem
      SSLCertificateChainFile /etc/letsencrypt/live/git.mallorn.com/chain.pem

      <Proxy *>
         Require all granted
      </Proxy>
   
      ProxyPreserveHost on
           
      SSLProxyEngine on
      SSLProxyVerify none
      SSLProxyCheckPeerCN off
      SSLProxyCheckPeerName off
      SSLProxyCheckPeerExpire off
      ProxyRequests Off
      ProxyPass / https://git.mallorn.com:4443/ nocanon
      ProxyPassReverse / https://git.mallorn.com:4443/ nocanon
      AllowEncodedSlashes NoDecode

      Header edit Location ^http://git.mallorn.com/ https://git.mallorn.com/
      RequestHeader unset Accept-Encoding
      RequestHeader set Host "git.mallorn.com"
      RequestHeader add X-Forwarded-Ssl on
      RequestHeader set X-Forwarded-Proto "https"
           
      LogFormat "%h %l %u %t \"%r\" %s %b \"%{Referer}i\" \"%{User-agent}i\""
      ErrorLog /var/log/httpd/git.mallorn.com/error
      TransferLog /var/log/httpd/git.mallorn.com/access
   </VirtualHost>

I ran apachectl graceful to reload Apache, and from that point forward git.mallorn.com answered HTTP requests.

Next I had to make ssh work. Create an ssh key:

ssh-keygen -t rsa -b 4096 -f ~/.ssh/gitlab.id_rsa

Then set up ~/.ssh/config to reflect that key:


   host git.mallorn.com
    HostName git.mallorn.com
    Port 2211
    IdentityFile ~/.ssh/mallorn-git.id_rsa
    User [your login in GitLab]

Paste the contents of your ~/.ssh/gitlab.id_rsa.pub file into your key settings in GitLab and you will now be able to check files in and out via ssh.

Finally, set up a cron job to copy your SSL certs into the container regularly. This cron entry (saved as /etc/cron.d/gitlab) copies the certificates in and restarts the container if they have changed.

   5 30 * * * root (cd /etc/letsencrypt/live/git.mallorn.com ; find . -mmin 2 -exec "docker cp -a -L cert.pem gitlab:/etc/gitlab/trusted-certs; docker cp -a -L privkey.pem gitlab:/etc/gitlab/trusted-certs; docker cp -a -L fullchain.pem gitlab:/etc/gitlab/trusted-certs; docker restart gitlab)

Note that you could make the certs available directly to the container instead by starting it with the -v option, and that may ultimately be the route that I take.

Lastly, don’t forget to setup backups!