Yet another 502 Bad Gateway problem

git-n-pissed

Like so many before me, I have a fresh install of Mailcow running behind an Nginx reverse proxy and I’m unable to access Mailcow because of 502 Bad Gateway errors. My reverse proxy and my subdomains are all running in Docker containers as is Mailcow. All my subdomains are running flawlessly behind the reverse proxy except Mailcow. I’ve followed the (limited) documentation on the subject to the letter. I.e. my HTTP(S)_BIND and HTTP(S)_PORT settings are as described here. My Nginx reverse proxy settings are as described here. There is no problem if I disable the reverse proxy and let Mailcow bind directly to ports 80 and 443. I’ve even got SKIP_LETS_ENCRYPT=y and I’m successfully using SSL certs I’ve already generated which I provide to Mailcow via volumes in docker-compose.override.yml when Mailcow is bound directly to ports 80 and 443.

The only thing I can think of that may be causing this issue is the fact that I already had a user defined docker network named reverse_proxy whose address space collided with the default value of IPV4_NETWORK=172.22.1, so I changed that to IPV4_NETWORK=172.23.1. To allow communication I added the reverse_proxy network as an external network to docker-compose.override.yml as described here, and when that failed I even tried to add it directly to docker-compose.yml. I’ve tried replacing the mailcow-network network in docker-compose.yml with my user defined network reverse_proxy and even tried getting rid of the reverse_proxy network and making all my containers (including the Nginx reverse proxy) use the mailcow-network network. I’ve typed sudo docker compose down -v and sudo docker compose up -d more times than I can count. Regardless of what I do, the result is the same, 502 Bad Gateway.

Would love to get this working, so any help would be appreciated.

git-n-pissed

The problem was with my reverse proxy not resolving the mailcow service. The Docker resolver can resolve based on the service name or the container name. I had intended to use the container name, but I didn’t add the “-1” that Docker adds to the end. So a few things for completeness sake:

Make sure you have the default Docker resolver in your Nginx http block. It would look like this resolver 127.0.0.11. Your default Nginx config (typically named default.conf doesn’t show the http block, it is the root of that file by default. I.e. put resolver 127.0.0.11 as a sibling of your server blocks:
Make sure you reference the correct service or container name in your Nginx config. By default, you would want to reference the nginx-mailcow service or the mailcow-nginx-mailcow-1 container. I decided I didn’t want to reference either of those, so I set the container name I proxy pass to to mailcow in the docker-compose.yml file. Remember the subdomain you access like mail.example.com IS NOT the same address you will proxy pass to.
Make sure your containers are all on the same network. I use a user-defined network instead of the default Docker network, so I had to add that to the appropriate Mailcow services. I did this in docker-compose.override.yml.

Nginx default.conf example:

resolver 127.0.0.11

server {
    listen 80;

    # Match requests with and with www, and with and without a subdomain
    # If a request has a subdomain, it will be stored in the $subdomain variable
    server_name ~^(www\.)?(?<subdomain>.+?)?\.?example\.com$;
    server_tokens off;

    # Redirect all HTTP traffic to HTTPS
    location / {
        if ($subdomain) {
            return 301 https://$subdomain.example.com$request_uri;
        }
        return 301 https://example.com$request_uri;
    }
}

server {
    listen 443 ssl;
    http2 on;

    server_name ~^(www\.)?(?<subdomain>.+?)?\.?example\.com$;
    server_tokens off;

    ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;

    # Disable TLS 1.0, TLS 1.1, and TLS 1.2 because they are old and insecure
    ssl_protocols TLSv1.3;

    location / {
        set $proxy_pass "";

        set $mailcow_container "mailcow";
        set $mailcow_subdomain "mail";
        set $mailcow_port "8080";
        if ($subdomain = $mailcow_subdomain) {
            set $proxy_pass "http://$mailcow_container:$mailcow_port$request_uri";
        }

        if ($proxy_pass = "") {
            return 403;
        }

        # Proxy the request to the corresponding Docker container based on subdomain
        proxy_pass $proxy_pass;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Mailcow docker-compose.override.yml example:

version: '2.1'
networks:
  d_net:
    external: true
services:
  dovecot-mailcow:
    networks:
      - d_net
    volumes:
      - ./example.com/fullchain.pem:/etc/ssl/mail/cert.pem:ro
      - ./example.com/privkey.pem:/etc/ssl/mail/key.pem:ro
  nginx-mailcow:
    container_name: mailcow
    networks:
      - d_net
    volumes:
      - ./example.com/fullchain.pem:/etc/ssl/mail/cert.pem:ro
      - ./example.com/privkey.pem:/etc/ssl/mail/key.pem:ro
  postfix-mailcow:
    networks:
      - d_net
    volumes:
      - ./example.com/fullchain.pem:/etc/ssl/mail/cert.pem:ro
      - ./example.com/privkey.pem:/etc/ssl/mail/key.pem:ro