mailcow behind jwilder reverse proxy loosing IPv4 access to server

DDocSnyd3r · Sep 9, 2023

Hi all,
I testet an implementation with jwilder reverse proxy. My server is ipv4/6 dual stack.
After starting mailcow some minutes in I can no longer reach the server via ip4, web ports not reachable and also SSH access no longer possible. Access via ipv6 is still possible. Once i stop the mailcow stack ip4 cennectivity returns.
This is a centos7 server. When I upgrade to Alma linux 9 how to handle iptables vs. firewalld vs. nftables. My centos7 has iptables running.

Here comes my huge compose file, the nginx-proxy network is my reverse proxy which will generate the certificates under /srv/nginx/certs/mail.xxx.eu/ I mount to the mailcow containers, maybe this can also be done in a better way…

`version: ‘2.1’
services:

unbound-mailcow:
  image: mailcow/unbound:1.17
  environment:
    - TZ=${TZ}
  volumes:
    - ./data/hooks/unbound:/hooks:Z
    - ./data/conf/unbound/unbound.conf:/etc/unbound/unbound.conf:ro,Z
  restart: always
  tty: true
  networks:
    mailcow-network:
      ipv4_address: ${IPV4_NETWORK:-172.22.1}.254
      aliases:
        - unbound

mysql-mailcow:
  image: mariadb:10.5
  depends_on:
    - unbound-mailcow
  stop_grace_period: 45s
  volumes:
    - mysql-vol-1:/var/lib/mysql/
    - mysql-socket-vol-1:/var/run/mysqld/
    - ./data/conf/mysql/:/etc/mysql/conf.d/:ro,Z
  environment:
    - TZ=${TZ}
    - MYSQL_ROOT_PASSWORD=${DBROOT}
    - MYSQL_DATABASE=${DBNAME}
    - MYSQL_USER=${DBUSER}
    - MYSQL_PASSWORD=${DBPASS}
    - MYSQL_INITDB_SKIP_TZINFO=1
  restart: always
  ports:
    - "${SQL_PORT:-127.0.0.1:13306}:3306"
  networks:
    mailcow-network:
      aliases:
        - mysql

redis-mailcow:
  image: redis:7-alpine
  volumes:
    - redis-vol-1:/data/
  restart: always
  ports:
    - "${REDIS_PORT:-127.0.0.1:7654}:6379"
  environment:
    - TZ=${TZ}
  sysctls:
    - net.core.somaxconn=4096
  networks:
    mailcow-network:
      ipv4_address: ${IPV4_NETWORK:-172.22.1}.249
      aliases:
        - redis

clamd-mailcow:
  image: mailcow/clamd:1.61
  restart: always
  depends_on:
    - unbound-mailcow
  dns:
    - ${IPV4_NETWORK:-172.22.1}.254
  environment:
    - TZ=${TZ}
    - SKIP_CLAMD=${SKIP_CLAMD:-n}
  volumes:
    - ./data/conf/clamav/:/etc/clamav/:Z
    - clamd-db-vol-1:/var/lib/clamav
  networks:
    mailcow-network:
      aliases:
        - clamd

rspamd-mailcow:
  image: mailcow/rspamd:1.92
  stop_grace_period: 30s
  depends_on:
    - dovecot-mailcow
  environment:
    - TZ=${TZ}
    - IPV4_NETWORK=${IPV4_NETWORK:-172.22.1}
    - IPV6_NETWORK=${IPV6_NETWORK:-fd4d:6169:6c63:6f77::/64}
    - REDIS_SLAVEOF_IP=${REDIS_SLAVEOF_IP:-}
    - REDIS_SLAVEOF_PORT=${REDIS_SLAVEOF_PORT:-}
  volumes:
    - ./data/hooks/rspamd:/hooks:Z
    - ./data/conf/rspamd/custom/:/etc/rspamd/custom:z
    - ./data/conf/rspamd/override.d/:/etc/rspamd/override.d:Z
    - ./data/conf/rspamd/local.d/:/etc/rspamd/local.d:Z
    - ./data/conf/rspamd/plugins.d/:/etc/rspamd/plugins.d:Z
    - ./data/conf/rspamd/lua/:/etc/rspamd/lua/:ro,Z
    - ./data/conf/rspamd/rspamd.conf.local:/etc/rspamd/rspamd.conf.local:Z
    - ./data/conf/rspamd/rspamd.conf.override:/etc/rspamd/rspamd.conf.override:Z
    - rspamd-vol-1:/var/lib/rspamd
  restart: always
  hostname: rspamd
  dns:
    - ${IPV4_NETWORK:-172.22.1}.254
  networks:
    mailcow-network:
      aliases:
        - rspamd

php-fpm-mailcow:
  image: mailcow/phpfpm:1.84
  command: "php-fpm -d date.timezone=${TZ} -d expose_php=0"
  depends_on:
    - redis-mailcow
  volumes:
    - ./data/hooks/phpfpm:/hooks:Z
    - ./data/web:/web:z
    - ./data/conf/rspamd/dynmaps:/dynmaps:ro,z
    - ./data/conf/rspamd/custom/:/rspamd_custom_maps:z
    - rspamd-vol-1:/var/lib/rspamd
    - mysql-socket-vol-1:/var/run/mysqld/
    - ./data/conf/sogo/:/etc/sogo/:z
    - ./data/conf/rspamd/meta_exporter:/meta_exporter:ro,z
    - ./data/conf/phpfpm/sogo-sso/:/etc/sogo-sso/:z
    - ./data/conf/phpfpm/php-fpm.d/pools.conf:/usr/local/etc/php-fpm.d/z-pools.conf:Z
    - ./data/conf/phpfpm/php-conf.d/opcache-recommended.ini:/usr/local/etc/php/conf.d/opcache-recommended.ini:Z
    - ./data/conf/phpfpm/php-conf.d/upload.ini:/usr/local/etc/php/conf.d/upload.ini:Z
    - ./data/conf/phpfpm/php-conf.d/other.ini:/usr/local/etc/php/conf.d/zzz-other.ini:Z
    - ./data/conf/dovecot/global_sieve_before:/global_sieve/before:z
    - ./data/conf/dovecot/global_sieve_after:/global_sieve/after:z
    - ./data/assets/templates:/tpls:z
    - ./data/conf/nginx/:/etc/nginx/conf.d/:z
  dns:
    - ${IPV4_NETWORK:-172.22.1}.254
  environment:
    - REDIS_SLAVEOF_IP=${REDIS_SLAVEOF_IP:-}
    - REDIS_SLAVEOF_PORT=${REDIS_SLAVEOF_PORT:-}
    - LOG_LINES=${LOG_LINES:-9999}
    - TZ=${TZ}
    - DBNAME=${DBNAME}
    - DBUSER=${DBUSER}
    - DBPASS=${DBPASS}
    - MAILCOW_HOSTNAME=${MAILCOW_HOSTNAME}
    - MAILCOW_PASS_SCHEME=${MAILCOW_PASS_SCHEME:-BLF-CRYPT}
    - IMAP_PORT=${IMAP_PORT:-143}
    - IMAPS_PORT=${IMAPS_PORT:-993}
    - POP_PORT=${POP_PORT:-110}
    - POPS_PORT=${POPS_PORT:-995}
    - SIEVE_PORT=${SIEVE_PORT:-4190}
    - IPV4_NETWORK=${IPV4_NETWORK:-172.22.1}
    - IPV6_NETWORK=${IPV6_NETWORK:-fd4d:6169:6c63:6f77::/64}
    - SUBMISSION_PORT=${SUBMISSION_PORT:-587}
    - SMTPS_PORT=${SMTPS_PORT:-465}
    - SMTP_PORT=${SMTP_PORT:-25}
    - API_KEY=${API_KEY:-invalid}
    - API_KEY_READ_ONLY=${API_KEY_READ_ONLY:-invalid}
    - API_ALLOW_FROM=${API_ALLOW_FROM:-invalid}
    - COMPOSE_PROJECT_NAME=${COMPOSE_PROJECT_NAME:-mailcow-dockerized}
    - SKIP_SOLR=${SKIP_SOLR:-y}
    - SKIP_CLAMD=${SKIP_CLAMD:-n}
    - SKIP_SOGO=${SKIP_SOGO:-n}
    - ALLOW_ADMIN_EMAIL_LOGIN=${ALLOW_ADMIN_EMAIL_LOGIN:-n}
    - MASTER=${MASTER:-y}
    - DEV_MODE=${DEV_MODE:-n}
    - DEMO_MODE=${DEMO_MODE:-n}
    - WEBAUTHN_ONLY_TRUSTED_VENDORS=${WEBAUTHN_ONLY_TRUSTED_VENDORS:-n}
    - CLUSTERMODE=${CLUSTERMODE:-}
  restart: always
  networks:
    mailcow-network:
      aliases:
        - phpfpm

sogo-mailcow:
  image: mailcow/sogo:1.118
  environment:
    - DBNAME=${DBNAME}
    - DBUSER=${DBUSER}
    - DBPASS=${DBPASS}
    - TZ=${TZ}
    - LOG_LINES=${LOG_LINES:-9999}
    - MAILCOW_HOSTNAME=${MAILCOW_HOSTNAME}
    - MAILCOW_PASS_SCHEME=${MAILCOW_PASS_SCHEME:-BLF-CRYPT}
    - ACL_ANYONE=${ACL_ANYONE:-disallow}
    - ALLOW_ADMIN_EMAIL_LOGIN=${ALLOW_ADMIN_EMAIL_LOGIN:-n}
    - IPV4_NETWORK=${IPV4_NETWORK:-172.22.1}
    - SOGO_EXPIRE_SESSION=${SOGO_EXPIRE_SESSION:-480}
    - SKIP_SOGO=${SKIP_SOGO:-n}
    - MASTER=${MASTER:-y}
    - REDIS_SLAVEOF_IP=${REDIS_SLAVEOF_IP:-}
    - REDIS_SLAVEOF_PORT=${REDIS_SLAVEOF_PORT:-}
  dns:
    - ${IPV4_NETWORK:-172.22.1}.254
  volumes:
    - ./data/hooks/sogo:/hooks:Z
    - ./data/conf/sogo/:/etc/sogo/:z
    - ./data/web/inc/init_db.inc.php:/init_db.inc.php:z
    - ./data/conf/sogo/custom-favicon.ico:/usr/lib/GNUstep/SOGo/WebServerResources/img/sogo.ico:z
    - ./data/conf/sogo/custom-theme.js:/usr/lib/GNUstep/SOGo/WebServerResources/js/theme.js:z
    - ./data/conf/sogo/custom-sogo.js:/usr/lib/GNUstep/SOGo/WebServerResources/js/custom-sogo.js:z
    - mysql-socket-vol-1:/var/run/mysqld/
    - sogo-web-vol-1:/sogo_web
    - sogo-userdata-backup-vol-1:/sogo_backup
  labels:
    ofelia.enabled: "true"
    ofelia.job-exec.sogo_sessions.schedule: "@every 1m"
    ofelia.job-exec.sogo_sessions.command: "/bin/bash -c \"[[ $${MASTER} == y ]] && /usr/local/bin/gosu sogo /usr/sbin/sogo-tool expire-sessions $${SOGO_EXPIRE_SESSION} || exit 0\""
    ofelia.job-exec.sogo_ealarms.schedule: "@every 1m"
    ofelia.job-exec.sogo_ealarms.command: "/bin/bash -c \"[[ $${MASTER} == y ]] && /usr/local/bin/gosu sogo /usr/sbin/sogo-ealarms-notify -p /etc/sogo/sieve.creds || exit 0\""
    ofelia.job-exec.sogo_eautoreply.schedule: "@every 5m"
    ofelia.job-exec.sogo_eautoreply.command: "/bin/bash -c \"[[ $${MASTER} == y ]] && /usr/local/bin/gosu sogo /usr/sbin/sogo-tool update-autoreply -p

/etc/sogo/sieve.creds exit 0\""
ofelia.job-exec.sogo_backup.schedule: “@every 24h”
ofelia.job-exec.sogo_backup.command: “/bin/bash -c \”[[ $${MASTER} == y ]] && /usr/local/bin/gosu sogo /usr/sbin/sogo-tool backup /sogo_backup ALL
exit 0\""
restart: always
networks:
mailcow-network:
ipv4_address: ${IPV4_NETWORK:-172.22.1}.248
aliases:
- sogo

dovecot-mailcow:
  image: mailcow/dovecot:1.24
  depends_on:
    - mysql-mailcow
  dns:
    - ${IPV4_NETWORK:-172.22.1}.254
  cap_add:
    - NET_BIND_SERVICE
  volumes:
    - ./data/hooks/dovecot:/hooks:Z
    - ./data/conf/dovecot:/etc/dovecot:z
    - ./data/assets/ssl:/etc/ssl/mail/:ro,z
    - ./data/conf/sogo/:/etc/sogo/:z
    - ./data/conf/phpfpm/sogo-sso/:/etc/phpfpm/:z
    - vmail-vol-1:/var/vmail
    - vmail-index-vol-1:/var/vmail_index
    - crypt-vol-1:/mail_crypt/
    - ./data/conf/rspamd/custom/:/etc/rspamd/custom:z
    - ./data/assets/templates:/templates:z
    - rspamd-vol-1:/var/lib/rspamd
    - mysql-socket-vol-1:/var/run/mysqld/
    - /srv/nginx/certs/mail.xxx.eu/fullchain.pem:/etc/ssl/mail/cert.pem:ro
    - /srv/nginx/certs/mail.xxx.eu/key.pem:/etc/ssl/mail/key.pem:ro
  environment:
    - DOVECOT_MASTER_USER=${DOVECOT_MASTER_USER:-}
    - DOVECOT_MASTER_PASS=${DOVECOT_MASTER_PASS:-}
    - LOG_LINES=${LOG_LINES:-9999}
    - DBNAME=${DBNAME}
    - DBUSER=${DBUSER}
    - DBPASS=${DBPASS}
    - TZ=${TZ}
    - MAILCOW_HOSTNAME=${MAILCOW_HOSTNAME}
    - MAILCOW_PASS_SCHEME=${MAILCOW_PASS_SCHEME:-BLF-CRYPT}
    - IPV4_NETWORK=${IPV4_NETWORK:-172.22.1}
    - ALLOW_ADMIN_EMAIL_LOGIN=${ALLOW_ADMIN_EMAIL_LOGIN:-n}
    - MAILDIR_GC_TIME=${MAILDIR_GC_TIME:-7200}
    - ACL_ANYONE=${ACL_ANYONE:-disallow}
    - SKIP_SOLR=${SKIP_SOLR:-y}
    - MAILDIR_SUB=${MAILDIR_SUB:-}
    - MASTER=${MASTER:-y}
    - REDIS_SLAVEOF_IP=${REDIS_SLAVEOF_IP:-}
    - REDIS_SLAVEOF_PORT=${REDIS_SLAVEOF_PORT:-}
    - COMPOSE_PROJECT_NAME=${COMPOSE_PROJECT_NAME:-mailcow-dockerized}
  ports:
    - "${DOVEADM_PORT:-127.0.0.1:19991}:12345"
    - "${IMAP_PORT:-143}:143"
    - "${IMAPS_PORT:-993}:993"
    - "${POP_PORT:-110}:110"
    - "${POPS_PORT:-995}:995"
    - "${SIEVE_PORT:-4190}:4190"
  restart: always
  tty: true
  labels:
    ofelia.enabled: "true"
    ofelia.job-exec.dovecot_imapsync_runner.schedule: "@every 1m"
    ofelia.job-exec.dovecot_imapsync_runner.no-overlap: "true"
    ofelia.job-exec.dovecot_imapsync_runner.command: "/bin/bash -c \"[[ $${MASTER} == y ]] && /usr/local/bin/gosu nobody /usr/local/bin/imapsync_runner.pl || exit 0\""
    ofelia.job-exec.dovecot_trim_logs.schedule: "@every 1m"
    ofelia.job-exec.dovecot_trim_logs.command: "/bin/bash -c \"[[ $${MASTER} == y ]] && /usr/local/bin/gosu vmail /usr/local/bin/trim_logs.sh || exit 0\""
    ofelia.job-exec.dovecot_quarantine.schedule: "@every 20m"
    ofelia.job-exec.dovecot_quarantine.command: "/bin/bash -c \"[[ $${MASTER} == y ]] && /usr/local/bin/gosu vmail /usr/local/bin/quarantine_notify.py

exit 0\""
ofelia.job-exec.dovecot_clean_q_aged.schedule: “@every 24h”
ofelia.job-exec.dovecot_clean_q_aged.command: “/bin/bash -c \”[[ $${MASTER} == y ]] && /usr/local/bin/gosu vmail /usr/local/bin/clean_q_aged.sh
exit 0\""
ofelia.job-exec.dovecot_maildir_gc.schedule: “@every 30m”
ofelia.job-exec.dovecot_maildir_gc.command: "/bin/bash -c \“source /source_env.sh ; /usr/local/bin/gosu vmail /usr/local/bin/maildir_gc.sh\”"
ofelia.job-exec.dovecot_sarules.schedule: “@every 24h”
ofelia.job-exec.dovecot_sarules.command: “/bin/bash -c \”/usr/local/bin/sa-rules.sh\""
ofelia.job-exec.dovecot_fts.schedule: “@every 24h”
ofelia.job-exec.dovecot_fts.command: “/usr/bin/curl http://solr:8983/solr/dovecot-fts/update?optimize=true

solr

http://solr:8983/solr/dovecot-fts/update?optimize=true

No preview could be generated for this link

”
ofelia.job-exec.dovecot_repl_health.schedule: “@every 5m”
ofelia.job-exec.dovecot_repl_health.command: “/bin/bash -c \”/usr/local/bin/gosu vmail /usr/local/bin/repl_health.sh\""
ulimits:
nproc: 65535
nofile:
soft: 20000
hard: 40000
networks:
mailcow-network:
ipv4_address: ${IPV4_NETWORK:-172.22.1}.250
aliases:
- dovecot

postfix-mailcow:
  image: mailcow/postfix:1.71
  depends_on:
    - mysql-mailcow
  volumes:
    - ./data/hooks/postfix:/hooks:Z
    - ./data/conf/postfix:/opt/postfix/conf:z
    - ./data/assets/ssl:/etc/ssl/mail/:ro,z
    - postfix-vol-1:/var/spool/postfix
    - crypt-vol-1:/var/lib/zeyple
    - rspamd-vol-1:/var/lib/rspamd
    - mysql-socket-vol-1:/var/run/mysqld/
    - /srv/nginx/certs/mail.xxx.eu/fullchain.pem:/etc/ssl/mail/cert.pem:ro
    - /srv/nginx/certs/mail.xxx.eu/key.pem:/etc/ssl/mail/key.pem:ro
  environment:
    - LOG_LINES=${LOG_LINES:-9999}
    - TZ=${TZ}
    - DBNAME=${DBNAME}
    - DBUSER=${DBUSER}
    - DBPASS=${DBPASS}
    - REDIS_SLAVEOF_IP=${REDIS_SLAVEOF_IP:-}
    - REDIS_SLAVEOF_PORT=${REDIS_SLAVEOF_PORT:-}
    - MAILCOW_HOSTNAME=${MAILCOW_HOSTNAME}
    - SPAMHAUS_DQS_KEY=${SPAMHAUS_DQS_KEY:-}
  cap_add:
    - NET_BIND_SERVICE
  ports:
    - "${SMTP_PORT:-25}:25"
    - "${SMTPS_PORT:-465}:465"
    - "${SUBMISSION_PORT:-587}:587"
  restart: always
  dns:
    - ${IPV4_NETWORK:-172.22.1}.254
  networks:
    mailcow-network:
      ipv4_address: ${IPV4_NETWORK:-172.22.1}.253
      aliases:
        - postfix

memcached-mailcow:
  image: memcached:alpine
  restart: always
  environment:
    - TZ=${TZ}
  networks:
    mailcow-network:
      aliases:
        - memcached

nginx-mailcow:
  depends_on:
    - sogo-mailcow
    - php-fpm-mailcow
    - redis-mailcow
  image: nginx:mainline-alpine
  dns:
    - ${IPV4_NETWORK:-172.22.1}.254
  command: /bin/sh -c "envsubst < /etc/nginx/conf.d/templates/listen_plain.template > /etc/nginx/conf.d/listen_plain.active &&
    envsubst < /etc/nginx/conf.d/templates/listen_ssl.template > /etc/nginx/conf.d/listen_ssl.active &&
    envsubst < /etc/nginx/conf.d/templates/sogo.template > /etc/nginx/conf.d/sogo.active &&
    . /etc/nginx/conf.d/templates/server_name.template.sh > /etc/nginx/conf.d/server_name.active &&
    . /etc/nginx/conf.d/templates/sites.template.sh > /etc/nginx/conf.d/sites.active &&
    . /etc/nginx/conf.d/templates/sogo_eas.template.sh > /etc/nginx/conf.d/sogo_eas.active &&
    nginx -qt &&
    until ping phpfpm -c1 > /dev/null; do sleep 1; done &&
    until ping sogo -c1 > /dev/null; do sleep 1; done &&
    until ping redis -c1 > /dev/null; do sleep 1; done &&
    until ping rspamd -c1 > /dev/null; do sleep 1; done &&
    exec nginx -g 'daemon off;'"
  environment:
    - VIRTUAL_HOST=${MAILCOW_HOSTNAME}
    - VIRTUAL_PORT=8080
    - VIRTUAL_PROTO=http
    - LETSENCRYPT_HOST=${MAILCOW_HOSTNAME}
    - LETSENCRYPT_EMAIL=spam@xxx.eu
    - HTTPS_PORT=${HTTPS_PORT:-443}
    - HTTP_PORT=${HTTP_PORT:-80}
    - MAILCOW_HOSTNAME=${MAILCOW_HOSTNAME}
    - IPV4_NETWORK=${IPV4_NETWORK:-172.22.1}
    - TZ=${TZ}
    - SKIP_SOGO=${SKIP_SOGO:-n}
    - ALLOW_ADMIN_EMAIL_LOGIN=${ALLOW_ADMIN_EMAIL_LOGIN:-n}
    - ADDITIONAL_SERVER_NAMES=${ADDITIONAL_SERVER_NAMES:-}
  volumes:
    - ./data/web:/web:ro,z
    - ./data/conf/rspamd/dynmaps:/dynmaps:ro,z
    - ./data/assets/ssl/:/etc/ssl/mail/:ro,z
    - ./data/conf/nginx/:/etc/nginx/conf.d/:z
    - ./data/conf/rspamd/meta_exporter:/meta_exporter:ro,z
    - sogo-web-vol-1:/usr/lib/GNUstep/SOGo/
  #ports:
  #  - "${HTTPS_BIND:-}:${HTTPS_PORT:-443}:${HTTPS_PORT:-443}"
  #  - "${HTTP_BIND:-}:${HTTP_PORT:-80}:${HTTP_PORT:-80}"
  expose:
    - "8080"
  restart: always
  networks:
    mailcow-network:
      aliases:
        - nginx
    nginx-proxy:
      aliases:
        - mail.xxx.eu

acme-mailcow:
  depends_on:
    - nginx-mailcow
  image: mailcow/acme:1.84
  dns:
    - ${IPV4_NETWORK:-172.22.1}.254
  environment:
    - LOG_LINES=${LOG_LINES:-9999}
    - ACME_CONTACT=${ACME_CONTACT:-}
    - ADDITIONAL_SAN=${ADDITIONAL_SAN}
    - MAILCOW_HOSTNAME=${MAILCOW_HOSTNAME}
    - DBNAME=${DBNAME}
    - DBUSER=${DBUSER}
    - DBPASS=${DBPASS}
    - SKIP_LETS_ENCRYPT=${SKIP_LETS_ENCRYPT:-n}
    - COMPOSE_PROJECT_NAME=${COMPOSE_PROJECT_NAME:-mailcow-dockerized}
    - DIRECTORY_URL=${DIRECTORY_URL:-}
    - ENABLE_SSL_SNI=${ENABLE_SSL_SNI:-n}
    - SKIP_IP_CHECK=${SKIP_IP_CHECK:-n}
    - SKIP_HTTP_VERIFICATION=${SKIP_HTTP_VERIFICATION:-n}
    - ONLY_MAILCOW_HOSTNAME=${ONLY_MAILCOW_HOSTNAME:-n}
    - LE_STAGING=${LE_STAGING:-n}
    - TZ=${TZ}
    - REDIS_SLAVEOF_IP=${REDIS_SLAVEOF_IP:-}
    - REDIS_SLAVEOF_PORT=${REDIS_SLAVEOF_PORT:-}
    - SNAT_TO_SOURCE=${SNAT_TO_SOURCE:-n}
    - SNAT6_TO_SOURCE=${SNAT6_TO_SOURCE:-n}
  volumes:
    - ./data/web/.well-known/acme-challenge:/var/www/acme:z
    - ./data/assets/ssl:/var/lib/acme/:z
    - ./data/assets/ssl-example:/var/lib/ssl-example/:ro,Z
    - mysql-socket-vol-1:/var/run/mysqld/
  restart: always
  networks:
    mailcow-network:
      aliases:
        - acme

netfilter-mailcow:
  image: mailcow/netfilter:1.52
  stop_grace_period: 30s
  depends_on:
    - dovecot-mailcow
    - postfix-mailcow
    - sogo-mailcow
    - php-fpm-mailcow
    - redis-mailcow
  restart: always
  privileged: true
  environment:
    - TZ=${TZ}
    - IPV4_NETWORK=${IPV4_NETWORK:-172.22.1}
    - IPV6_NETWORK=${IPV6_NETWORK:-fd4d:6169:6c63:6f77::/64}
    - SNAT_TO_SOURCE=${SNAT_TO_SOURCE:-n}
    - SNAT6_TO_SOURCE=${SNAT6_TO_SOURCE:-n}
    - REDIS_SLAVEOF_IP=${REDIS_SLAVEOF_IP:-}
    - REDIS_SLAVEOF_PORT=${REDIS_SLAVEOF_PORT:-}
  network_mode: "host"
  volumes:
    - /lib/modules:/lib/modules:ro

watchdog-mailcow:
  image: mailcow/watchdog:1.97
  dns:
    - ${IPV4_NETWORK:-172.22.1}.254
  tmpfs:
    - /tmp
  volumes:
    - rspamd-vol-1:/var/lib/rspamd
    - mysql-socket-vol-1:/var/run/mysqld/
    - postfix-vol-1:/var/spool/postfix
    - ./data/assets/ssl:/etc/ssl/mail/:ro,z
  restart: always
  environment:
    - IPV6_NETWORK=${IPV6_NETWORK:-fd4d:6169:6c63:6f77::/64}
    - LOG_LINES=${LOG_LINES:-9999}
    - TZ=${TZ}
    - DBNAME=${DBNAME}
    - DBUSER=${DBUSER}
    - DBPASS=${DBPASS}
    - DBROOT=${DBROOT}
    - USE_WATCHDOG=${USE_WATCHDOG:-n}
    - WATCHDOG_NOTIFY_EMAIL=${WATCHDOG_NOTIFY_EMAIL:-}
    - WATCHDOG_NOTIFY_BAN=${WATCHDOG_NOTIFY_BAN:-y}
    - WATCHDOG_SUBJECT=${WATCHDOG_SUBJECT:-Watchdog ALERT}
    - WATCHDOG_EXTERNAL_CHECKS=${WATCHDOG_EXTERNAL_CHECKS:-n}
    - WATCHDOG_MYSQL_REPLICATION_CHECKS=${WATCHDOG_MYSQL_REPLICATION_CHECKS:-n}
    - WATCHDOG_VERBOSE=${WATCHDOG_VERBOSE:-n}
    - MAILCOW_HOSTNAME=${MAILCOW_HOSTNAME}
    - COMPOSE_PROJECT_NAME=${COMPOSE_PROJECT_NAME:-mailcow-dockerized}
    - IPV4_NETWORK=${IPV4_NETWORK:-172.22.1}
    - IP_BY_DOCKER_API=${IP_BY_DOCKER_API:-0}
    - CHECK_UNBOUND=${CHECK_UNBOUND:-1}
    - SKIP_CLAMD=${SKIP_CLAMD:-n}
    - SKIP_LETS_ENCRYPT=${SKIP_LETS_ENCRYPT:-n}
    - SKIP_SOGO=${SKIP_SOGO:-n}
    - HTTPS_PORT=${HTTPS_PORT:-443}
    - REDIS_SLAVEOF_IP=${REDIS_SLAVEOF_IP:-}
    - REDIS_SLAVEOF_PORT=${REDIS_SLAVEOF_PORT:-}
    - EXTERNAL_CHECKS_THRESHOLD=${EXTERNAL_CHECKS_THRESHOLD:-1}
    - NGINX_THRESHOLD=${NGINX_THRESHOLD:-5}
    - UNBOUND_THRESHOLD=${UNBOUND_THRESHOLD:-5}
    - REDIS_THRESHOLD=${REDIS_THRESHOLD:-5}
    - MYSQL_THRESHOLD=${MYSQL_THRESHOLD:-5}
    - MYSQL_REPLICATION_THRESHOLD=${MYSQL_REPLICATION_THRESHOLD:-1}
    - SOGO_THRESHOLD=${SOGO_THRESHOLD:-3}
    - POSTFIX_THRESHOLD=${POSTFIX_THRESHOLD:-8}
    - CLAMD_THRESHOLD=${CLAMD_THRESHOLD:-15}
    - DOVECOT_THRESHOLD=${DOVECOT_THRESHOLD:-12}
    - DOVECOT_REPL_THRESHOLD=${DOVECOT_REPL_THRESHOLD:-20}
    - PHPFPM_THRESHOLD=${PHPFPM_THRESHOLD:-5}
    - RATELIMIT_THRESHOLD=${RATELIMIT_THRESHOLD:-1}
    - FAIL2BAN_THRESHOLD=${FAIL2BAN_THRESHOLD:-1}
    - ACME_THRESHOLD=${ACME_THRESHOLD:-1}
    - RSPAMD_THRESHOLD=${RSPAMD_THRESHOLD:-5}
    - OLEFY_THRESHOLD=${OLEFY_THRESHOLD:-5}
    - MAILQ_THRESHOLD=${MAILQ_THRESHOLD:-20}
    - MAILQ_CRIT=${MAILQ_CRIT:-30}
  networks:
    mailcow-network:
      aliases:
        - watchdog

dockerapi-mailcow:
  image: mailcow/dockerapi:2.05
  security_opt:
    - label=disable
  restart: always
  dns:
    - ${IPV4_NETWORK:-172.22.1}.254
  environment:
    - DBROOT=${DBROOT}
    - TZ=${TZ}
    - REDIS_SLAVEOF_IP=${REDIS_SLAVEOF_IP:-}
    - REDIS_SLAVEOF_PORT=${REDIS_SLAVEOF_PORT:-}
  volumes:
    - /var/run/docker.sock:/var/run/docker.sock:ro
  networks:
    mailcow-network:
      aliases:
        - dockerapi

solr-mailcow:
  image: mailcow/solr:1.8.1
  restart: always
  volumes:
    - solr-vol-1:/opt/solr/server/solr/dovecot-fts/data
  ports:
    - "${SOLR_PORT:-127.0.0.1:18983}:8983"
  environment:
    - TZ=${TZ}
    - SOLR_HEAP=${SOLR_HEAP:-1024}
    - SKIP_SOLR=${SKIP_SOLR:-y}
  networks:
    mailcow-network:
      aliases:
        - solr

olefy-mailcow:
  image: mailcow/olefy:1.11
  restart: always
  environment:
    - TZ=${TZ}
    - OLEFY_BINDADDRESS=0.0.0.0
    - OLEFY_BINDPORT=10055
    - OLEFY_TMPDIR=/tmp
    - OLEFY_PYTHON_PATH=/usr/bin/python3
    - OLEFY_OLEVBA_PATH=/usr/bin/olevba
    - OLEFY_LOGLVL=20
    - OLEFY_MINLENGTH=500
    - OLEFY_DEL_TMP=1
  networks:
    mailcow-network:
      aliases:
        - olefy

ofelia-mailcow:
  image: mcuadros/ofelia:latest
  restart: always
  command: daemon --docker
  environment:
    - TZ=${TZ}
  depends_on:
    - sogo-mailcow
    - dovecot-mailcow
  labels:
    ofelia.enabled: "true"
  security_opt:
    - label=disable
  volumes:
    - /var/run/docker.sock:/var/run/docker.sock:ro
  networks:
    mailcow-network:
      aliases:
        - ofelia

ipv6nat-mailcow:
  depends_on:
    - unbound-mailcow
    - mysql-mailcow
    - redis-mailcow
    - clamd-mailcow
    - rspamd-mailcow
    - php-fpm-mailcow
    - sogo-mailcow
    - dovecot-mailcow
    - postfix-mailcow
    - memcached-mailcow
    - nginx-mailcow
    - acme-mailcow
    - netfilter-mailcow
    - watchdog-mailcow
    - dockerapi-mailcow
    - solr-mailcow
  environment:
    - TZ=${TZ}
  image: robbertkl/ipv6nat
  security_opt:
    - label=disable
  restart: always
  privileged: true
  network_mode: "host"
  volumes:
    - /var/run/docker.sock:/var/run/docker.sock:ro
    - /lib/modules:/lib/modules:ro

networks:
mailcow-network:
driver: bridge
driver_opts:
com.docker.network.bridge.name: br-mailcow
enable_ipv6: true
ipam:
driver: default
config:
- subnet: ${IPV4_NETWORK:-172.22.1}.0/24
- subnet: ${IPV6_NETWORK:-fd4d:6169:6c63:6f77::/64}
nginx-proxy:
name: nginx-internal
external: true
volumes:
vmail-vol-1:
vmail-index-vol-1:
mysql-vol-1:
mysql-socket-vol-1:
redis-vol-1:
rspamd-vol-1:
solr-vol-1:
postfix-vol-1:
crypt-vol-1:
sogo-web-vol-1:
sogo-userdata-backup-vol-1:
clamd-db-vol-1:`

Sorry for the compose file mess, I cant seem to edit the post, here is the compose as a file.

docker-compose-mailcow-redacted.txt

24kB

esackbauer · Sep 9, 2023

Not reachable from where? From host to container? from proxy to host? from outside to proxy?

DDocSnyd3r · Sep 9, 2023

Thanks for the reply, the public ipv4 is no longer reachable from the outside, only v6.
When I issue docker compose down on cow stack via ipv6 ssh session, ping relys to ipv4 are returning. The strange thing is it takes minutes to happen after container start.

esackbauer · Sep 9, 2023

DocSnyd3r Sorry this is then not a topic of mailcow, but of your jwilder reverse proxy, which is not supported…
Or its something with your hosting. Check if you can connect to 443, 25 and so on from within your host running docker.

DDocSnyd3r · Sep 9, 2023

Also there is no mailcow network and I found the ip range on this bridge
11: br-a12b81188341: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:c5:3d:ba:1d brd ff:ff:ff:ff:ff:ff
inet 172.22.0.1/16 brd 172.22.255.255 scope global br-a12b81188341
valid_lft forever preferred_lft forever

I do not get why /16 is the default for docker, seems liek a huge waste. Initialle cowmail had another IP space which was already in use by another container.

docker network ls
NETWORK ID NAME DRIVER SCOPE
f989234d177f bridge bridge local
0646c9ac1dd8 host host local
3610dce0a4e3 nextcloud_backend bridge local
a12b81188341 nextcloud_default bridge local
f4bf5f8b511b nginx-internal bridge local
63023c7048a6 nginx_default bridge local
9eac7f4c98a5 none null local
17c0ad5a0c05 redir_default bridge local
ae52cbd9668a ts3_default bridge local
f076a5fa04b5 watchtower_default bridge local

esackbauer · Sep 9, 2023

I would recommend you start over with a clean docker-compose file…

Have you cheked this? See interface settings as with Hetzner
https://docs.mailcow.email/prerequisite/prerequisite-system/?h=hetzner#hetzner-cloud-and-probably-others

docs.mailcow.email

https://docs.mailcow.email/prerequisite/prerequisite-system/?h=hetzner#hetzner-cloud-and-probably-others

No preview could be generated for this link

DDocSnyd3r · Sep 9, 2023

i do not have it in my centos7, so is there a reason Alma 8 is mentioned on the page but not Alma 9, only Rocky 9?

The only strange thing I encounteres was:
docker info | grep selinux
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled
[mailcow-dockerized]$ cat /etc/docker/daemon.json
{
“selinux-enabled”: true
}

So docker has no selinux but the file is created and system had a clean boot.

esackbauer · Sep 10, 2023

Have you installed the necessary rpm as indicated here?
https://docs.mailcow.email/i_u_m/i_u_m_install/?h=selinux#check-selinux-specifics

docs.mailcow.email

https://docs.mailcow.email/i_u_m/i_u_m_install/?h=selinux#check-selinux-specifics

No preview could be generated for this link

DDocSnyd3r · Sep 10, 2023

rpm -qa | grep container-selinux
container-selinux-2.119.2-1.911c772.el7_8.noarch

Yes, it is installed

esackbauer · Sep 10, 2023

You should start over with a fresh docker-compose and read the documentation about cohosting nextcloud and nginx reverse proxy closely.
Read the requirements

docs.mailcow.email

https://docs.mailcow.email/prerequisite/prerequisite-system/

No preview could be generated for this link

and especially the red boxes.
And read the docker-compose logs!
Or better, use a separate host for mailcow, and preferrably Debian 12

Rro78 · Sep 30, 2023

I’ve the exact same issue. I’ve been running mailcow on Debian 11 for years now. Recently I had to reinstall the server (not only upgrade) with Debian 12. Since then, after a random time (at least once a day), the IPv4 of my host stop answering from my ISP. It works from other ISPs. I see a SYN, but no SYN ACK sent back. Can’t ping, can’t SSH, can’t access 80/443, etc. Like you, IPv6 is OK. As soon as I stop the mailcow containers with docker compose down, everything is back to normal.

Rro78 · Oct 1, 2023

It happened again today. I took time to restart containers one by one. The issue is fixed as soon as I restart the netfilter container.

esackbauer · Oct 1, 2023

Strange, I am running on Debian 12 (upgraded from 11) and have no issues.
I would guess that probably your hoster is the cause for this. I am running on virtualized on my homeserver.
Or it could be the dual stack IPv4 and IPv6 which gives netfilter a problem.
I am using only IPv4, but have not explicitly disabled IPv6 in mailcow.

Rro78 · Oct 1, 2023

esackbauer what should I do to troubleshoot the netfilter container and help the community to fix this issue?

Rro78 · Oct 1, 2023

I found the reason on my side, finally. Both my dedicated server & my ISP have dual stack (IPv4 + IPv6).
IPv6 has the priority when connection can be made on both.
A device on my home network (my wife’s phone) was trying to login to IMAP and for an unknown reason yet, failed to do so. After a few attempts, the IPv6 is banned. As each devices at home have a different IPv6, I don’t notice it on my side. But because the IPv6 of the phone is banned, it then tries to login to IMAP using IPv4. And at this moment, after a few attempts, my public IPv4 is banned. I don’t understand why banning my IPv4 in the netfilter container also blocks my IPv4 on the host itself, but now I understand the logic. I’ll investigate on the phone.