Be careful with update from 2024-07 to 2024-08 leads to dns error in unbound

Sstefan21 · Aug 17, 2024

mailcow 2024-07 is running on:
Proxmox Virtual Environment 8.1.4
1 socket 4 cpu, 16 GB RAM, x86-64-v2-AES

cat /etc/debian_version –> 12.6

cat /etc/apt/sources.list

deb Index of /debian

ftp-stud.hs-esslingen.de

Index of /debian

bookworm main non-free-firmware
deb-src Index of /debian

ftp-stud.hs-esslingen.de

Index of /debian

bookworm main non-free-firmware

deb Index of /debian-security

security.debian.org

Index of /debian-security

bookworm-security main non-free-firmware
deb-src Index of /debian-security

security.debian.org

Index of /debian-security

bookworm-security main non-free-firmware

deb Index of /debian

ftp-stud.hs-esslingen.de

Index of /debian

bookworm-updates main non-free-firmware
deb-src Index of /debian

ftp-stud.hs-esslingen.de

Index of /debian

bookworm-updates main non-free-firmware

cat sources.list.d/docker.list
deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.gpg] Index of linux/debian/

download.docker.com

Index of linux/debian/

bookworm stable

apt update
OK:1 Index of /debian

ftp-stud.hs-esslingen.de

Index of /debian

bookworm InRelease
Holen:2 Index of linux/debian/

download.docker.com

Index of linux/debian/

bookworm InRelease [43,3 kB]
Holen:3 Index of /debian

ftp-stud.hs-esslingen.de

Index of /debian

bookworm-updates InRelease [55,4 kB]
Holen:4 Index of /debian-security

security.debian.org

Index of /debian-security

bookworm-security InRelease [48,0 kB]
Es wurden 147 kB in 2 s geholt (95,8 kB/s).
Paketlisten werden gelesen… Fertig
Abhängigkeitsbaum wird aufgebaut… Fertig
Statusinformationen werden eingelesen… Fertig
Alle Pakete sind aktuell.

docker version
Client: Docker Engine - Community
Version: 27.1.2
API version: 1.46
Go version: go1.21.13
Git commit: d01f264
Built: Mon Aug 12 11:50:58 2024
OS/Arch: linux/amd64
Context: default

Server: Docker Engine - Community
Engine:
Version: 27.1.2
API version: 1.46 (minimum version 1.24)
Go version: go1.21.13
Git commit: f9522e5
Built: Mon Aug 12 11:50:58 2024
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.7.20
GitCommit: 8fc6bcff51318944179630522a095cc9dbf9f353
runc:
Version: 1.1.13
GitCommit: v1.1.13-0-g58aa920
docker-init:
Version: 0.19.0
GitCommit: de40ad0

docker compose version
Docker Compose version v2.29.1

docker compose exec unbound-mailcow /bin/bash
./healthcheck.sh
2024-08-17 16:16:47: Starting health check - logs can be found in /var/log/healthcheck.log
PING 1.1.1.1 (1.1.1.1): 56 data bytes

— 1.1.1.1 ping statistics —
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 21.755/26.958/34.800 ms
PING 8.8.8.8 (8.8.8.8): 56 data bytes

— 8.8.8.8 ping statistics —
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 21.730/25.025/31.548 ms
PING 9.9.9.9 (9.9.9.9): 56 data bytes

— 9.9.9.9 ping statistics —
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 26.463/29.066/34.093 ms

mailcow is working flawlessly.

Performing the update 2024-08 leads to an unbound error while # docker compose up -d. Container is not starting. Attaching to the unbound container the healthcheck is giving only errors.

Nothing has been changed in the environment.

What’s wrong? Any help is appreciated. I restored a backup, so no need to hurry…

More info while beeing attached to the unbound container:

./healthcheck.sh
2024-08-17 16:37:58: Healthcheck: DNS Resolution Failed on attempt 1 for fuzzy.mailcow.email! Trying again…
2024-08-17 16:37:58: Healthcheck: DNS Resolution Failed on attempt 2 for fuzzy.mailcow.email! Trying again…
2024-08-17 16:37:58: Healthcheck: DNS Resolution Failed on attempt 3 for fuzzy.mailcow.email! Trying again…
2024-08-17 16:37:58: Healthcheck: DNS Resolution not possible after 3 attempts for fuzzy.mailcow.email… Gave up!
2024-08-17 16:37:58: Healthcheck: DNS Resolution Failed on attempt 1 for github.com! Trying again…
2024-08-17 16:37:58: Healthcheck: DNS Resolution Failed on attempt 2 for github.com! Trying again…
2024-08-17 16:37:58: Healthcheck: DNS Resolution Failed on attempt 3 for github.com! Trying again…
2024-08-17 16:37:58: Healthcheck: DNS Resolution not possible after 3 attempts for github.com… Gave up!
2024-08-17 16:37:58: Healthcheck: DNS Resolution Failed on attempt 1 for hub.docker.com! Trying again…
2024-08-17 16:37:58: Healthcheck: DNS Resolution Failed on attempt 2 for hub.docker.com! Trying again…
2024-08-17 16:37:58: Healthcheck: DNS Resolution Failed on attempt 3 for hub.docker.com! Trying again…
2024-08-17 16:37:58: Healthcheck: DNS Resolution not possible after 3 attempts for hub.docker.com… Gave up!
2024-08-17 16:37:58: Healthcheck: Too many DNS failures (1 failures allowed, you got 3 failures), marking Healthcheck as unhealthy…

ping -c1 1.1.1.1
PING 1.1.1.1 (1.1.1.1): 56 data bytes
64 bytes from 1.1.1.1: seq=0 ttl=53 time=29.332 ms

— 1.1.1.1 ping statistics —
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 29.332/29.332/29.332 ms
ping -c1 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=54 time=28.553 ms

— 8.8.8.8 ping statistics —
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 28.553/28.553/28.553 ms
ping -c1 9.9.9.9
PING 9.9.9.9 (9.9.9.9): 56 data bytes
64 bytes from 9.9.9.9: seq=0 ttl=54 time=35.367 ms

— 9.9.9.9 ping statistics —
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 35.367/35.367/35.367 ms

dig +short fuzzy.mailcow.email
95.217.129.125

dig +short github.com
140.82.121.4

dig +short hub.docker.com
elb-default.us-east-1.aws.dckr.io.
prodextdefgreen-k0uuibjyui-4ec6503f7037d339.elb.us-east-1.amazonaws.com.
44.219.3.189
3.224.227.198
44.193.181.103

More info while beeing attached to the unbound container:

./healthcheck.sh
2024-08-17 16:37:58: Healthcheck: DNS Resolution Failed on attempt 1 for fuzzy.mailcow.email! Trying again…
2024-08-17 16:37:58: Healthcheck: DNS Resolution Failed on attempt 2 for fuzzy.mailcow.email! Trying again…
2024-08-17 16:37:58: Healthcheck: DNS Resolution Failed on attempt 3 for fuzzy.mailcow.email! Trying again…
2024-08-17 16:37:58: Healthcheck: DNS Resolution not possible after 3 attempts for fuzzy.mailcow.email… Gave up!
2024-08-17 16:37:58: Healthcheck: DNS Resolution Failed on attempt 1 for github.com! Trying again…
2024-08-17 16:37:58: Healthcheck: DNS Resolution Failed on attempt 2 for github.com! Trying again…
2024-08-17 16:37:58: Healthcheck: DNS Resolution Failed on attempt 3 for github.com! Trying again…
2024-08-17 16:37:58: Healthcheck: DNS Resolution not possible after 3 attempts for github.com… Gave up!
2024-08-17 16:37:58: Healthcheck: DNS Resolution Failed on attempt 1 for hub.docker.com! Trying again…
2024-08-17 16:37:58: Healthcheck: DNS Resolution Failed on attempt 2 for hub.docker.com! Trying again…
2024-08-17 16:37:58: Healthcheck: DNS Resolution Failed on attempt 3 for hub.docker.com! Trying again…
2024-08-17 16:37:58: Healthcheck: DNS Resolution not possible after 3 attempts for hub.docker.com… Gave up!
2024-08-17 16:37:58: Healthcheck: Too many DNS failures (1 failures allowed, you got 3 failures), marking Healthcheck as unhealthy…

ping -c1 1.1.1.1
PING 1.1.1.1 (1.1.1.1): 56 data bytes
64 bytes from 1.1.1.1: seq=0 ttl=53 time=29.332 ms

— 1.1.1.1 ping statistics —
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 29.332/29.332/29.332 ms
ping -c1 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=54 time=28.553 ms

— 8.8.8.8 ping statistics —
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 28.553/28.553/28.553 ms
ping -c1 9.9.9.9
PING 9.9.9.9 (9.9.9.9): 56 data bytes
64 bytes from 9.9.9.9: seq=0 ttl=54 time=35.367 ms

— 9.9.9.9 ping statistics —
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 35.367/35.367/35.367 ms

dig +short fuzzy.mailcow.email
95.217.129.125

dig +short github.com
140.82.121.4

dig +short hub.docker.com
elb-default.us-east-1.aws.dckr.io.
prodextdefgreen-k0uuibjyui-4ec6503f7037d339.elb.us-east-1.amazonaws.com.
44.219.3.189
3.224.227.198
44.193.181.103

More info while beeing attached to the unbound container:

healthcheck.sh
2024-08-17 16:37:58: Healthcheck: DNS Resolution Failed on attempt 1 for fuzzy.mailcow.email! Trying again…
2024-08-17 16:37:58: Healthcheck: DNS Resolution Failed on attempt 2 for fuzzy.mailcow.email! Trying again…
2024-08-17 16:37:58: Healthcheck: DNS Resolution Failed on attempt 3 for fuzzy.mailcow.email! Trying again…
2024-08-17 16:37:58: Healthcheck: DNS Resolution not possible after 3 attempts for fuzzy.mailcow.email… Gave up!
2024-08-17 16:37:58: Healthcheck: DNS Resolution Failed on attempt 1 for github.com! Trying again…
2024-08-17 16:37:58: Healthcheck: DNS Resolution Failed on attempt 2 for github.com! Trying again…
2024-08-17 16:37:58: Healthcheck: DNS Resolution Failed on attempt 3 for github.com! Trying again…
2024-08-17 16:37:58: Healthcheck: DNS Resolution not possible after 3 attempts for github.com… Gave up!
2024-08-17 16:37:58: Healthcheck: DNS Resolution Failed on attempt 1 for hub.docker.com! Trying again…
2024-08-17 16:37:58: Healthcheck: DNS Resolution Failed on attempt 2 for hub.docker.com! Trying again…
2024-08-17 16:37:58: Healthcheck: DNS Resolution Failed on attempt 3 for hub.docker.com! Trying again…
2024-08-17 16:37:58: Healthcheck: DNS Resolution not possible after 3 attempts for hub.docker.com… Gave up!
2024-08-17 16:37:58: Healthcheck: Too many DNS failures (1 failures allowed, you got 3 failures), marking Healthcheck as unhealthy…

ping -c1 1.1.1.1
PING 1.1.1.1 (1.1.1.1): 56 data bytes
64 bytes from 1.1.1.1: seq=0 ttl=53 time=29.332 ms

— 1.1.1.1 ping statistics —
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 29.332/29.332/29.332 ms
ping -c1 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=54 time=28.553 ms

— 8.8.8.8 ping statistics —
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 28.553/28.553/28.553 ms
ping -c1 9.9.9.9
PING 9.9.9.9 (9.9.9.9): 56 data bytes
64 bytes from 9.9.9.9: seq=0 ttl=54 time=35.367 ms

— 9.9.9.9 ping statistics —
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 35.367/35.367/35.367 ms

dig +short fuzzy.mailcow.email
95.217.129.125

dig +short github.com
140.82.121.4

dig +short hub.docker.com
elb-default.us-east-1.aws.dckr.io.
prodextdefgreen-k0uuibjyui-4ec6503f7037d339.elb.us-east-1.amazonaws.com.
44.219.3.189
3.224.227.198
44.193.181.103

More info while beeing attached to the unbound container:

./healthcheck.sh
2024-08-17 16:37:58: Healthcheck: DNS Resolution Failed on attempt 1 for fuzzy.mailcow.email! Trying again…
2024-08-17 16:37:58: Healthcheck: DNS Resolution Failed on attempt 2 for fuzzy.mailcow.email! Trying again…
2024-08-17 16:37:58: Healthcheck: DNS Resolution Failed on attempt 3 for fuzzy.mailcow.email! Trying again…
2024-08-17 16:37:58: Healthcheck: DNS Resolution not possible after 3 attempts for fuzzy.mailcow.email… Gave up!
2024-08-17 16:37:58: Healthcheck: DNS Resolution Failed on attempt 1 for github.com! Trying again…
2024-08-17 16:37:58: Healthcheck: DNS Resolution Failed on attempt 2 for github.com! Trying again…
2024-08-17 16:37:58: Healthcheck: DNS Resolution Failed on attempt 3 for github.com! Trying again…
2024-08-17 16:37:58: Healthcheck: DNS Resolution not possible after 3 attempts for github.com… Gave up!
2024-08-17 16:37:58: Healthcheck: DNS Resolution Failed on attempt 1 for hub.docker.com! Trying again…
2024-08-17 16:37:58: Healthcheck: DNS Resolution Failed on attempt 2 for hub.docker.com! Trying again…
2024-08-17 16:37:58: Healthcheck: DNS Resolution Failed on attempt 3 for hub.docker.com! Trying again…
2024-08-17 16:37:58: Healthcheck: DNS Resolution not possible after 3 attempts for hub.docker.com… Gave up!
2024-08-17 16:37:58: Healthcheck: Too many DNS failures (1 failures allowed, you got 3 failures), marking Healthcheck as unhealthy…

ping -c1 1.1.1.1
PING 1.1.1.1 (1.1.1.1): 56 data bytes
64 bytes from 1.1.1.1: seq=0 ttl=53 time=29.332 ms

— 1.1.1.1 ping statistics —
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 29.332/29.332/29.332 ms
ping -c1 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=54 time=28.553 ms

— 8.8.8.8 ping statistics —
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 28.553/28.553/28.553 ms
ping -c1 9.9.9.9
PING 9.9.9.9 (9.9.9.9): 56 data bytes
64 bytes from 9.9.9.9: seq=0 ttl=54 time=35.367 ms

— 9.9.9.9 ping statistics —
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 35.367/35.367/35.367 ms

dig +short fuzzy.mailcow.email
95.217.129.125

dig +short github.com
140.82.121.4

dig +short hub.docker.com
elb-default.us-east-1.aws.dckr.io.
prodextdefgreen-k0uuibjyui-4ec6503f7037d339.elb.us-east-1.amazonaws.com.
44.219.3.189
3.224.227.198
44.193.181.103

Sstefan21 · Aug 17, 2024

Workaround:

Skip unbound health check in mailcow.conf followed by docker compose down and docker compose up -d

Not really happy with this quick fix…

Another workaround:

Leave the healthcheck in mailcow.conf as it was (skip healthcheck no). Add in unbound.conf i.e. the IP of your router or an IP of a trusted DNS:

forward-zone:
name: “.”
forward-addr: 1.1.1.1

Also not happy with this…

pkernstock · Aug 17, 2024

I haven’t heard any other complaints so far - you’re the first one. Also my two mailcow installations are running fine, no watchdog issues being reported whatsoever. DNS resolution working fine. So it might be something isolated to your setup…

A few questions:

How have you performed the update, running ./update.sh I suppose?
Can you perform manual dig/nslookup/ping DNS resolutions inside the watchdog container, to see if it just affects the script?
What have you defined in setting COMPOSE_PROJECT_NAME?
Does DNS resolution of nginx.${COMPOSE_PROJECT_NAME}_mailcow-docker work inside the watchdog container?

It might be potentially related to: mailcow/mailcow-dockerized5967. Just a guess.

Sstefan21 · Aug 17, 2024

pkernstock How have you performed the update, running ./update.sh I suppose?

yes.

pkernstock Can you perform manual dig/nslookup/ping DNS resolutions inside the watchdog container, to see if it just affects the script?

No. Because
✘ Container mailcowdockerized-unbound-mailcow-1 Error
dependency failed to start: container mailcowdockerized-unbound-mailcow-1 is unhealthy

therefore:
docker compose exec watchdog-mailcow /bin/bash
service “watchdog-mailcow” is not running

pkernstock What have you defined in setting COMPOSE_PROJECT_NAME
COMPOSE_PROJECT_NAME=mailcowdockerized

pkernstock Does DNS resolution of nginx.${COMPOSE_PROJECT_NAME}_mailcow-docker work inside the watchdog container?

Can’t check. Not able to attach to the conatiner.

pkernstock So it might be something isolated to your setup…

Hmm, how could I check? Before this update all mailcow-updates before (including OS and docker) have been causing no issues. So no, I doubt that there is a problem with my setup. Anyway - give me a hint. I’ll check. But it’s quite a vanilla setup behind a opnsense and a haproxy handling the certs and restricting acces outside from the LAN or VPN. Again - until the last update no issues at all.

pkernstock Also my two mailcow installations are running fine

May I ask on a VE? If so, what specific? Which OS in the VM?

pkernstock Does DNS resolution of nginx.${COMPOSE_PROJECT_NAME}_mailcow-docker work inside the watchdog container?

With an ip forwarding to the opnsense, it looks like this:

dig nginx.mailcowdockerized_mailcow-docker

; <<>> DiG 9.18.27 <<>> nginx.mailcowdockerized_mailcow-docker
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 19530
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;nginx.mailcowdockerized_mailcow-docker. IN A

;; AUTHORITY SECTION:
. 60 IN SOA a.root-servers.net. nstld.verisign-grs.com. 2024081700 1800 900 604800 86400

;; Query time: 64 msec
;; SERVER: 127.0.0.11#53(127.0.0.11) (UDP)
;; WHEN: Sat Aug 17 17:39:20 CEST 2024
;; MSG SIZE rcvd: 142

Well - no surprise, I guess.

pkernstock It might be potentially related to: mailcow/mailcow-dockerized5967. Just a guess.

I do the certs on the opnsense. SKIP_LETS_ENCRYPT=y

Just don’t know if this could be related…

Sstefan21 · Aug 18, 2024

🕶️🐄 Moogust Update 2024 | Passwort vergessen?, SOGo 5.11, Rspamd 3.9.1 und mehr

mailcow: dockerized - Blog

🕶️🐄 Moogust Update 2024 | Passwort vergessen?, SOGo 5.11, Rspamd 3.9.1 und mehr

Moohoo alle zusammen! Der August ist da, und auch mailcow beschert euch ein größeres Update für den (hoffentlich bei euch nicht zu heißen) August. Dieses Mal sind einige größere Änderungen mit an Bord, die wir euch hiermit vorstellen wollen:

Im Unbound-Container wurden die Healthchecks überarbeitet.

Well. Someone might check the differences… Something’s definitely different.

pkernstock · Aug 18, 2024

Does docker logs --tail=150 mailcowdockerized-unbound-mailcow-1 output anything useful? Beside the failing healthchecks (which are obviously expected when unbound as the DNS server is not running) there must be something stated on why the unbound container crashes/stops.

stefan21 forward-zone:
name: “.”
forward-addr: 1.1.1.1

Also not happy with this…

Understandably, especially as external resolvers shouldn’t be used.

stefan21 May I ask on a VE? If so, what specific? Which OS in the VM?

VMware ESXi, Ubuntu 22.04, Docker 27.1.2.

stefan21 Well. Someone might check the differences… Something’s definitely different.

It’s those changes, as per release details: mailcow/mailcow-dockerized6004

Maybe something’s blocking DNS queries on your firewall?

Ganzjahresgriller · Aug 19, 2024

I just installed the update. My system is Debian12 on Arm. No problems, everything works as it should

Sstefan21 · Aug 19, 2024

pkernstock Maybe something’s blocking DNS queries on your firewall?

If so as you assume IMVHO logically the version 2024-07 must not work.

Any query from LAN to port 53 is redirected to the opnsense. Opnsense is configured to use unbound. Forwarded ports from the opnsense to the mailcow are 25, 465 and 587.

Again, this setup is working flawlessly to the monent. I restored to 2024-07. No problems at all.

Alright, for a test I’ll forward the port 53 to the mailcow. I let you know.

Aaccolon · Aug 19, 2024

stefan21

While the mailcow team shuffled things around a bit in the Unbound container’s healtcheck.sh, the check itself remained the same. It tries to resolve three different domains using Unbound inside the container:
dig +short +timeout=2 +tries=1 "$domain" @127.0.0.1

According to your error messages, all three fail.

Did you try entering the Unbound container to manually check if DNS resolution works?

Sstefan21 · Aug 19, 2024

pkernstock docker logs –tail=150 mailcowdockerized-unbound-mailcow-1

Here the output:

Setting console permissions…
Receiving anchor key…
Receiving root hints…
######################################################################## 100.0%
setup in directory /etc/unbound
Certificate request self-signature ok
subject=CN=unbound-control
removing artifacts
Setup success. Certificates created. Enable in unbound.conf file to use
2024-08-19 19:39:05,215 INFO Set uid to user 0 succeeded
2024-08-19 19:39:05,219 INFO supervisord started with pid 1
2024-08-19 19:39:06,227 INFO spawned: ‘processes’ with pid 22
2024-08-19 19:39:06,236 INFO spawned: ‘syslog-ng’ with pid 23
2024-08-19 19:39:06,246 INFO spawned: ‘unbound’ with pid 24
2024-08-19 19:39:06,254 INFO spawned: ‘unbound-healthcheck’ with pid 25
[1724089146] unbound[24:0] notice: init module 0: validator
[1724089146] unbound[24:0] notice: init module 1: iterator
Aug 19 19:39:06 1b9362a6bf20 syslog-ng[23]: syslog-ng starting up; version=‘4.7.1’
[1724089146] unbound[24:0] info: start of service (unbound 1.20.0).
2024-08-19 19:39:07,312 INFO success: processes entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-08-19 19:39:07,312 INFO success: syslog-ng entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-08-19 19:39:07,313 INFO success: unbound entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-08-19 19:39:07,313 INFO success: unbound-healthcheck entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-08-19 19:39:12: Healthcheck: DNS Resolution Failed on attempt 1 for fuzzy.mailcow.email! Trying again…
2024-08-19 19:39:12: Healthcheck: DNS Resolution Failed on attempt 2 for fuzzy.mailcow.email! Trying again…
2024-08-19 19:39:12: Healthcheck: DNS Resolution Failed on attempt 3 for fuzzy.mailcow.email! Trying again…
2024-08-19 19:39:12: Healthcheck: DNS Resolution not possible after 3 attempts for fuzzy.mailcow.email… Gave up!
2024-08-19 19:39:12: Healthcheck: DNS Resolution Failed on attempt 1 for github.com! Trying again…
2024-08-19 19:39:13: Healthcheck: DNS Resolution Failed on attempt 2 for github.com! Trying again…
2024-08-19 19:39:13: Healthcheck: DNS Resolution Failed on attempt 3 for github.com! Trying again…
2024-08-19 19:39:13: Healthcheck: DNS Resolution not possible after 3 attempts for github.com… Gave up!
2024-08-19 19:39:13: Healthcheck: DNS Resolution Failed on attempt 1 for hub.docker.com! Trying again…
2024-08-19 19:39:13: Healthcheck: DNS Resolution Failed on attempt 2 for hub.docker.com! Trying again…
2024-08-19 19:39:13: Healthcheck: DNS Resolution Failed on attempt 3 for hub.docker.com! Trying again…
2024-08-19 19:39:13: Healthcheck: DNS Resolution not possible after 3 attempts for hub.docker.com… Gave up!
2024-08-19 19:39:13: Healthcheck: Too many DNS failures (1 failures allowed, you got 3 failures), marking Healthcheck as unhealthy…
2024-08-19 19:39:49: Healthcheck: DNS Resolution Failed on attempt 1 for fuzzy.mailcow.email! Trying again…
2024-08-19 19:39:49: Healthcheck: DNS Resolution Failed on attempt 2 for fuzzy.mailcow.email! Trying again…
2024-08-19 19:39:49: Healthcheck: DNS Resolution Failed on attempt 3 for fuzzy.mailcow.email! Trying again…
2024-08-19 19:39:49: Healthcheck: DNS Resolution not possible after 3 attempts for fuzzy.mailcow.email… Gave up!
2024-08-19 19:39:49: Healthcheck: DNS Resolution Failed on attempt 1 for github.com! Trying again…
2024-08-19 19:39:49: Healthcheck: DNS Resolution Failed on attempt 2 for github.com! Trying again…
2024-08-19 19:39:49: Healthcheck: DNS Resolution Failed on attempt 3 for github.com! Trying again…
2024-08-19 19:39:49: Healthcheck: DNS Resolution not possible after 3 attempts for github.com… Gave up!
2024-08-19 19:39:50: Healthcheck: DNS Resolution Failed on attempt 1 for hub.docker.com! Trying again…
2024-08-19 19:39:50: Healthcheck: DNS Resolution Failed on attempt 2 for hub.docker.com! Trying again…
2024-08-19 19:39:50: Healthcheck: DNS Resolution Failed on attempt 3 for hub.docker.com! Trying again…
2024-08-19 19:39:50: Healthcheck: DNS Resolution not possible after 3 attempts for hub.docker.com… Gave up!
2024-08-19 19:39:50: Healthcheck: Too many DNS failures (1 failures allowed, you got 3 failures), marking Healthcheck as unhealthy…
2024-08-19 19:40:26: Healthcheck: DNS Resolution Failed on attempt 1 for fuzzy.mailcow.email! Trying again…
2024-08-19 19:40:26: Healthcheck: DNS Resolution Failed on attempt 2 for fuzzy.mailcow.email! Trying again…
2024-08-19 19:40:26: Healthcheck: DNS Resolution Failed on attempt 3 for fuzzy.mailcow.email! Trying again…
2024-08-19 19:40:26: Healthcheck: DNS Resolution not possible after 3 attempts for fuzzy.mailcow.email… Gave up!
2024-08-19 19:40:26: Healthcheck: DNS Resolution Failed on attempt 1 for github.com! Trying again…
2024-08-19 19:40:26: Healthcheck: DNS Resolution Failed on attempt 2 for github.com! Trying again…
2024-08-19 19:40:26: Healthcheck: DNS Resolution Failed on attempt 3 for github.com! Trying again…
2024-08-19 19:40:26: Healthcheck: DNS Resolution not possible after 3 attempts for github.com… Gave up!
2024-08-19 19:40:26: Healthcheck: DNS Resolution Failed on attempt 1 for hub.docker.com! Trying again…
2024-08-19 19:40:27: Healthcheck: DNS Resolution Failed on attempt 2 for hub.docker.com! Trying again…
2024-08-19 19:40:27: Healthcheck: DNS Resolution Failed on attempt 3 for hub.docker.com! Trying again…
2024-08-19 19:40:27: Healthcheck: DNS Resolution not possible after 3 attempts for hub.docker.com… Gave up!
2024-08-19 19:40:27: Healthcheck: Too many DNS failures (1 failures allowed, you got 3 failures), marking Healthcheck as unhealthy…

accolon Did you try entering the Unbound container to manually check if DNS resolution works?

Here’s the info:

docker compose exec unbound-mailcow /bin/bash
1b9362a6bf20:/# ping -c2 1.1.1.1
PING 1.1.1.1 (1.1.1.1): 56 data bytes
64 bytes from 1.1.1.1: seq=0 ttl=53 time=30.357 ms
64 bytes from 1.1.1.1: seq=1 ttl=53 time=21.828 ms

— 1.1.1.1 ping statistics —
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 21.828/26.092/30.357 ms
1b9362a6bf20:/# ping -c2 9.9.9.9
PING 9.9.9.9 (9.9.9.9): 56 data bytes
64 bytes from 9.9.9.9: seq=0 ttl=54 time=37.348 ms
64 bytes from 9.9.9.9: seq=1 ttl=54 time=30.712 ms

— 9.9.9.9 ping statistics —
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 30.712/34.030/37.348 ms
1b9362a6bf20:/# ping -c2 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=113 time=31.186 ms
64 bytes from 8.8.8.8: seq=1 ttl=113 time=22.515 ms

— 8.8.8.8 ping statistics —
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 22.515/26.850/31.186 ms

and:

dig +short +timeout=2 +tries=1 fuzzy.mailcow.email
95.217.129.125
dig +short +timeout=2 +tries=1 github.com
140.82.121.4
dig +short +timeout=2 +tries=1 hub.docker.com
elb-default.us-east-1.aws.dckr.io.
prodextdefgreen-k0uuibjyui-4ec6503f7037d339.elb.us-east-1.amazonaws.com.
44.219.3.189
3.224.227.198
44.193.181.103

What exactly does ip-forwarding?

forward-zone:
name: “.”
forward-addr: ip-of-opnsense where unbound is running as DNS

If in unbound.conf configured like this, the container starts “healthy”. Mailcow is running flawlessly.

Sstefan21 · Aug 19, 2024

stefan21 Alright, for a test I’ll forward the port 53 to the mailcow. I let you know.

Does not work.

Sstefan21 · Aug 20, 2024

For another test I disabled the forward-zone follwed by docker compose down and docker compose up -d.

From the cli: ✘ Container mailcowdockerized-unbound-mailcow-1 Error
dependency failed to start: container mailcowdockerized-unbound-mailcow-1 is unhealthy

Checking the gui reports:
unbound-mailcow (mailcow/unbound:1.23)
(Gestartet am 20.08.2024, 20:43:31)
Läuft . . .

Now what? Bug?

Alright. There’s an update 2024-08a. I’ll test and report.

Sstefan21 · Aug 20, 2024

After update to 2024-08a:

✘ Container mailcowdockerized-unbound-mailcow-1 Error
dependency failed to start: container mailcowdockerized-unbound-mailcow-1 is unhealthy

The gui reports:
unbound-mailcow (mailcow/unbound:1.23)
(Gestartet am 20.08.2024, 21:12:06)
Läuft . . .

docker logs –tail=150 mailcowdockerized-unbound-mailcow-1
Setting console permissions…
Receiving anchor key…
Receiving root hints…
######################################################################## 100.0%
setup in directory /etc/unbound
Certificate request self-signature ok
subject=CN=unbound-control
removing artifacts
Setup success. Certificates created. Enable in unbound.conf file to use
2024-08-20 21:12:12,220 INFO Set uid to user 0 succeeded
2024-08-20 21:12:12,224 INFO supervisord started with pid 1
2024-08-20 21:12:13,230 INFO spawned: ‘processes’ with pid 22
2024-08-20 21:12:13,237 INFO spawned: ‘syslog-ng’ with pid 23
2024-08-20 21:12:13,243 INFO spawned: ‘unbound’ with pid 24
2024-08-20 21:12:13,247 INFO spawned: ‘unbound-healthcheck’ with pid 25
[1724181133] unbound[24:0] notice: init module 0: validator
[1724181133] unbound[24:0] notice: init module 1: iterator
Aug 20 21:12:13 a6613c8891ac syslog-ng[23]: syslog-ng starting up; version=‘4.7.1’
[1724181133] unbound[24:0] info: start of service (unbound 1.20.0).
2024-08-20 21:12:14,316 INFO success: processes entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-08-20 21:12:14,316 INFO success: syslog-ng entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-08-20 21:12:14,316 INFO success: unbound entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-08-20 21:12:14,316 INFO success: unbound-healthcheck entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-08-20 21:12:19: Healthcheck: DNS Resolution Failed on attempt 1 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:12:19: Healthcheck: DNS Resolution Failed on attempt 2 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:12:19: Healthcheck: DNS Resolution Failed on attempt 3 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:12:19: Healthcheck: DNS Resolution not possible after 3 attempts for fuzzy.mailcow.email… Gave up!
2024-08-20 21:12:19: Healthcheck: DNS Resolution Failed on attempt 1 for github.com! Trying again…
2024-08-20 21:12:19: Healthcheck: DNS Resolution Failed on attempt 2 for github.com! Trying again…
2024-08-20 21:12:19: Healthcheck: DNS Resolution Failed on attempt 3 for github.com! Trying again…
2024-08-20 21:12:19: Healthcheck: DNS Resolution not possible after 3 attempts for github.com… Gave up!
2024-08-20 21:12:19: Healthcheck: DNS Resolution Failed on attempt 1 for hub.docker.com! Trying again…
2024-08-20 21:12:20: Healthcheck: DNS Resolution Failed on attempt 2 for hub.docker.com! Trying again…
2024-08-20 21:12:20: Healthcheck: DNS Resolution Failed on attempt 3 for hub.docker.com! Trying again…
2024-08-20 21:12:20: Healthcheck: DNS Resolution not possible after 3 attempts for hub.docker.com… Gave up!
2024-08-20 21:12:20: Healthcheck: Too many DNS failures (1 failures allowed, you got 3 failures), marking Healthcheck as unhealthy…
2024-08-20 21:12:56: Healthcheck: DNS Resolution Failed on attempt 1 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:12:56: Healthcheck: DNS Resolution Failed on attempt 2 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:12:56: Healthcheck: DNS Resolution Failed on attempt 3 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:12:56: Healthcheck: DNS Resolution not possible after 3 attempts for fuzzy.mailcow.email… Gave up!
2024-08-20 21:12:56: Healthcheck: DNS Resolution Failed on attempt 1 for github.com! Trying again…
2024-08-20 21:12:56: Healthcheck: DNS Resolution Failed on attempt 2 for github.com! Trying again…
2024-08-20 21:12:56: Healthcheck: DNS Resolution Failed on attempt 3 for github.com! Trying again…
2024-08-20 21:12:56: Healthcheck: DNS Resolution not possible after 3 attempts for github.com… Gave up!
2024-08-20 21:12:56: Healthcheck: DNS Resolution Failed on attempt 1 for hub.docker.com! Trying again…
2024-08-20 21:12:57: Healthcheck: DNS Resolution Failed on attempt 2 for hub.docker.com! Trying again…
2024-08-20 21:12:57: Healthcheck: DNS Resolution Failed on attempt 3 for hub.docker.com! Trying again…
2024-08-20 21:12:57: Healthcheck: DNS Resolution not possible after 3 attempts for hub.docker.com… Gave up!
2024-08-20 21:12:57: Healthcheck: Too many DNS failures (1 failures allowed, you got 3 failures), marking Healthcheck as unhealthy…
2024-08-20 21:13:33: Healthcheck: DNS Resolution Failed on attempt 1 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:13:33: Healthcheck: DNS Resolution Failed on attempt 2 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:13:33: Healthcheck: DNS Resolution Failed on attempt 3 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:13:33: Healthcheck: DNS Resolution not possible after 3 attempts for fuzzy.mailcow.email… Gave up!
2024-08-20 21:13:33: Healthcheck: DNS Resolution Failed on attempt 1 for github.com! Trying again…
2024-08-20 21:13:33: Healthcheck: DNS Resolution Failed on attempt 2 for github.com! Trying again…
2024-08-20 21:13:33: Healthcheck: DNS Resolution Failed on attempt 3 for github.com! Trying again…
2024-08-20 21:13:33: Healthcheck: DNS Resolution not possible after 3 attempts for github.com… Gave up!
2024-08-20 21:13:33: Healthcheck: DNS Resolution Failed on attempt 1 for hub.docker.com! Trying again…
2024-08-20 21:13:33: Healthcheck: DNS Resolution Failed on attempt 2 for hub.docker.com! Trying again…
2024-08-20 21:13:34: Healthcheck: DNS Resolution Failed on attempt 3 for hub.docker.com! Trying again…
2024-08-20 21:13:34: Healthcheck: DNS Resolution not possible after 3 attempts for hub.docker.com… Gave up!
2024-08-20 21:13:34: Healthcheck: Too many DNS failures (1 failures allowed, you got 3 failures), marking Healthcheck as unhealthy…
2024-08-20 21:14:10: Healthcheck: DNS Resolution Failed on attempt 1 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:14:10: Healthcheck: DNS Resolution Failed on attempt 2 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:14:10: Healthcheck: DNS Resolution Failed on attempt 3 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:14:10: Healthcheck: DNS Resolution not possible after 3 attempts for fuzzy.mailcow.email… Gave up!
2024-08-20 21:14:10: Healthcheck: DNS Resolution Failed on attempt 1 for github.com! Trying again…
2024-08-20 21:14:10: Healthcheck: DNS Resolution Failed on attempt 2 for github.com! Trying again…
2024-08-20 21:14:10: Healthcheck: DNS Resolution Failed on attempt 3 for github.com! Trying again…
2024-08-20 21:14:10: Healthcheck: DNS Resolution not possible after 3 attempts for github.com… Gave up!
2024-08-20 21:14:10: Healthcheck: DNS Resolution Failed on attempt 1 for hub.docker.com! Trying again…
2024-08-20 21:14:10: Healthcheck: DNS Resolution Failed on attempt 2 for hub.docker.com! Trying again…
2024-08-20 21:14:11: Healthcheck: DNS Resolution Failed on attempt 3 for hub.docker.com! Trying again…
2024-08-20 21:14:11: Healthcheck: DNS Resolution not possible after 3 attempts for hub.docker.com… Gave up!
2024-08-20 21:14:11: Healthcheck: Too many DNS failures (1 failures allowed, you got 3 failures), marking Healthcheck as unhealthy…
2024-08-20 21:14:47: Healthcheck: DNS Resolution Failed on attempt 1 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:14:47: Healthcheck: DNS Resolution Failed on attempt 2 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:14:47: Healthcheck: DNS Resolution Failed on attempt 3 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:14:47: Healthcheck: DNS Resolution not possible after 3 attempts for fuzzy.mailcow.email… Gave up!
2024-08-20 21:14:47: Healthcheck: DNS Resolution Failed on attempt 1 for github.com! Trying again…
2024-08-20 21:14:47: Healthcheck: DNS Resolution Failed on attempt 2 for github.com! Trying again…
2024-08-20 21:14:47: Healthcheck: DNS Resolution Failed on attempt 3 for github.com! Trying again…
2024-08-20 21:14:47: Healthcheck: DNS Resolution not possible after 3 attempts for github.com… Gave up!
2024-08-20 21:14:47: Healthcheck: DNS Resolution Failed on attempt 1 for hub.docker.com! Trying again…
2024-08-20 21:14:47: Healthcheck: DNS Resolution Failed on attempt 2 for hub.docker.com! Trying again…
2024-08-20 21:14:47: Healthcheck: DNS Resolution Failed on attempt 3 for hub.docker.com! Trying again…
2024-08-20 21:14:47: Healthcheck: DNS Resolution not possible after 3 attempts for hub.docker.com… Gave up!
2024-08-20 21:14:47: Healthcheck: Too many DNS failures (1 failures allowed, you got 3 failures), marking Healthcheck as unhealthy…
2024-08-20 21:15:24: Healthcheck: DNS Resolution Failed on attempt 1 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:15:24: Healthcheck: DNS Resolution Failed on attempt 2 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:15:24: Healthcheck: DNS Resolution Failed on attempt 3 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:15:24: Healthcheck: DNS Resolution not possible after 3 attempts for fuzzy.mailcow.email… Gave up!
2024-08-20 21:15:24: Healthcheck: DNS Resolution Failed on attempt 1 for github.com! Trying again…
2024-08-20 21:15:24: Healthcheck: DNS Resolution Failed on attempt 2 for github.com! Trying again…
2024-08-20 21:15:24: Healthcheck: DNS Resolution Failed on attempt 3 for github.com! Trying again…
2024-08-20 21:15:24: Healthcheck: DNS Resolution not possible after 3 attempts for github.com… Gave up!
2024-08-20 21:15:24: Healthcheck: DNS Resolution Failed on attempt 1 for hub.docker.com! Trying again…
2024-08-20 21:15:24: Healthcheck: DNS Resolution Failed on attempt 2 for hub.docker.com! Trying again…
2024-08-20 21:15:24: Healthcheck: DNS Resolution Failed on attempt 3 for hub.docker.com! Trying again…
2024-08-20 21:15:24: Healthcheck: DNS Resolution not possible after 3 attempts for hub.docker.com… Gave up!
2024-08-20 21:15:24: Healthcheck: Too many DNS failures (1 failures allowed, you got 3 failures), marking Healthcheck as unhealthy…
2024-08-20 21:16:00: Healthcheck: DNS Resolution Failed on attempt 1 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:16:00: Healthcheck: DNS Resolution Failed on attempt 2 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:16:01: Healthcheck: DNS Resolution Failed on attempt 3 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:16:01: Healthcheck: DNS Resolution not possible after 3 attempts for fuzzy.mailcow.email… Gave up!
2024-08-20 21:16:01: Healthcheck: DNS Resolution Failed on attempt 1 for github.com! Trying again…
2024-08-20 21:16:01: Healthcheck: DNS Resolution Failed on attempt 2 for github.com! Trying again…
2024-08-20 21:16:01: Healthcheck: DNS Resolution Failed on attempt 3 for github.com! Trying again…
2024-08-20 21:16:01: Healthcheck: DNS Resolution not possible after 3 attempts for github.com… Gave up!
2024-08-20 21:16:01: Healthcheck: DNS Resolution Failed on attempt 1 for hub.docker.com! Trying again…
2024-08-20 21:16:01: Healthcheck: DNS Resolution Failed on attempt 2 for hub.docker.com! Trying again…
2024-08-20 21:16:01: Healthcheck: DNS Resolution Failed on attempt 3 for hub.docker.com! Trying again…
2024-08-20 21:16:01: Healthcheck: DNS Resolution not possible after 3 attempts for hub.docker.com… Gave up!
2024-08-20 21:16:01: Healthcheck: Too many DNS failures (1 failures allowed, you got 3 failures), marking Healthcheck as unhealthy…
2024-08-20 21:16:37: Healthcheck: DNS Resolution Failed on attempt 1 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:16:37: Healthcheck: DNS Resolution Failed on attempt 2 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:16:37: Healthcheck: DNS Resolution Failed on attempt 3 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:16:37: Healthcheck: DNS Resolution not possible after 3 attempts for fuzzy.mailcow.email… Gave up!
2024-08-20 21:16:38: Healthcheck: DNS Resolution Failed on attempt 1 for github.com! Trying again…
2024-08-20 21:16:38: Healthcheck: DNS Resolution Failed on attempt 2 for github.com! Trying again…
2024-08-20 21:16:38: Healthcheck: DNS Resolution Failed on attempt 3 for github.com! Trying again…
2024-08-20 21:16:38: Healthcheck: DNS Resolution not possible after 3 attempts for github.com… Gave up!
2024-08-20 21:16:38: Healthcheck: DNS Resolution Failed on attempt 1 for hub.docker.com! Trying again…
2024-08-20 21:16:38: Healthcheck: DNS Resolution Failed on attempt 2 for hub.docker.com! Trying again…
2024-08-20 21:16:38: Healthcheck: DNS Resolution Failed on attempt 3 for hub.docker.com! Trying again…
2024-08-20 21:16:38: Healthcheck: DNS Resolution not possible after 3 attempts for hub.docker.com… Gave up!
2024-08-20 21:16:38: Healthcheck: Too many DNS failures (1 failures allowed, you got 3 failures), marking Healthcheck as unhealthy…
2024-08-20 21:17:14: Healthcheck: DNS Resolution Failed on attempt 1 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:17:14: Healthcheck: DNS Resolution Failed on attempt 2 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:17:14: Healthcheck: DNS Resolution Failed on attempt 3 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:17:14: Healthcheck: DNS Resolution not possible after 3 attempts for fuzzy.mailcow.email… Gave up!
2024-08-20 21:17:14: Healthcheck: DNS Resolution Failed on attempt 1 for github.com! Trying again…
2024-08-20 21:17:15: Healthcheck: DNS Resolution Failed on attempt 2 for github.com! Trying again…
2024-08-20 21:17:15: Healthcheck: DNS Resolution Failed on attempt 3 for github.com! Trying again…
2024-08-20 21:17:15: Healthcheck: DNS Resolution not possible after 3 attempts for github.com… Gave up!
2024-08-20 21:17:15: Healthcheck: DNS Resolution Failed on attempt 1 for hub.docker.com! Trying again…
2024-08-20 21:17:15: Healthcheck: DNS Resolution Failed on attempt 2 for hub.docker.com! Trying again…
2024-08-20 21:17:15: Healthcheck: DNS Resolution Failed on attempt 3 for hub.docker.com! Trying again…
2024-08-20 21:17:15: Healthcheck: DNS Resolution not possible after 3 attempts for hub.docker.com… Gave up!
2024-08-20 21:17:15: Healthcheck: Too many DNS failures (1 failures allowed, you got 3 failures), marking Healthcheck as unhealthy…

Inside the unbound container:
ping -c2 1.1.1.1
PING 1.1.1.1 (1.1.1.1): 56 data bytes
64 bytes from 1.1.1.1: seq=0 ttl=53 time=42.549 ms
64 bytes from 1.1.1.1: seq=1 ttl=53 time=23.276 ms

— 1.1.1.1 ping statistics —
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 23.276/32.912/42.549 ms
a6613c8891ac:/# ping -c2 9.9.9.9
PING 9.9.9.9 (9.9.9.9): 56 data bytes
64 bytes from 9.9.9.9: seq=0 ttl=54 time=53.028 ms
64 bytes from 9.9.9.9: seq=1 ttl=54 time=27.660 ms

— 9.9.9.9 ping statistics —
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 27.660/40.344/53.028 ms
a6613c8891ac:/# ping -c2 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=113 time=34.309 ms
64 bytes from 8.8.8.8: seq=1 ttl=113 time=21.203 ms

dig +short github.com
140.82.121.3
a6613c8891ac:/# dig +short fuzzy.mailcow.email
95.217.129.125
a6613c8891ac:/# dig +short hub.docker.com
elb-default.us-east-1.aws.dckr.io.
prodextdefgreen-k0uuibjyui-4ec6503f7037d339.elb.us-east-1.amazonaws.com.
44.219.3.189
3.224.227.198
44.193.181.103

nslookup community.mailcow.email
Server: 127.0.0.11
Address: 127.0.0.11#53

Non-authoritative answer:
community.mailcow.email canonical name = web01.kernstock.net.
Name: web01.kernstock.net
Address: 5.1.87.87

nslookup google.de 127.0.0.11
Server: 127.0.0.11
Address: 127.0.0.11#53

Non-authoritative answer:
Name: google.de
Address: 216.58.206.67
Name: google.de
Address: 2a00:1450:4001:81d::2003

Weird. I don’t get it.

Sstefan21 · Aug 20, 2024

The GUI reports:
unbound-mailcow (mailcow/unbound:1.23)

Report from docker logs –tail=150 mailcowdockerized-unbound-mailcow-1
[1724181133] unbound[24:0] info: start of service (unbound 1.20.0).

What version is running?

Aaccolon · Aug 21, 2024

stefan21 What version is running?

Unbound 1.20 inside a Docker container based on mailcow’s latest container image tagged as version 1.23

hub.docker.com

https://hub.docker.com/r/mailcow/unbound/tags

No preview could be generated for this link

. These are different version numbers.

stefan21 a6613c8891ac:/# dig +short fuzzy.mailcow.email
95.217.129.125

I don’t get this, either. Looks like DNS resolution is working, so no idea why the healthcheck script thinks it doesn’t.

The script logs an error when either the dig command doesn’t return error code 0 or its output is empty:
mailcow/mailcow-dockerizedblob/8971b11c49bde0899cc5c3ea49c3e8c975af54ee/data/Dockerfiles/unbound/healthcheck.sh#L56

If I was affected, I would probably build a minimal version of healthcheck.sh with just the check_dnspart and try to debug this…

Sstefan21 · Aug 21, 2024

accolon Looks like DNS resolution is working

Yes. Ping is working, dig is working and dnslookup is working from inside of the container.

I’m not happy with tweaking on email servers. I’d like to have this part of IT really stable and slow.

DocFraggle · Aug 21, 2024

I suggest you edit your data/Dockerfiles/unbound/healthcheck.sh script and add some debug log output starting at line 53

mailcow/mailcow-dockerizedblob/master/data/Dockerfiles/unbound/healthcheck.sh#L53

For example, output the details of the dig command in $dig_output and maybe even the return code. The script just checks if rc=0, but maybe it’s not 0, dig can return multiple return codes apart from 1

The script may still run things differently from you running commands inside the container

pkernstock · Aug 21, 2024

Maybe something related to this particular issue where dockerd misbehaved apparently? mailcow/mailcow-dockerized6042

What I’m confused about is: You said that unbound container isn’t starting properly. But you’re still running dig commands in the same container, right? Are you running them shortly before the container stops, or when you have your workaround in place?

Sstefan21 · Aug 22, 2024

@pkernstock

To clarify:

Without a forward-zone:
docker compose up -d leads to

✘ Container mailcowdockerized-unbound-mailcow-1 Error
dependency failed to start: container mailcowdockerized-unbound-mailcow-1 is unhealthy

and:

docker compose exec unbound-mailcow /bin/bash enters the container. Inside the container ping, dig and nslookup are working. Therefore I assume DNS is working.

Defining a forward-zone to the opnsense leads to a healthy start of the unbound-container (all containers). No errors at all.

Inside the container:

nslookup fuzzy.mailcow.email
Server: 127.0.0.11
Address: 127.0.0.11#53

Non-authoritative answer:
Name: fuzzy.mailcow.email
Address: 95.217.129.125

less healthcheck.sh:

for domain in “${domains[@]}” ; do
success=false
for ((i=1; i<=3; i++)); do
dig_output=$(dig +short +timeout=2 +tries=1 “$domain” @127.0.0.1 2>/dev/null)
dig_rc=$?

    if [ $dig_rc -ne 0 ] || [ -z "$dig_output" ]; then
        log_to_stdout "Healthcheck: DNS Resolution Failed on attempt $i for $domain! Trying again..."
    else
        success=true
        break
    fi
done

Testing:

1.) dig +short +timeout=2 +tries=1 fuzzy.mailcow.email @127.0.0.1 is giving nothing back
2.) dig +short +timeout=2 +tries=1 fuzzy.mailcow.email @127.0.0.11
95.217.129.125

Altering healthcheck.sh inside the container will not persist a restart.

Sstefan21 · Aug 22, 2024

Must be the script.

Disabled the forward-zone. Did a docker compose down followed by a docker compose up -d. Entered the unbound container and altered the healthcheck.sh to 127.0.0.11. Did a docker compose restart unbound-mailcow.
[+] Restarting 1/1
Container mailcowdockerized-unbound-mailcow-1 Started

docker compose ps
mailcowdockerized-unbound-mailcow-1 mailcow/unbound:1.23 “/docker-entrypoint.…” unbound-mailcow 24 minutes ago Up 11 minutes (healthy) 53/tcp, 53/udp

I’ll define a forward-zone as workaround until this is fixed.