I just installed the update. My system is Debian12 on Arm. No problems, everything works as it should
EnglishBe careful with update from 2024-07 to 2024-08 leads to dns error in unbound
pkernstock Maybe something’s blocking DNS queries on your firewall?
If so as you assume IMVHO logically the version 2024-07 must not work.
Any query from LAN to port 53 is redirected to the opnsense. Opnsense is configured to use unbound. Forwarded ports from the opnsense to the mailcow are 25, 465 and 587.
Again, this setup is working flawlessly to the monent. I restored to 2024-07. No problems at all.
Alright, for a test I’ll forward the port 53 to the mailcow. I let you know.
While the mailcow team shuffled things around a bit in the Unbound container’s healtcheck.sh, the check itself remained the same. It tries to resolve three different domains using Unbound inside the container:
dig +short +timeout=2 +tries=1 "$domain" @127.0.0.1
According to your error messages, all three fail.
Did you try entering the Unbound container to manually check if DNS resolution works?
pkernstock docker logs –tail=150 mailcowdockerized-unbound-mailcow-1
Here the output:
Setting console permissions…
Receiving anchor key…
Receiving root hints…
######################################################################## 100.0%
setup in directory /etc/unbound
Certificate request self-signature ok
subject=CN=unbound-control
removing artifacts
Setup success. Certificates created. Enable in unbound.conf file to use
2024-08-19 19:39:05,215 INFO Set uid to user 0 succeeded
2024-08-19 19:39:05,219 INFO supervisord started with pid 1
2024-08-19 19:39:06,227 INFO spawned: ‘processes’ with pid 22
2024-08-19 19:39:06,236 INFO spawned: ‘syslog-ng’ with pid 23
2024-08-19 19:39:06,246 INFO spawned: ‘unbound’ with pid 24
2024-08-19 19:39:06,254 INFO spawned: ‘unbound-healthcheck’ with pid 25
[1724089146] unbound[24:0] notice: init module 0: validator
[1724089146] unbound[24:0] notice: init module 1: iterator
Aug 19 19:39:06 1b9362a6bf20 syslog-ng[23]: syslog-ng starting up; version=‘4.7.1’
[1724089146] unbound[24:0] info: start of service (unbound 1.20.0).
2024-08-19 19:39:07,312 INFO success: processes entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-08-19 19:39:07,312 INFO success: syslog-ng entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-08-19 19:39:07,313 INFO success: unbound entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-08-19 19:39:07,313 INFO success: unbound-healthcheck entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-08-19 19:39:12: Healthcheck: DNS Resolution Failed on attempt 1 for fuzzy.mailcow.email! Trying again…
2024-08-19 19:39:12: Healthcheck: DNS Resolution Failed on attempt 2 for fuzzy.mailcow.email! Trying again…
2024-08-19 19:39:12: Healthcheck: DNS Resolution Failed on attempt 3 for fuzzy.mailcow.email! Trying again…
2024-08-19 19:39:12: Healthcheck: DNS Resolution not possible after 3 attempts for fuzzy.mailcow.email… Gave up!
2024-08-19 19:39:12: Healthcheck: DNS Resolution Failed on attempt 1 for github.com! Trying again…
2024-08-19 19:39:13: Healthcheck: DNS Resolution Failed on attempt 2 for github.com! Trying again…
2024-08-19 19:39:13: Healthcheck: DNS Resolution Failed on attempt 3 for github.com! Trying again…
2024-08-19 19:39:13: Healthcheck: DNS Resolution not possible after 3 attempts for github.com… Gave up!
2024-08-19 19:39:13: Healthcheck: DNS Resolution Failed on attempt 1 for hub.docker.com! Trying again…
2024-08-19 19:39:13: Healthcheck: DNS Resolution Failed on attempt 2 for hub.docker.com! Trying again…
2024-08-19 19:39:13: Healthcheck: DNS Resolution Failed on attempt 3 for hub.docker.com! Trying again…
2024-08-19 19:39:13: Healthcheck: DNS Resolution not possible after 3 attempts for hub.docker.com… Gave up!
2024-08-19 19:39:13: Healthcheck: Too many DNS failures (1 failures allowed, you got 3 failures), marking Healthcheck as unhealthy…
2024-08-19 19:39:49: Healthcheck: DNS Resolution Failed on attempt 1 for fuzzy.mailcow.email! Trying again…
2024-08-19 19:39:49: Healthcheck: DNS Resolution Failed on attempt 2 for fuzzy.mailcow.email! Trying again…
2024-08-19 19:39:49: Healthcheck: DNS Resolution Failed on attempt 3 for fuzzy.mailcow.email! Trying again…
2024-08-19 19:39:49: Healthcheck: DNS Resolution not possible after 3 attempts for fuzzy.mailcow.email… Gave up!
2024-08-19 19:39:49: Healthcheck: DNS Resolution Failed on attempt 1 for github.com! Trying again…
2024-08-19 19:39:49: Healthcheck: DNS Resolution Failed on attempt 2 for github.com! Trying again…
2024-08-19 19:39:49: Healthcheck: DNS Resolution Failed on attempt 3 for github.com! Trying again…
2024-08-19 19:39:49: Healthcheck: DNS Resolution not possible after 3 attempts for github.com… Gave up!
2024-08-19 19:39:50: Healthcheck: DNS Resolution Failed on attempt 1 for hub.docker.com! Trying again…
2024-08-19 19:39:50: Healthcheck: DNS Resolution Failed on attempt 2 for hub.docker.com! Trying again…
2024-08-19 19:39:50: Healthcheck: DNS Resolution Failed on attempt 3 for hub.docker.com! Trying again…
2024-08-19 19:39:50: Healthcheck: DNS Resolution not possible after 3 attempts for hub.docker.com… Gave up!
2024-08-19 19:39:50: Healthcheck: Too many DNS failures (1 failures allowed, you got 3 failures), marking Healthcheck as unhealthy…
2024-08-19 19:40:26: Healthcheck: DNS Resolution Failed on attempt 1 for fuzzy.mailcow.email! Trying again…
2024-08-19 19:40:26: Healthcheck: DNS Resolution Failed on attempt 2 for fuzzy.mailcow.email! Trying again…
2024-08-19 19:40:26: Healthcheck: DNS Resolution Failed on attempt 3 for fuzzy.mailcow.email! Trying again…
2024-08-19 19:40:26: Healthcheck: DNS Resolution not possible after 3 attempts for fuzzy.mailcow.email… Gave up!
2024-08-19 19:40:26: Healthcheck: DNS Resolution Failed on attempt 1 for github.com! Trying again…
2024-08-19 19:40:26: Healthcheck: DNS Resolution Failed on attempt 2 for github.com! Trying again…
2024-08-19 19:40:26: Healthcheck: DNS Resolution Failed on attempt 3 for github.com! Trying again…
2024-08-19 19:40:26: Healthcheck: DNS Resolution not possible after 3 attempts for github.com… Gave up!
2024-08-19 19:40:26: Healthcheck: DNS Resolution Failed on attempt 1 for hub.docker.com! Trying again…
2024-08-19 19:40:27: Healthcheck: DNS Resolution Failed on attempt 2 for hub.docker.com! Trying again…
2024-08-19 19:40:27: Healthcheck: DNS Resolution Failed on attempt 3 for hub.docker.com! Trying again…
2024-08-19 19:40:27: Healthcheck: DNS Resolution not possible after 3 attempts for hub.docker.com… Gave up!
2024-08-19 19:40:27: Healthcheck: Too many DNS failures (1 failures allowed, you got 3 failures), marking Healthcheck as unhealthy…
accolon Did you try entering the Unbound container to manually check if DNS resolution works?
Here’s the info:
docker compose exec unbound-mailcow /bin/bash
1b9362a6bf20:/# ping -c2 1.1.1.1
PING 1.1.1.1 (1.1.1.1): 56 data bytes
64 bytes from 1.1.1.1: seq=0 ttl=53 time=30.357 ms
64 bytes from 1.1.1.1: seq=1 ttl=53 time=21.828 ms
— 1.1.1.1 ping statistics —
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 21.828/26.092/30.357 ms
1b9362a6bf20:/# ping -c2 9.9.9.9
PING 9.9.9.9 (9.9.9.9): 56 data bytes
64 bytes from 9.9.9.9: seq=0 ttl=54 time=37.348 ms
64 bytes from 9.9.9.9: seq=1 ttl=54 time=30.712 ms
— 9.9.9.9 ping statistics —
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 30.712/34.030/37.348 ms
1b9362a6bf20:/# ping -c2 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=113 time=31.186 ms
64 bytes from 8.8.8.8: seq=1 ttl=113 time=22.515 ms
— 8.8.8.8 ping statistics —
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 22.515/26.850/31.186 ms
and:
dig +short +timeout=2 +tries=1 fuzzy.mailcow.email
95.217.129.125
dig +short +timeout=2 +tries=1 github.com
140.82.121.4
dig +short +timeout=2 +tries=1 hub.docker.com
elb-default.us-east-1.aws.dckr.io.
prodextdefgreen-k0uuibjyui-4ec6503f7037d339.elb.us-east-1.amazonaws.com.
44.219.3.189
3.224.227.198
44.193.181.103
What exactly does ip-forwarding?
forward-zone:
name: “.”
forward-addr: ip-of-opnsense where unbound is running as DNS
If in unbound.conf configured like this, the container starts “healthy”. Mailcow is running flawlessly.
For another test I disabled the forward-zone follwed by docker compose down and docker compose up -d.
From the cli: ✘ Container mailcowdockerized-unbound-mailcow-1 Error
dependency failed to start: container mailcowdockerized-unbound-mailcow-1 is unhealthy
Checking the gui reports:
unbound-mailcow (mailcow/unbound:1.23)
(Gestartet am 20.08.2024, 20:43:31)
Läuft . . .
Now what? Bug?
Alright. There’s an update 2024-08a. I’ll test and report.
After update to 2024-08a:
✘ Container mailcowdockerized-unbound-mailcow-1 Error
dependency failed to start: container mailcowdockerized-unbound-mailcow-1 is unhealthy
The gui reports:
unbound-mailcow (mailcow/unbound:1.23)
(Gestartet am 20.08.2024, 21:12:06)
Läuft . . .
docker logs –tail=150 mailcowdockerized-unbound-mailcow-1
Setting console permissions…
Receiving anchor key…
Receiving root hints…
######################################################################## 100.0%
setup in directory /etc/unbound
Certificate request self-signature ok
subject=CN=unbound-control
removing artifacts
Setup success. Certificates created. Enable in unbound.conf file to use
2024-08-20 21:12:12,220 INFO Set uid to user 0 succeeded
2024-08-20 21:12:12,224 INFO supervisord started with pid 1
2024-08-20 21:12:13,230 INFO spawned: ‘processes’ with pid 22
2024-08-20 21:12:13,237 INFO spawned: ‘syslog-ng’ with pid 23
2024-08-20 21:12:13,243 INFO spawned: ‘unbound’ with pid 24
2024-08-20 21:12:13,247 INFO spawned: ‘unbound-healthcheck’ with pid 25
[1724181133] unbound[24:0] notice: init module 0: validator
[1724181133] unbound[24:0] notice: init module 1: iterator
Aug 20 21:12:13 a6613c8891ac syslog-ng[23]: syslog-ng starting up; version=‘4.7.1’
[1724181133] unbound[24:0] info: start of service (unbound 1.20.0).
2024-08-20 21:12:14,316 INFO success: processes entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-08-20 21:12:14,316 INFO success: syslog-ng entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-08-20 21:12:14,316 INFO success: unbound entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-08-20 21:12:14,316 INFO success: unbound-healthcheck entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-08-20 21:12:19: Healthcheck: DNS Resolution Failed on attempt 1 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:12:19: Healthcheck: DNS Resolution Failed on attempt 2 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:12:19: Healthcheck: DNS Resolution Failed on attempt 3 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:12:19: Healthcheck: DNS Resolution not possible after 3 attempts for fuzzy.mailcow.email… Gave up!
2024-08-20 21:12:19: Healthcheck: DNS Resolution Failed on attempt 1 for github.com! Trying again…
2024-08-20 21:12:19: Healthcheck: DNS Resolution Failed on attempt 2 for github.com! Trying again…
2024-08-20 21:12:19: Healthcheck: DNS Resolution Failed on attempt 3 for github.com! Trying again…
2024-08-20 21:12:19: Healthcheck: DNS Resolution not possible after 3 attempts for github.com… Gave up!
2024-08-20 21:12:19: Healthcheck: DNS Resolution Failed on attempt 1 for hub.docker.com! Trying again…
2024-08-20 21:12:20: Healthcheck: DNS Resolution Failed on attempt 2 for hub.docker.com! Trying again…
2024-08-20 21:12:20: Healthcheck: DNS Resolution Failed on attempt 3 for hub.docker.com! Trying again…
2024-08-20 21:12:20: Healthcheck: DNS Resolution not possible after 3 attempts for hub.docker.com… Gave up!
2024-08-20 21:12:20: Healthcheck: Too many DNS failures (1 failures allowed, you got 3 failures), marking Healthcheck as unhealthy…
2024-08-20 21:12:56: Healthcheck: DNS Resolution Failed on attempt 1 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:12:56: Healthcheck: DNS Resolution Failed on attempt 2 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:12:56: Healthcheck: DNS Resolution Failed on attempt 3 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:12:56: Healthcheck: DNS Resolution not possible after 3 attempts for fuzzy.mailcow.email… Gave up!
2024-08-20 21:12:56: Healthcheck: DNS Resolution Failed on attempt 1 for github.com! Trying again…
2024-08-20 21:12:56: Healthcheck: DNS Resolution Failed on attempt 2 for github.com! Trying again…
2024-08-20 21:12:56: Healthcheck: DNS Resolution Failed on attempt 3 for github.com! Trying again…
2024-08-20 21:12:56: Healthcheck: DNS Resolution not possible after 3 attempts for github.com… Gave up!
2024-08-20 21:12:56: Healthcheck: DNS Resolution Failed on attempt 1 for hub.docker.com! Trying again…
2024-08-20 21:12:57: Healthcheck: DNS Resolution Failed on attempt 2 for hub.docker.com! Trying again…
2024-08-20 21:12:57: Healthcheck: DNS Resolution Failed on attempt 3 for hub.docker.com! Trying again…
2024-08-20 21:12:57: Healthcheck: DNS Resolution not possible after 3 attempts for hub.docker.com… Gave up!
2024-08-20 21:12:57: Healthcheck: Too many DNS failures (1 failures allowed, you got 3 failures), marking Healthcheck as unhealthy…
2024-08-20 21:13:33: Healthcheck: DNS Resolution Failed on attempt 1 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:13:33: Healthcheck: DNS Resolution Failed on attempt 2 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:13:33: Healthcheck: DNS Resolution Failed on attempt 3 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:13:33: Healthcheck: DNS Resolution not possible after 3 attempts for fuzzy.mailcow.email… Gave up!
2024-08-20 21:13:33: Healthcheck: DNS Resolution Failed on attempt 1 for github.com! Trying again…
2024-08-20 21:13:33: Healthcheck: DNS Resolution Failed on attempt 2 for github.com! Trying again…
2024-08-20 21:13:33: Healthcheck: DNS Resolution Failed on attempt 3 for github.com! Trying again…
2024-08-20 21:13:33: Healthcheck: DNS Resolution not possible after 3 attempts for github.com… Gave up!
2024-08-20 21:13:33: Healthcheck: DNS Resolution Failed on attempt 1 for hub.docker.com! Trying again…
2024-08-20 21:13:33: Healthcheck: DNS Resolution Failed on attempt 2 for hub.docker.com! Trying again…
2024-08-20 21:13:34: Healthcheck: DNS Resolution Failed on attempt 3 for hub.docker.com! Trying again…
2024-08-20 21:13:34: Healthcheck: DNS Resolution not possible after 3 attempts for hub.docker.com… Gave up!
2024-08-20 21:13:34: Healthcheck: Too many DNS failures (1 failures allowed, you got 3 failures), marking Healthcheck as unhealthy…
2024-08-20 21:14:10: Healthcheck: DNS Resolution Failed on attempt 1 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:14:10: Healthcheck: DNS Resolution Failed on attempt 2 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:14:10: Healthcheck: DNS Resolution Failed on attempt 3 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:14:10: Healthcheck: DNS Resolution not possible after 3 attempts for fuzzy.mailcow.email… Gave up!
2024-08-20 21:14:10: Healthcheck: DNS Resolution Failed on attempt 1 for github.com! Trying again…
2024-08-20 21:14:10: Healthcheck: DNS Resolution Failed on attempt 2 for github.com! Trying again…
2024-08-20 21:14:10: Healthcheck: DNS Resolution Failed on attempt 3 for github.com! Trying again…
2024-08-20 21:14:10: Healthcheck: DNS Resolution not possible after 3 attempts for github.com… Gave up!
2024-08-20 21:14:10: Healthcheck: DNS Resolution Failed on attempt 1 for hub.docker.com! Trying again…
2024-08-20 21:14:10: Healthcheck: DNS Resolution Failed on attempt 2 for hub.docker.com! Trying again…
2024-08-20 21:14:11: Healthcheck: DNS Resolution Failed on attempt 3 for hub.docker.com! Trying again…
2024-08-20 21:14:11: Healthcheck: DNS Resolution not possible after 3 attempts for hub.docker.com… Gave up!
2024-08-20 21:14:11: Healthcheck: Too many DNS failures (1 failures allowed, you got 3 failures), marking Healthcheck as unhealthy…
2024-08-20 21:14:47: Healthcheck: DNS Resolution Failed on attempt 1 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:14:47: Healthcheck: DNS Resolution Failed on attempt 2 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:14:47: Healthcheck: DNS Resolution Failed on attempt 3 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:14:47: Healthcheck: DNS Resolution not possible after 3 attempts for fuzzy.mailcow.email… Gave up!
2024-08-20 21:14:47: Healthcheck: DNS Resolution Failed on attempt 1 for github.com! Trying again…
2024-08-20 21:14:47: Healthcheck: DNS Resolution Failed on attempt 2 for github.com! Trying again…
2024-08-20 21:14:47: Healthcheck: DNS Resolution Failed on attempt 3 for github.com! Trying again…
2024-08-20 21:14:47: Healthcheck: DNS Resolution not possible after 3 attempts for github.com… Gave up!
2024-08-20 21:14:47: Healthcheck: DNS Resolution Failed on attempt 1 for hub.docker.com! Trying again…
2024-08-20 21:14:47: Healthcheck: DNS Resolution Failed on attempt 2 for hub.docker.com! Trying again…
2024-08-20 21:14:47: Healthcheck: DNS Resolution Failed on attempt 3 for hub.docker.com! Trying again…
2024-08-20 21:14:47: Healthcheck: DNS Resolution not possible after 3 attempts for hub.docker.com… Gave up!
2024-08-20 21:14:47: Healthcheck: Too many DNS failures (1 failures allowed, you got 3 failures), marking Healthcheck as unhealthy…
2024-08-20 21:15:24: Healthcheck: DNS Resolution Failed on attempt 1 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:15:24: Healthcheck: DNS Resolution Failed on attempt 2 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:15:24: Healthcheck: DNS Resolution Failed on attempt 3 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:15:24: Healthcheck: DNS Resolution not possible after 3 attempts for fuzzy.mailcow.email… Gave up!
2024-08-20 21:15:24: Healthcheck: DNS Resolution Failed on attempt 1 for github.com! Trying again…
2024-08-20 21:15:24: Healthcheck: DNS Resolution Failed on attempt 2 for github.com! Trying again…
2024-08-20 21:15:24: Healthcheck: DNS Resolution Failed on attempt 3 for github.com! Trying again…
2024-08-20 21:15:24: Healthcheck: DNS Resolution not possible after 3 attempts for github.com… Gave up!
2024-08-20 21:15:24: Healthcheck: DNS Resolution Failed on attempt 1 for hub.docker.com! Trying again…
2024-08-20 21:15:24: Healthcheck: DNS Resolution Failed on attempt 2 for hub.docker.com! Trying again…
2024-08-20 21:15:24: Healthcheck: DNS Resolution Failed on attempt 3 for hub.docker.com! Trying again…
2024-08-20 21:15:24: Healthcheck: DNS Resolution not possible after 3 attempts for hub.docker.com… Gave up!
2024-08-20 21:15:24: Healthcheck: Too many DNS failures (1 failures allowed, you got 3 failures), marking Healthcheck as unhealthy…
2024-08-20 21:16:00: Healthcheck: DNS Resolution Failed on attempt 1 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:16:00: Healthcheck: DNS Resolution Failed on attempt 2 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:16:01: Healthcheck: DNS Resolution Failed on attempt 3 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:16:01: Healthcheck: DNS Resolution not possible after 3 attempts for fuzzy.mailcow.email… Gave up!
2024-08-20 21:16:01: Healthcheck: DNS Resolution Failed on attempt 1 for github.com! Trying again…
2024-08-20 21:16:01: Healthcheck: DNS Resolution Failed on attempt 2 for github.com! Trying again…
2024-08-20 21:16:01: Healthcheck: DNS Resolution Failed on attempt 3 for github.com! Trying again…
2024-08-20 21:16:01: Healthcheck: DNS Resolution not possible after 3 attempts for github.com… Gave up!
2024-08-20 21:16:01: Healthcheck: DNS Resolution Failed on attempt 1 for hub.docker.com! Trying again…
2024-08-20 21:16:01: Healthcheck: DNS Resolution Failed on attempt 2 for hub.docker.com! Trying again…
2024-08-20 21:16:01: Healthcheck: DNS Resolution Failed on attempt 3 for hub.docker.com! Trying again…
2024-08-20 21:16:01: Healthcheck: DNS Resolution not possible after 3 attempts for hub.docker.com… Gave up!
2024-08-20 21:16:01: Healthcheck: Too many DNS failures (1 failures allowed, you got 3 failures), marking Healthcheck as unhealthy…
2024-08-20 21:16:37: Healthcheck: DNS Resolution Failed on attempt 1 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:16:37: Healthcheck: DNS Resolution Failed on attempt 2 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:16:37: Healthcheck: DNS Resolution Failed on attempt 3 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:16:37: Healthcheck: DNS Resolution not possible after 3 attempts for fuzzy.mailcow.email… Gave up!
2024-08-20 21:16:38: Healthcheck: DNS Resolution Failed on attempt 1 for github.com! Trying again…
2024-08-20 21:16:38: Healthcheck: DNS Resolution Failed on attempt 2 for github.com! Trying again…
2024-08-20 21:16:38: Healthcheck: DNS Resolution Failed on attempt 3 for github.com! Trying again…
2024-08-20 21:16:38: Healthcheck: DNS Resolution not possible after 3 attempts for github.com… Gave up!
2024-08-20 21:16:38: Healthcheck: DNS Resolution Failed on attempt 1 for hub.docker.com! Trying again…
2024-08-20 21:16:38: Healthcheck: DNS Resolution Failed on attempt 2 for hub.docker.com! Trying again…
2024-08-20 21:16:38: Healthcheck: DNS Resolution Failed on attempt 3 for hub.docker.com! Trying again…
2024-08-20 21:16:38: Healthcheck: DNS Resolution not possible after 3 attempts for hub.docker.com… Gave up!
2024-08-20 21:16:38: Healthcheck: Too many DNS failures (1 failures allowed, you got 3 failures), marking Healthcheck as unhealthy…
2024-08-20 21:17:14: Healthcheck: DNS Resolution Failed on attempt 1 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:17:14: Healthcheck: DNS Resolution Failed on attempt 2 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:17:14: Healthcheck: DNS Resolution Failed on attempt 3 for fuzzy.mailcow.email! Trying again…
2024-08-20 21:17:14: Healthcheck: DNS Resolution not possible after 3 attempts for fuzzy.mailcow.email… Gave up!
2024-08-20 21:17:14: Healthcheck: DNS Resolution Failed on attempt 1 for github.com! Trying again…
2024-08-20 21:17:15: Healthcheck: DNS Resolution Failed on attempt 2 for github.com! Trying again…
2024-08-20 21:17:15: Healthcheck: DNS Resolution Failed on attempt 3 for github.com! Trying again…
2024-08-20 21:17:15: Healthcheck: DNS Resolution not possible after 3 attempts for github.com… Gave up!
2024-08-20 21:17:15: Healthcheck: DNS Resolution Failed on attempt 1 for hub.docker.com! Trying again…
2024-08-20 21:17:15: Healthcheck: DNS Resolution Failed on attempt 2 for hub.docker.com! Trying again…
2024-08-20 21:17:15: Healthcheck: DNS Resolution Failed on attempt 3 for hub.docker.com! Trying again…
2024-08-20 21:17:15: Healthcheck: DNS Resolution not possible after 3 attempts for hub.docker.com… Gave up!
2024-08-20 21:17:15: Healthcheck: Too many DNS failures (1 failures allowed, you got 3 failures), marking Healthcheck as unhealthy…
Inside the unbound container:
ping -c2 1.1.1.1
PING 1.1.1.1 (1.1.1.1): 56 data bytes
64 bytes from 1.1.1.1: seq=0 ttl=53 time=42.549 ms
64 bytes from 1.1.1.1: seq=1 ttl=53 time=23.276 ms
— 1.1.1.1 ping statistics —
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 23.276/32.912/42.549 ms
a6613c8891ac:/# ping -c2 9.9.9.9
PING 9.9.9.9 (9.9.9.9): 56 data bytes
64 bytes from 9.9.9.9: seq=0 ttl=54 time=53.028 ms
64 bytes from 9.9.9.9: seq=1 ttl=54 time=27.660 ms
— 9.9.9.9 ping statistics —
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 27.660/40.344/53.028 ms
a6613c8891ac:/# ping -c2 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=113 time=34.309 ms
64 bytes from 8.8.8.8: seq=1 ttl=113 time=21.203 ms
dig +short github.com
140.82.121.3
a6613c8891ac:/# dig +short fuzzy.mailcow.email
95.217.129.125
a6613c8891ac:/# dig +short hub.docker.com
elb-default.us-east-1.aws.dckr.io.
prodextdefgreen-k0uuibjyui-4ec6503f7037d339.elb.us-east-1.amazonaws.com.
44.219.3.189
3.224.227.198
44.193.181.103
nslookup community.mailcow.email
Server: 127.0.0.11
Address: 127.0.0.11#53
Non-authoritative answer:
community.mailcow.email canonical name = web01.kernstock.net.
Name: web01.kernstock.net
Address: 5.1.87.87
nslookup google.de 127.0.0.11
Server: 127.0.0.11
Address: 127.0.0.11#53
Non-authoritative answer:
Name: google.de
Address: 216.58.206.67
Name: google.de
Address: 2a00:1450:4001:81d::2003
Weird. I don’t get it.
The GUI reports:
unbound-mailcow (mailcow/unbound:1.23)
Report from docker logs –tail=150 mailcowdockerized-unbound-mailcow-1
[1724181133] unbound[24:0] info: start of service (unbound 1.20.0).
What version is running?
stefan21 What version is running?
Unbound 1.20 inside a Docker container based on mailcow’s latest container image tagged as version 1.23. These are different version numbers.
stefan21 a6613c8891ac:/# dig +short fuzzy.mailcow.email
95.217.129.125
I don’t get this, either. Looks like DNS resolution is working, so no idea why the healthcheck script thinks it doesn’t.
The script logs an error when either the dig command doesn’t return error code 0 or its output is empty:
mailcow/mailcow-dockerizedblob/8971b11c49bde0899cc5c3ea49c3e8c975af54ee/data/Dockerfiles/unbound/healthcheck.sh#L56
If I was affected, I would probably build a minimal version of healthcheck.sh with just the check_dnspart and try to debug this…
I suggest you edit your data/Dockerfiles/unbound/healthcheck.sh script and add some debug log output starting at line 53
mailcow/mailcow-dockerizedblob/master/data/Dockerfiles/unbound/healthcheck.sh#L53
For example, output the details of the dig command in $dig_output and maybe even the return code. The script just checks if rc=0, but maybe it’s not 0, dig can return multiple return codes apart from 1
The script may still run things differently from you running commands inside the container
Maybe something related to this particular issue where dockerd misbehaved apparently? mailcow/mailcow-dockerized6042
What I’m confused about is: You said that unbound container isn’t starting properly. But you’re still running dig commands in the same container, right? Are you running them shortly before the container stops, or when you have your workaround in place?
- Edited
To clarify:
Without a forward-zone:
docker compose up -d leads to
✘ Container mailcowdockerized-unbound-mailcow-1 Error
dependency failed to start: container mailcowdockerized-unbound-mailcow-1 is unhealthy
and:
docker compose exec unbound-mailcow /bin/bash enters the container. Inside the container ping, dig and nslookup are working. Therefore I assume DNS is working.
Defining a forward-zone to the opnsense leads to a healthy start of the unbound-container (all containers). No errors at all.
Inside the container:
nslookup fuzzy.mailcow.email
Server: 127.0.0.11
Address: 127.0.0.11#53
Non-authoritative answer:
Name: fuzzy.mailcow.email
Address: 95.217.129.125
less healthcheck.sh:
for domain in “${domains[@]}” ; do
success=false
for ((i=1; i<=3; i++)); do
dig_output=$(dig +short +timeout=2 +tries=1 “$domain” @127.0.0.1 2>/dev/null)
dig_rc=$?
if [ $dig_rc -ne 0 ] || [ -z "$dig_output" ]; then
log_to_stdout "Healthcheck: DNS Resolution Failed on attempt $i for $domain! Trying again..."
else
success=true
break
fi
doneTesting:
1.) dig +short +timeout=2 +tries=1 fuzzy.mailcow.email @127.0.0.1 is giving nothing back
2.) dig +short +timeout=2 +tries=1 fuzzy.mailcow.email @127.0.0.11
95.217.129.125
Altering healthcheck.sh inside the container will not persist a restart.
Must be the script.
Disabled the forward-zone. Did a docker compose down followed by a docker compose up -d. Entered the unbound container and altered the healthcheck.sh to 127.0.0.11. Did a docker compose restart unbound-mailcow.
[+] Restarting 1/1
Container mailcowdockerized-unbound-mailcow-1 Started
docker compose ps
mailcowdockerized-unbound-mailcow-1 mailcow/unbound:1.23 “/docker-entrypoint.…” unbound-mailcow 24 minutes ago Up 11 minutes (healthy) 53/tcp, 53/udp
I’ll define a forward-zone as workaround until this is fixed.
stefan21 altered the healthcheck.sh to 127.0.0.11
127.0.0.11 is Docker’s own internal DNS service. All containers use 127.0.0.11 to include internal names of Docker containers. Your change simply asks another DNS server instead of Unbound, which effectively makes the health check useless.
stefan21 1.) dig +short +timeout=2 +tries=1 fuzzy.mailcow.email @127.0.0.1 is giving nothing back
That’s why the health check fails, so it’s working correctly. In your tests above, I missed that you skipped the @127.0.0.1 part, i.e. you were not asking Unbound directly.
So the question now is why your Unbound is not resolving DNS requests.
DocFraggle I suggest you edit your data/Dockerfiles/unbound/healthcheck.sh script and add some debug log output starting at line 53
mailcow/mailcow-dockerizedblob/master/data/Dockerfiles/unbound/healthcheck.sh#L53
For example, output the details of the dig command in $dig_output and maybe even the return code. The script just checks if rc=0, but maybe it’s not 0, dig can return multiple return codes apart from 1
The script may still run things differently from you running commands inside the container
Which brings me back to my suggestion from above… why don’t you have a deeper look and log the details from the healthcheck script?
- Edited
Found it.
I’m redirecting all DNS requests from LAN to the opnsense. As I already wrote, on the opnsense UNBOUND is running. I have to think about creating seperate rules for the mailcow, or leave the workaround with the forward-zone.
Question remains why did it work in version 2024-07 (obviously because of changing the healthcheck script), and what reason for is DNS working inside of the unbound-container working with ping, dig and nslookup. Why in general, changing the script? Running mailcow behind a proper configured firewall will cause hickups.
I’ll stop this now. Maybe someone can mark this thread as solved or tell me, how to do it.
BTW - did I overlook something in the doku? Can’t find anything about this special? issue. Is it special to redirect any DNS request from a LAN to a firewall and let the firewall do the job? AFAIK there are reasons to configure a FW in this way…
stefan21 BTW - did I overlook something in the doku?
Yes: 
Why unbound? - mailcow: dockerized documentation. It is NOT recommended to use any other external resolver and have unbound resolve everything for you, by directly using the root nameservers. docs.mailcow.email
I read this. I do understand. My opnsense is using unbound as resolver. Because I do know about the problems with external resolvers. There are reasons why a sysadmin forces any device in the LAN to use only the resolver from a firewall.
What I don’t like is drilling holes in a firewall… of course a mail server needs ports (25, 465, 587) to communicate with other mail servers. Why icmp and DNS can’t be used from a firewall, IDK.
Anyway - as I know the pro’s and con’s of my setup, I’ll stay with a forward-zone in the mailcow pointing to the IP of my opnsense.
You didn’t answer to my question, why the healthscript worked before. I didn’t change anything in my firewall. Do you know the reason?
This seems to be the same …
container unbound unhealthy