The GUI reports:
unbound-mailcow (mailcow/unbound:1.23)
Report from docker logs –tail=150 mailcowdockerized-unbound-mailcow-1
[1724181133] unbound[24:0] info: start of service (unbound 1.20.0).
What version is running?
English
The GUI reports:
unbound-mailcow (mailcow/unbound:1.23)
Report from docker logs –tail=150 mailcowdockerized-unbound-mailcow-1
[1724181133] unbound[24:0] info: start of service (unbound 1.20.0).
What version is running?
stefan21 What version is running?
Unbound 1.20 inside a Docker container based on mailcow’s latest . These are different version numbers.
stefan21 a6613c8891ac:/# dig +short fuzzy.mailcow.email
95.217.129.125
I don’t get this, either. Looks like DNS resolution is working, so no idea why the healthcheck script thinks it doesn’t.
The script logs an error when either the dig command doesn’t return error code 0 or its output is empty:
mailcow/mailcow-dockerizedblob/8971b11c49bde0899cc5c3ea49c3e8c975af54ee/data/Dockerfiles/unbound/healthcheck.sh#L56
If I was affected, I would probably build a minimal version of healthcheck.sh
with just the check_dns
part and try to debug this…
I suggest you edit your data/Dockerfiles/unbound/healthcheck.sh script and add some debug log output starting at line 53
mailcow/mailcow-dockerizedblob/master/data/Dockerfiles/unbound/healthcheck.sh#L53
For example, output the details of the dig command in $dig_output and maybe even the return code. The script just checks if rc=0, but maybe it’s not 0, dig can return multiple return codes apart from 1
The script may still run things differently from you running commands inside the container
Maybe something related to this particular issue where dockerd misbehaved apparently? mailcow/mailcow-dockerized6042
What I’m confused about is: You said that unbound container isn’t starting properly. But you’re still running dig
commands in the same container, right? Are you running them shortly before the container stops, or when you have your workaround in place?
To clarify:
Without a forward-zone:
docker compose up -d leads to
✘ Container mailcowdockerized-unbound-mailcow-1 Error
dependency failed to start: container mailcowdockerized-unbound-mailcow-1 is unhealthy
and:
docker compose exec unbound-mailcow /bin/bash enters the container. Inside the container ping, dig and nslookup are working. Therefore I assume DNS is working.
Defining a forward-zone to the opnsense leads to a healthy start of the unbound-container (all containers). No errors at all.
Inside the container:
nslookup fuzzy.mailcow.email
Server: 127.0.0.11
Address: 127.0.0.11#53
Non-authoritative answer:
Name: fuzzy.mailcow.email
Address: 95.217.129.125
less healthcheck.sh:
for domain in “${domains[@]}” ; do
success=false
for ((i=1; i<=3; i++)); do
dig_output=$(dig +short +timeout=2 +tries=1 “$domain” @127.0.0.1 2>/dev/null)
dig_rc=$?
if [ $dig_rc -ne 0 ] || [ -z "$dig_output" ]; then
log_to_stdout "Healthcheck: DNS Resolution Failed on attempt $i for $domain! Trying again..."
else
success=true
break
fi
done
Testing:
1.) dig +short +timeout=2 +tries=1 fuzzy.mailcow.email @127.0.0.1 is giving nothing back
2.) dig +short +timeout=2 +tries=1 fuzzy.mailcow.email @127.0.0.11
95.217.129.125
Altering healthcheck.sh inside the container will not persist a restart.
Must be the script.
Disabled the forward-zone. Did a docker compose down followed by a docker compose up -d. Entered the unbound container and altered the healthcheck.sh to 127.0.0.11. Did a docker compose restart unbound-mailcow.
[+] Restarting 1/1
Container mailcowdockerized-unbound-mailcow-1 Started
docker compose ps
mailcowdockerized-unbound-mailcow-1 mailcow/unbound:1.23 “/docker-entrypoint.…” unbound-mailcow 24 minutes ago Up 11 minutes (healthy) 53/tcp, 53/udp
I’ll define a forward-zone as workaround until this is fixed.
stefan21 altered the healthcheck.sh to 127.0.0.11
127.0.0.11 is Docker’s own internal DNS service. All containers use 127.0.0.11 to include internal names of Docker containers. Your change simply asks another DNS server instead of Unbound, which effectively makes the health check useless.
stefan21 1.) dig +short +timeout=2 +tries=1 fuzzy.mailcow.email @127.0.0.1 is giving nothing back
That’s why the health check fails, so it’s working correctly. In your tests above, I missed that you skipped the @127.0.0.1
part, i.e. you were not asking Unbound directly.
So the question now is why your Unbound is not resolving DNS requests.
DocFraggle I suggest you edit your data/Dockerfiles/unbound/healthcheck.sh script and add some debug log output starting at line 53
For example, output the details of the dig command in $dig_output and maybe even the return code. The script just checks if rc=0, but maybe it’s not 0, dig can return multiple return codes apart from 1
The script may still run things differently from you running commands inside the container
Which brings me back to my suggestion from above… why don’t you have a deeper look and log the details from the healthcheck script?
Found it.
I’m redirecting all DNS requests from LAN to the opnsense. As I already wrote, on the opnsense UNBOUND is running. I have to think about creating seperate rules for the mailcow, or leave the workaround with the forward-zone.
Question remains why did it work in version 2024-07 (obviously because of changing the healthcheck script), and what reason for is DNS working inside of the unbound-container working with ping, dig and nslookup. Why in general, changing the script? Running mailcow behind a proper configured firewall will cause hickups.
I’ll stop this now. Maybe someone can mark this thread as solved or tell me, how to do it.
BTW - did I overlook something in the doku? Can’t find anything about this special? issue. Is it special to redirect any DNS request from a LAN to a firewall and let the firewall do the job? AFAIK there are reasons to configure a FW in this way…
stefan21 BTW - did I overlook something in the doku?
Yes: . It is NOT recommended to use any other external resolver and have unbound resolve everything for you, by directly using the root nameservers.
I read this. I do understand. My opnsense is using unbound as resolver. Because I do know about the problems with external resolvers. There are reasons why a sysadmin forces any device in the LAN to use only the resolver from a firewall.
What I don’t like is drilling holes in a firewall… of course a mail server needs ports (25, 465, 587) to communicate with other mail servers. Why icmp and DNS can’t be used from a firewall, IDK.
Anyway - as I know the pro’s and con’s of my setup, I’ll stay with a forward-zone in the mailcow pointing to the IP of my opnsense.
You didn’t answer to my question, why the healthscript worked before. I didn’t change anything in my firewall. Do you know the reason?
I am having this issue with a fresh mailcow install.
After the heathcheck fails, I exec into the container and run the dig command from the healthcheck and get an error
Seems unbound isn’t actually running here? Thoughts/suggestions?
I have the same issue. Manually updating healtchec.sh to @127.0.0.11 solve this healtcheck issue, but unbound is still not workuing correctly for the other docker containers. As i use multiple docker networks i use a non standard IPV4_NETWORK=172.40.1 in mailcow.conf which might correlate with the problem.
Just want to mention that everything works without problems if you insert @127.0.0.11 in healtcheck: sed 's/127.0.0.1 /127.0.0.11 /' ./data/Dockerfiles/unbound/healthcheck.sh > ./data/Dockerfiles/unbound/healthcheck.sh
delete all dns:
entries and its items in docker-compose.yml
exec docker build data/Dockerfiles/unbound -t mailcow/unbound:own
add in docker-compose.overwrite.yml
:
services:
unbound-mailcow:
build: ./data/Dockerfiles/unbound
image: mailcow/unbound:own
execute update.sh
I just upgraded to the November release and had this exact problem. I did find a root cause IN MY SETUP. Sharing in case it helps.
Setup:
Issue:
Solution:
Tagging @DocFraggle here as well. Maybe you know what I should do to debug this properly? Thank you! <3
Skittluier Do you have a special setup for you server? Any local firewall running on the host system which may block DNS requests from inside the docker network? The PITA-award-winning selinux enabled?