I am running mailcow on ARM via Oracle cloud. It was working fine until a week prior that I find out my SSL certificate didn't get renewed. I transferred my Mailcow setup from an x86 machine to ARM following the Migration instruction on mailcow documentation. So I am not sure if ACME ever worked on ARM. Right now when I check logs on ACME docker, it stopped at waiting for nginx. I didn't see any error messages. At the moment, I manually get SSL certificate and overwrite files in data/assets/ssl folder to make it work. But I do want to figure out what's wrong with my ACME docker.

You need to dive more deeply into docker logs. Find out why it is waiting. Maybe nginx is also waiting for another container like unbound container (which does DNS resolution)? or watchdog container logs give a hint? Oracle Cloud is known for giving problems for mailcow...

@"DocFraggle"#p17638 I reboot the docker and still same result as * Could not resolve host: nginx * Closing connection curl: (6) Could not resolve host: nginx / #

@"DocFraggle"#p17640 Yes. I did the docker compose down and up

ACME not renewing certificated

dnwk

esackbauer
It is very confusing for me on what’s going on because I don’t see any errors on acme container. The logs there are very minimal. I tried force_renew and nothing happens and no errors.

esackbauer
Nginx isn’t waiting on something else as web interface is up and functioning.

This is all I see in the logs
root@g:/opt/mailcow-dockerized# docker logs mailcowdockerized-acme-mailcow-1 Fri Aug 9 15:42:09 PDT 2024 - Waiting for Docker API... Fri Aug 9 15:42:09 PDT 2024 - Docker API OK Fri Aug 9 15:42:09 PDT 2024 - Waiting for Postfix... Fri Aug 9 15:42:09 PDT 2024 - Postfix OK Fri Aug 9 15:42:09 PDT 2024 - Waiting for Dovecot... Fri Aug 9 15:42:09 PDT 2024 - Dovecot OK Fri Aug 9 15:42:09 PDT 2024 - Waiting for database... Fri Aug 9 15:42:09 PDT 2024 - Database OK Fri Aug 9 15:42:09 PDT 2024 - Waiting for Nginx...

DocFraggle

This is the part for Nginx:

log_f "Waiting for Nginx..."
until $(curl --output /dev/null --silent --head --fail http://nginx:8081); do
  sleep 2
done

(https://github.com/search?q=repo%3Amailcow%2Fmailcow-dockerized%20%22Waiting%20for%20Nginx...%22&type=code)

You could try a

curl -v http://nginx:8081

from inside the acme container to see what’s the problem here

dnwk

DocFraggle

When I try to run curl for nginx, I got this error

Could not resolve host: nginx
Closing connection
curl: (6) Could not resolve host: nginx

I think this maybe the reason that ACME can’t reach nginx. But why?

In docker-compose.yml, I do see alias for nginx.
networks:
mailcow-network:
aliases:
- nginx

So I guess for some reason, this alias didn’t work.
I am running it in Oracle Cloud ARM. I don’t have other issue with my instance.

DocFraggle

You ran this command from inside the ACME container and got this error? That’s strange… did you try to restart the whole stack?

docker compose down
docker compose up -d

dnwk

DocFraggle

I reboot the docker and still same result as

Could not resolve host: nginx
Closing connection
curl: (6) Could not resolve host: nginx
/ #

DocFraggle

Did you reboot or run the command I posted above? Rebooting the server won’t fix problems

dnwk

DocFraggle
Yes. I did the docker compose down and up

dnwk

DocFraggle

This seems very weird. Even though curl can’t get to nginx alias and complains host not found. But I can ping nginx just fine within ACME container.

/ # ping -c 3 nginx
PING nginx (172.22.1.8): 56 data bytes
64 bytes from 172.22.1.8: seq=0 ttl=64 time=0.163 ms
64 bytes from 172.22.1.8: seq=1 ttl=64 time=0.113 ms
64 bytes from 172.22.1.8: seq=2 ttl=64 time=0.113 ms

— nginx ping statistics —
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.113/0.129/0.163 ms
/ # curl -v http://nginx:8081

Could not resolve host: nginx
Closing connection
curl: (6) Could not resolve host: nginx

Going back on ACME logs, it seems like this happened around July 1st. I don’t remember changing any settings on the 1st.

DocFraggle

Are you running the latest version? There was a curl bug in a recent mailcow version

dnwk

DocFraggle
Yes. I ran update.sh multiple times yesterday. I am using ARM version.

marcoswos

I had this problem. In my case, when I entered acme-mailcow, I noticed that the command curl http://nginx:8081 was not responding and was not seeing nginx.

To solve this, I put the local IP of nginx in the etc host of acme-mailcow.

docker-compose exec acme-mailcow /bin/sh
in /etc/hosts I put:
172.22.1.8 nginx

I ran the script to update the certificates.
./srv/acme.sh
exit #exit the container.
docker-compose down
docker-compose up -d

dnwk

marcoswos
I thought the file you changed in docker is not permanent? Reboot or update will wipe the file.

For now I am running certbot on the host machine. If I have to run a script to modify docker everytime, I might just run certbot on the host.

Also, I am not sure why ping nginx works as expected. Does Ping command not use DNS?

DocFraggle

As I wrote before, there was a DNS bug in curl in an earlier version which has been fixed (or downgraded, not sure). But I’m not sure if it was fixed for ARM as well

dnwk

DocFraggle

Shall I open an issue in Github for Mailcow?

Ok. Look like there is a confirmed issue about ACME DNS mailcow/mailcow-dockerized5962
And there is a pull request to the nightly release mailcow/mailcow-dockerized5981

I don’t see any documentation on when it would be push to stable release.