custom certificate not working - old certificates still used

WolfTongue

Hello,

I want to use my own reverse proxy which is working perfectly fine with my mailcow webinterfaces. The only thing is about the certificates.

I followed the steps described here: https://docs.mailcow.email/post_installation/firststeps-ssl/#how-to-use-your-own-certificate

disable letsencrypt
keep track about your own cert.pem and key.pem (using fullchain.pem and privkey.pem from issued letsencrypt cert of my reverse proxy)
restart postfix, nginx and dovecot

Right now it somehow uses my old certificate which was made by LetsEncrypt and should expire end of this month. Somehow it is not switching to the new one and I really need help here.

Things I discovered:

postfix uses /etc/ssl/certs/ssl-cert-snakeoil.pem (and same name but .key) which is still the old certificate (no idea where this is put in from)
working old letsencrypt cert is SHA256withRSA and new is SHA384withECDSA

Things I did:

full restart docker compose down and docker compose up -d
server restart
verified that the cert name and alt names are the same (they are even in correct order)

I checked log files but I was unable to discover errors. All services are up and running but all are using the old certificate. Can someone please help me why it is not working?

esackbauer

WolfTongue keep track about your own cert.pem and key.pem

What do you mean with “keep track”? those files have to be copied from the reverse proxy (not linked!) and have to use exactly these names.

accolon

WolfTongue postfix uses /etc/ssl/certs/ssl-cert-snakeoil.pem (and same name but .key) which is still the old certificate (no idea where this is put in from)

I guess this is your main problem.

The certificates used by mailcow’s Postfix are configured in ./data/conf/postfix/main.cf which is a static file tracked by Github. The configuration looks like this and has been the same for years:

smtpd_tls_cert_file = /etc/ssl/mail/cert.pem
smtpd_tls_key_file = /etc/ssl/mail/key.pem

The cert path is a Docker volume which points to ./data/assets/ssl inside the mailcow directory on the host.

If this folder doesn’t exist, the mailcow update.sh script creates it and copies the snake oil default certs from ./data/assets/ssl-example which have been the same for several years as well.

At no point mailcow uses files called ssl-cert-snakeoil.pem.

What does your ./data/conf/postfix/main.cf look like? Did you change this file manually?

WolfTongue

I am using a script which is exactly doing the same as the post hook script from the documentation: https://docs.mailcow.email/post_installation/reverse-proxy/r_p/#optional-post-hook-script-for-non-mailcow-acme-clients

It copies the files so that there is a cert.pem and a key.pem. I checked both and both are perfectly fine. I checked the postfix-mailcow container and they get correctly mounted into /etc/ssl/mail with the correct content (can be access from within the container). Also the permissions seem to be fine: key.pem is 600 and cert.pem is 644

esackbauer

Mailcow keeps using the default “snakeoil” certificates if the replacement process is incomplete, or if the correct filenames, permissions, or container reloads are missing after copying custom certificates to /data/assets/ssl
Check if all SAN names required are included in the LE cert from the proxy (e.g. also autodiscover, autoconfig as indicated in DNS requirements.

WolfTongue

esackbauer Thx for the fast feedback.
The DNS entries are fine, one SRV was missing so it was a good hint. But all in all it did not change anything unfortunately. I used the proposed tests which passed all so it should be fine

Can it be that an RSA certificate is needed and ECDSA is just not working? Also do you know where to find the snakeoil implementation than I could read through the code

WolfTongue

I tried now different things but none worked. So I stopped this try now as it seems like mailcow is not ECDSA ready or compatible. Thx again for the help

esackbauer

WolfTongue as it seems like mailcow is not ECDSA ready or compatible.

That is not true. Mine is connecting via LE certificates as ECDSA with SHA-384.

WolfTongue

accolon
First of all I reverted all my changes and stick to default mailcow behaviour. My goal was to bring a web application firewall to protect the webpages but it seems to be not possible if not using RSA for some reason. I checked a ton of github issues having more or less the same issue and everyone talks like “works out of the box” but for me it is not. It seems like mailcow is not supporting ECDSA fr me.

Regarding the main.cf:
The content of the main.cf in the host data folder is correct. I also figured out that it is mounted to /opt/postfix/conf. Initial I thought it was mounted to /etc/postfix which contains the snakeoil pem files.

If I exchange the certificates from ./data/assets/ssl with my ECDSA ones I just get problems and stuff I can not figure out why it is this way. Just for the records, I sticked to the official documentation: https://docs.mailcow.email/post_installation/firststeps-ssl/#how-to-use-your-own-certificate

So to solve it step by step I wanted to get postfix running first. But sadly I was unable to fix postfix as it was extremely strange how it behaved. The tls certificates in /opt/postfix/conf/main.cf are set to use /etc/ssl/mail/cert.pem (and key.pem). This file contains the ECDSA cert valid till january. But for some reason if I check the certificate using openssl cli it tells it is still the old one (commands see https://docs.mailcow.email/post_installation/firststeps-ssl/#check-your-configuration). I tried to test with other tools like online tools and some tell the old and some tell it is broken (not sure if some cache or not).

As I spent hours to solve this I now stopped putting more power in this. I am very sad about this as using a WAF for the last 1-2 months showed me that my mailcow is constantly under attack. Not only for logins but also all kind of tries to reach config files as well as sending bad or broken requests to hack into the system. But as said I gave up and maybe try again as soon as mailcow switch to ECDSA certificates.

esackbauer

I use Sophos Firewall as WAF and also as MTA in front of mailcow, so I do not care so much what mailcow is supporting.
It is free for personal use.

WolfTongue

esackbauer I was a bit fustrated when I wrote my last reply as nothing worked out. So don’t get it wrong - I really like MailCow and would never trade it for another solution. That is also why I donate to the project from time to time to support them

barrtttt

I had similar issue, in my case I created dockers with self generated cert by mailcow. Then I added custom ssl generated by certbot (I use wildcard cert).

Check if old folder with self generated certs is in this localization /opt/mailcow-dockerized/data/assets/ssl/
if yes, you must delete it then restart containers and check (https://docs.mailcow.email/post_installation/reverse-proxy/r_p/#optional-post-hook-script-for-non-mailcow-acme-clients)

if you followed the guide the folder name is mail.yourdomain.com
when I deleted this, mailcow could get correct ssl certs

I checked fingerprint for ssl cerst using this command. (In my case i use webmail.mydomain.com for web ui)
If anyone knows better way to check ssl cerst, pls let me know

HOST="mail.mydomain.com"; WEBHOST="webmail.mydomain.com"; CERT="/opt/mailcow-dockerized/data/assets/ssl/cert.pem"; \
openssl x509 -noout -fingerprint -sha256 -in "$CERT"; \
openssl s_client -starttls smtp -connect "$HOST:587" -servername "$HOST" < /dev/null | openssl x509 -noout -fingerprint -sha256; \
openssl s_client -connect "$HOST:465" -servername "$HOST" < /dev/null | openssl x509 -noout -fingerprint -sha256; \
openssl s_client -starttls imap -connect "$HOST:143" -servername "$HOST" < /dev/null | openssl x509 -noout -fingerprint -sha256; \
openssl s_client -connect "$HOST:993" -servername "$HOST" < /dev/null | openssl x509 -noout -fingerprint -sha256; \
openssl s_client -connect "$HOST:443" -servername "$HOST" < /dev/null | openssl x509 -noout -fingerprint -sha256; \
openssl s_client -connect "$WEBHOST:443" -servername "$WEBHOST" < /dev/null | openssl x509 -noout -fingerprint -sha256

idk if this will solve your issue

ps. I also removed acme-mailcow container and commented in docker compose but it was not the root case
I’m not sure if should be removed or not with custom ssl