SMTP from other docker to Mailcow-Dockerized on the same host

funktionierbar

Hello everybody.
I run a VPN with a few Dockers, one of them mailcow, all of them behind an apache reverse proxy. Mailcow is handed new externally created letsencrypt-certificates every month via a cronjob. All is well. Except:
I want to SMTP-send Mails from the other dockers (e.g. vaultwarden and ghost) via my mailcow-instance.
I am stuck since a few weeks now, and the documentation on this is meager and above my head.

I have two different ways of debugging. I change the config of vaultwarden and send test-mails, and i use openssl s_client -connect from the dockers internal shell. I read the dockers and mailcows postfix logs.

I have tried so far:

using the mail.example.com FQDN host from vaultwarden, and it returns: SMTP error: Connection error: Connection refused (os error 111)
using the internal IP of mailcow 172.22.1.1 and I get the error SMTP error: Connection error: error:0A000086:SSL routines:tls_post_process_server_certificate:certificate verify failed:../ssl/statem/statem_clnt.c:1889: (IP address mismatch)

How is that internal relay adress used anyway? I use Letsencrypt-Certificates for the FQDN and the autodiscover domains, but they dont contain local hostnames or IPs. Are they supposed to? I don’t want to allow unencrypted transfer via port 25, or is there a way to allow it only from the inside?

I added the vaultwarden docker to the mailcow-network, and tried to use postfix’ internal IP 172.22.1.253 (same erorr as above, and postfix container name, then I get the error SMTP error: Connection error: error:0A000086:SSL routines:tls_post_process_server_certificate:certificate verify failed:../ssl/statem/statem_clnt.c:1889: (hostname mismatch)

When I check vaultwardens option to accept invalid certs I can send mail, but the other docker I want to use (ghost) does not have that option.

And, why does mailcow block my request from the other docker when using the FQDN? I can SMTP that way from my laptop, with openssl and thunderbird, why not from the other docker? Also, while vaultwarden and ghost have the identical problem, nextcloud doesn’t seem to mind at all, it just works with the same SMTP configuration. Firewall is off for now.

I tried to follow this guide: https://docs.mailcow.email/de/post_installation/firststeps-local_mta/ but it didn’t make any difference.

Thank you for thinking with me! I feel kind of stupid here, I have a feeling the solution is going to be so very obvious…

esackbauer

mailcow exposes its services on the IP address of its docker host.
Use this IP address in Vaultwarden. It works with my installation.

funktionierbar

Thanks for your reply. When I try to use 127.0.0.1, localhost, or 0.0.0.0 as the SMTP host in vaultwarden, I get the same error as when I use my FQDN: SMTP error: Connection error: Connection refused (os error 111)
The request never shows up on mailcows/mailcows-postifxes logs.
Is that what you suggested?

esackbauer

Do not use the loopback address, it is not routable. and why do you think 0.0.0.0 is a reachable IP address???
It seems you need to learn some TCP/IP basics…

funktionierbar

DocFraggle

Well yes, I thought so too. A solution would be to include the IP in the certificate, but afaik Letsencrypt does not allow that.

Vaultwarden does even have an UI Option to accept invalid certificates, and if I check that I am able to send emails from vaultwarden, but Ghost does not have that, and,
I would much rather just use valid encryption and send mails via my FQDN to mailcow, just as it work from other systems. Why is the connection refused when I try to do that? How can I find out?

When I enter the FQDN as a SMPT host in Ghost, the connection also gets refused, but with a small difference, the logs read:

Error: connect ECONNREFUSED 127.0.0.1:587
    at createMailError (/var/lib/ghost/versions/5.95.0/core/server/services/mail/GhostMailer.js:105:12)
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1555:16)

So apparently the FQDN does not get resolved to my [VPS IP] but to the loopback ip, and that connection gets refused. Does it maybe have something to do with that? esackbauer

funktionierbar

You’re right, I am kind of fishing in the dark here. Can you suggest where I should start reading?

Regarding this problem, I dont think I get what you suggest. I found out through docker network inspect mailcowdockerized_mailcow-network that mailcows gateway IP is 172.22.1.1, just as described in the documentation. Using it produces the error above(connection refused). How do I find out the IP of mailcows docker host? Do you mean my public facing IP of the VPN?

esackbauer

No, I mean the IP address of the operating system you installed docker on.
And it would help describing your setup, is it hosted in the cloud? Why do you think you need a VPN?

funktionierbar

aah, thats embarrassing, sorry. I meant to write VPS of course, yes, its on the cloud. Its an Ubuntu 22.04 LTS VPS with Contabo.

When I use the VPS’ IP as a host I also get “connection refused”.

esackbauer

Use port 25 and STARTTLS.

DocFraggle

Are you running a firewall on your docker host? ufw? Sounds like a firewall issue to me

funktionierbar

I thought so too, thats why I disabled ufw for now.

root@mail:~/# ufw status Status: inactive

I have to correct myself. When I try to use the VPS’ IP as the SMTP Host, I get this error in Vaultwardens logs:
SMTP error: Connection error: Connection error: error:0A000086:SSL routines:tls_post_process_server_certificate:certificate verify failed:../ssl/statem/statem_clnt.c:1889: (IP address mismatch)

Which I guess is adequate, since the systems IP is not part of the certificate.

Still, how can I find out, where the refusal of the connection comes from when I try to use the FQDN as the SMTP-host? The same command openssl s_client -connect mail.example.com:465 returns successfully when I try it from my local laptop, but gets refused when used from the vaultwarden-docker on the same VPS as mailcow:

803B6E3D677F0000:error:8000006F:system library:BIO_connect:Connection refused:../crypto/bio/bio_sock2.c:114:calling connect()
803B6E3D677F0000:error:10000067:BIO routines:BIO_connect:connect error:../crypto/bio/bio_sock2.c:116:
connect:errno=111

esackbauer

Why don’t you try port 25 and STARTTLS?

funktionierbar

When I openssl the VPNs’ IP on Port 25 from inside the vaultwarden docker, it returns sucessfully:

root@[dockerid]:/# openssl s_client -connect [VPN IP]:25
CONNECTED(00000003)
805BFC42CA7F0000:error:0A00010B:SSL routines:ssl3_get_record:wrong version number:../ssl/record/ssl3_record.c:354:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 5 bytes and written 297 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---

But when I try to set it as a host in vaultwarden it shows this in the logs when I send a test-mail:

[2024-09-30 08:36:12.700][request][INFO] POST /admin/test/smtp
[2024-09-30 08:36:12.978][vaultwarden::mail][ERROR] SMTP error: Connection error: Connection error: error:0A000086:SSL routines:tls_post_process_server_certificate:certificate verify failed:../ssl/statem/statem_clnt.c:1889: (IP address mismatch)
[2024-09-30 08:36:12.980][response][INFO] (test_smtp) POST /admin/test/smtp => 400 Bad Request

DocFraggle

funktionierbar root@[dockerid]:/# openssl s_client -connect [VPN IP]:25

For port 25 you have to use the “-starttls smtp” parameter!

openssl s_client -connect [VPN IP]:25 -starttls smtp

funktionierbar IP address mismatch

This is due to you entering the IP address in vaultwarden, which is not part of the certificate. So the certificate verification fails, I don’t know if you can disable it in vaultwarden

EDIT: a quick Google search presented this:

dani-garcia/vaultwardenblob/main/.env.template#L549

funktionierbar

I wrote what happened with STARTTLS and Port 25 up here:

funktionierbar

It seems I could not communicate what I wanted here. I didnt want to just make vaultwarden work, i wanted to understand and fix the underlying problem, as while vaultwarden does have an option to accept invalid certificates and is therefore able to use The VPNs IP as the SMTP host, Ghost and other dockers do not, and have the same problem.

I found another strange clue, when I nmap the localhost from inside the container I get the following result:

root@[docker id]:/# nmap -p 60-1000 localhost    
Starting Nmap 7.93 ( https://nmap.org ) at 2024-10-01 08:42 UTC
Nmap scan report for localhost (127.0.0.1)
Host is up (0.0000070s latency).
Other addresses for localhost (not scanned): ::1
Not shown: 940 closed tcp ports (reset)
PORT   STATE SERVICE
80/tcp open  http

Nmap done: 1 IP address (1 host up) scanned in 0.15 seconds

but when I nmap the VPN IP, the correct ports are accessible.

root@bc5670c8e7b4:/# nmap -p 60-1000 [VPN IP]
Starting Nmap 7.93 ( https://nmap.org ) at 2024-10-01 08:42 UTC
Nmap scan report for [VPN IP](128.0.64.168)
Host is up (0.0000070s latency).
Not shown: 929 closed tcp ports (reset)
PORT    STATE SERVICE
80/tcp  open  http
110/tcp open  pop3
143/tcp open  imap
443/tcp open  https
465/tcp open  smtps
587/tcp open  submission
993/tcp open  imaps
995/tcp open  pop3s

Nmap done: 1 IP address (1 host up) scanned in 0.14 seconds

I dont know why that is. Why are the hosts ports closed to requests from inside a docker?

But it led to to find a workaround myself. I found out the reason for my FQDN being resolved to localhost is that the FQDN was in my /etc/hosts file on the host, and removed it from there. Now my requests to FQDN dont get resolved to localhost but to the VPN IP and everything suddenly works.

Thank you for thinking with me.

esackbauer

funktionierbar Vaultwarden does even have an UI Option to accept invalid certificates

For me it works with “invalid” certificate (meaning the IP is not part of the SAN), I do not get errors (with the IP of mailcow configured in VaultWarden)

funktionierbar So apparently the FQDN does not get resolved to my [VPS IP] but to the loopback ip,

Therefore I have not configured a FQDN in Vaultwarden…

Can you please tell us when you will try port 25 and STARTTLS? Encryption is BTW completely unnecessary, because the whole sending will happen within the host itself.

esackbauer

Happy to hear that it worked finally.

funktionierbar I wrote what happened with STARTTLS and Port 25 up here:

Just for completeness: No, you did a TLS connection because you missed the -starttls parameter as DocFraggle pointed out.