Is this a bug? Underlying OS: debian 12, up-to-date mailcow: latest version docker and docker compose: debian repos, latest version workstation: windows 10 expected behaviour: emails composed in a desktop email client with utf8 encoding should be processed with german umlaute (äöüß) through mailcow. Tests: 1. sogo build-in of mailcow (NOT a desktop client) --> working correct 2. email composed in thunderbird 115.4.1 (64-Bit) --> not working 3. email composed in betterbird --> not working 4. email composed in sylpheed --> not working Using system wide footer is working with sogo, but not working (again umlaute) using a email desktop client. Email sent through a different mail server (nethserver) with same clients are formatted as expected. Please advice where to investigate.

Follow up while digging around: 1. I had to change my fritzbox 6490 cable in bridge mode. Exposed host for the asus router behind was not enough. 2. Now the asus shows a WAN IP, which in (real) bridge mode of the fritzbox is a different one as shown in the fritzbox. The WAN IP shown in the asus is to be set in the DNS settings at the hoster. 3. The laptop is configured as DMZ in the asus. 4. On the laptop I installed nginx as reversed proxy. I took the example config from mailcow and changed it to my domain. 5. I started over with a new mailcow-docker installation. 6. Unbound container was unhealthy. As the asus is running with unbound, I tried to point the forward zone in unbound.conf to the IP of the asus. That didn't work. Unbound container showed healthy, but error during the acme challenge occurred. I changed the IP to 1.1.1.1. That worked. The challenge completed. 7. I created a domain and a test user. 8. I took the dkim key to the DNS (as well as the other settings to make). 9. I tested from sogo an email to email-tester composed in html. So far o.k., dkim passes, not a 10/10 because auf rDNS setting. This was to expect. 10. Changed sogo to plain text, email test also o.k. 11. Tried thunderbird - email tests also o.k. 12. Did a backup. Well - it is was not so easy to track this down. Still some questions open. But maybe it helps someone. I'll keep this running and testing at home for a while. If the mailcow runs smooth, I'll migrate my email servers in business. And will donate for this great work. regards, stefan

> @"stefan21"#p11432 mailcow: latest version Does that mean nightly version? Or latest stable version 2023-10a? What is the receiving side using?

Thank you for taking time to look into. It's the stable version 2023-10a. The clients are configured to use the FQHN in the LAN as mail server. For receiving and sending email. Nothing special, just how any desktop email client would be configured to use an internal email server in the LAN. What do you mean with the receiving side?

Logs from dovecot-mailcow and postfix-mailcow showing nothing special (to me). @"[deleted]"#2109 If it's helpful I'll send you an email to any account you name. @"[deleted]"#2109 If it's helpful I'll send you an email to any account you name. Sorry for double post. The browser requested a refresh...

> @"esackbauer"#p11437 What is the receiving side using? Do you mean where I sent the email to? 1. kabelbw.de 2. mailbox.org

During several investigations I'm facing another problem with the DKIM key. No matter if composing email form build-in sogo or any desktop email client, any email I send does not pass the DKIM check. Messages are: User-Agent: Mozilla Thunderbird: dkim=fail (2048-bit key) reason="fail (message has been altered)" or: User-Agent: SOGoMail 5.9.0 dkim=fail (2048-bit key) reason="fail (message has been altered)" Hmm. Don't know this from my other mail servers...? What's wrong?

Here's an email from nethserver: dkim=pass (2048-bit key) From: =?utf-8?q?x=2E_y?= Content-Type: text/plain; charset="utf-8" Reply-To: x@y.de X-Forward: 10.0.0.3

More testing: Now from my laptop, OS archlinux, but NOT in a VM. I should had make this clear before: all tests above have been made behind a opnsense in a VM hosted from proxmox, no firewall on proxmox enabled. dkim=pass (2048-bit key; unprotected) header.d=x.de header.i=@x.de header.a=rsa-sha256 header.s=dkim header.b=wO+Fa4dv; dkim-atps=neutral FROM: =?utf-8?q?x=2E_y?= text/plain; charset="utf-8" quoted-printable User-Agent: SOGoMail 5.9.0 German umlaute are o.k. Now from an email client, still my laptop: dkim=pass (2048-bit key; unprotected) header.d=x.de header.i=@x.de header.a=rsa-sha256 header.s=dkim header.b=dGLkfmta; dkim-atps=neutral FROM: "x" text/plain; charset=UTF-8; format=flowed User-Agent: Mozilla Thunderbird German umlaute are o.k.

Resume: In a VM, debian 12, hosted on proxmox, no firewall on proxmox, VM in a company LAN behind OPNsense, german umlaute are not processed from mailcow-dockerized, and DKIM is false. Receiving and sending emails is working. On the OPNsense no reverse proxy is configured. Let's encrypt certs are handled by acme-plugin from the OPNsense and are copied via cron job to the VM. Access to mailcow-dockerized via FQHN in LAN is working flawless. Only open port(s) redirected from LAN to WAN are 80 and 443. The mailcow is not intended to be accessable from WAN. Only via wireguard VPN. From a laptop at home, on top of archlinux, no issues. As I intend to substitute a company email server (nethserver 7) with mailcow-dockerized, I need help where to investigate. Any help would be greatly appreciated.

I have no idea what is happening, but I never had any problems that you have. My guess is you are at some point not following the prerequisites or installation docs. Limiting outgoing to only 80 and 443 is bad, because unbound cannot do DNS resolving then. DKIM is reliant on DNS records, which might need some time to sync across all DNS servers out there. So try it later again.

Every DNS (port 53) request from the VM with mailcow (the whole LAN) is redirected to the OPNsense unbound. BTW unbound-mailcow shows healthy status. And port 53 is open. For email the only open port is 25. All other ports (as in the docs described) are only allowed in the LAN. The mailserver is meant for internal access. No public access at all. May I ask if you have mailcow-dockerized running in a VM? If so, qemu, virtualbox, proxmox, ...? What host OS? For the prerequisites, what could be wrong? I mean, on my laptop at home I did the same as in the VM. Except that at home the host in archlinux and not a VM based on debian 12. Well, I can try with another host OS for the VM. Have you got a special recommendation?

encoding from different mail clients using mailcow as mail server

esackbauer

With that little information, we can only guess. Probably you are mixing up DKIM selectors, and/or not waiting until updated DNS records have been published on every DNS server out there.

stefan21

Getting closer. Your guess might lead in the right direction. Thank you for pointing to the records.

I did a dkim check in another tool. mxtoolbox does indeed show a different record as dkimvalidator.com. And here we go - mail-tester.com and mxtoolbox are identical. And correct.

So, for the DKIM it’s a dkimvalidator and cache/update problem. Thank you so far.

Now I will try to find the error with the umlaute problem. I’ll keep investigating.

stefan21

@esackbauer

Could you tell me where exactly are the dkim keys in mailcow stored? Files or database? And I’d like to know how to get there.

More testing with emails to a vodafone account:

mx5.vodafonemail.de/4SV90B37Vyz9ryY;
dkim=fail reason=“signature verification failed” (2048-bit key; unprotected) header.d=x.de header.i=@x.de header.a=rsa-sha256 header.s=dkim header.b=cVJitv3G;
dkim-atps=neutral

I did a diff -s file1 <– dkim from mailcow file2 <– dkim copied to DNS. They are identical.

Could it be, that if restoring a mailcow backup, the keys are messed up? I’d like to check the keys (public and private) before restore and after restore.

esackbauer

stefan21 Could it be, that if restoring a mailcow backup, the keys are messed up?

That could be. Probably they are stored in the Redis DB. I remember a thread in this forum where they discussed problems with the DKIM keys.

stefan21

Well, I’d like to check this. What’s the best approach to access the DB?

BTW here’s the email to the vodafone account:

"
test
–=20
Mit freundlichen Gr=C3=BC=C3=9Fen,

Mit freundlichen Gr=C3=BC=C3=9Fen,

=C3=B6=C3=A4=C3=BC=C3=9F

Umlaute in signature and domain footer… as already seen. Of course with a failed dkim key after a restore. To verify the suspicion I’ll wipe/delete the mailcow install on my laptop and start over with a new, clean install. No restores, no imports, nothing. Means from scratch. Will report.

esackbauer

stefan21
Now I am thinking that the Umlaute conversion is the reason why the DKIM fails. Something is changing the character encoding in transit and changing the “checksum”. Maybe the DKIM keys are fine.

I never tried to access the Redis DB, don’t know.

stefan21

esackbauer Now I am thinking that the Umlaute conversion is the reason why the DKIM fails. Something is changing the character encoding in transit and changing the “checksum”.

Let me start over with a fresh clean install. I’ll create a domain (same I use for tests), a user, will copy the new dkim key to DNS, nothing more. I’ll do a “vanilla” backup. First email will be with no umlaute. Second email will have umlaute. Third email will have signature and Umlaute. Forth one will have the signature and a system wide footer with umlaute.

I’ll report.

stefan21

I did the tests. Here are the results:

Sending an email from SOGO in HTML format, no matter what language, with signature and footer, with german umlaute, works correct. The dkim test pass:
mx5.vodafonemail.de/4SVGtc1n9xz9rxk;
dkim=pass (2048-bit key; unprotected) header.d=y.de header.i=@y.de header.a=rsa-sha256 header.s=dkim header.b=An7pjvp1;
dkim-atps=neutral
Sending an email from SOGO in plain text format, no matter what language, with signature and footer, with german umlaute, will not work. The dkim test fails.
mx5.vodafonemail.de/4SVH2f6TNFz6vhw;
dkim=fail reason=“signature verification failed” (2048-bit key; unprotected) header.d=y.de header.i=@y.de header.a=rsa-sha256 header.s=dkim header.b=QWmUDGNz;
dkim-atps=neutral

Reverting in SOGO back to html text will pass the dkim test (again).

Sending an email form email-client thunderbird in HTML format, no matter what language, with signature and footer, with german umlaute, will not work. dkim test fails. The layout of the email looks like:
"
umlaute: =C3=B6=C3=A4=C3=BC=C3=9F

–=20
Mit freundlichen Gr=C3=BC=C3=9Fen,

Sending emails from thunderbird will always fail the dkim test. No matter if plain text or html.

To resume:

sending emails from SOGO in html containing german umlaute, works. DKIM will pass.
sending emails from SOGO in plain text containing german umlaute leads to DKIM fails.
sending emails from email-client thunderbird will always fail the dkim test. And the german umlaute are in a wrong format.

As we need thunderbird out of several reasons, at this moment with the results above, I cannot migrate my email servers to mailcow. We send, and will this probably do also in future, emails only in plain text format. And we need dkim, dmarc and spf - valid.

Besides the question of restoring a backup to different box with another underlying OS and the decryption of the emails, while doing a restore of all items including the keys. But this point for the moment is less important, it could be solved in another way.

How to proceed?

esackbauer

With your great write-up of your findings you should open an issue on github:
mailcow/mailcow-dockerized/issues

DocFraggle

Is this a Vodafone-specific issue? I just sent some plaintext mails containing “üöäÜÖÄß” via Sogo, and also tested it via https://dkimvalidator.com/, the DKIM test passed everywhere

stefan21

Can’t tell. Sending emails composed in sogo with plain text and umlaute to i.e. domain hosted at allinkl brings up the same result:

Authentication-Results: q.kasserver.com;
dkim=fail reason=“signature verification failed” (2048-bit key; unprotected) header.d=x.de header.i=@x.de header.a=rsa-sha256 header.s=dkim header.b=JevnIlCW;
dkim-atps=neutral

I’m sending from a socalled pool IP: IP-x.x.x.x-um38.pools.vodafone-ip.de. Means not a static IP. But if this causes the failure, then the failure should also be while sending in HTML. As I pointed out, DKIM passes with email sent from the same (dynamic) IP(4).

IMVHO I don’t think so.

DocFraggle

stefan21 OK, according to your header examples from above it was always mx5.vodafonemail.de which had the problem with your DKIM, so I thought it’s Vodafone specific.
But, as I wrote above, no DKIM problems here sending plaintext mails with Sogo

stefan21

@DocFraggle

If you like, you could tell me one of your email adresses (is in this forum a pm function?). I’ll send you an email. Let’s see how this works.

DocFraggle

No PM option here… private discussions are blocked as well…
Did you try the https://dkimvalidator.com/ ?

stefan21

DocFraggle
@esackbauer

Here’s the result from the dkimvalidator:

#######
Original Message:

Received: from mail.z.de (ip-109-192-q-q.um38.pools.vodafone-ip.de [109.192.q.q])
by relay-1.us-west-2.relay-prod (Postfix) with ESMTPS id BB5E628E53
for RO3ER2LYBB7X5n@dkimvalidator.com; Fri, 17 Nov 2023 11:44:00 +0000 (UTC)
Received: from [127.0.0.1] (localhost [127.0.0.1]) by localhost (Mailerdaemon) with ESMTPA id 203793C009C
for RO3ER2LYBB7X5n@dkimvalidator.com; Fri, 17 Nov 2023 12:43:54 +0100 (CET)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=z.de; s=dkim;
t=1700221435;
h=from:reply-to:subject:date:message-id:to:mime-version:content-type:
content-transfer-encoding; bh=lZj4ZrwRyEiLyclvgIilptURN6KHsH85iH83mFdpfT0=;
b=WOezYpMCyhYml9qghfu2Iikoe9pM7Rbdk1G3L0aTEsLcknLE13xr2kza07cc6y+/cQTBDz
vLwzHs2pUqdohgDrFvuvUnGzKa8SD/3rLDDImyor37oWIYCe2f/GTjr6H18Gfq3mzOc/wq
Fd45OF0rbl/ACT7bmNfpWFOuBLyIpdhadysT6ggYT+dSaXYrFDTnCGC/Ux4tYaD+Uuy/r2
A09tU6tqTNFCk3OKc6QdelA1D5663ZQI9y58nF1v8MZ2BCl1MgtaPdaEmPQKFsjNeR9plG
GBokcfLamSFqHDajTTphfw4QGAWMC4RavRs508MPfd5Hl0YFhjobYFZ7xbsLug==
From: “X Y” x.y@z.de
Content-Type: text/plain; charset=“utf-8”
Reply-To: x.y@z.de
Date: Fri, 17 Nov 2023 12:43:54 +0100
To: RO3ER2LYBB7X5n@dkimvalidator.com
MIME-Version: 1.0
Message-ID: <42-65575200-3-560d3d80@48021463>
Subject: =?utf-8?q?T=C3=84SCHT?=
User-Agent: SOGoMail 5.9.0
Content-Transfer-Encoding: quoted-printable
X-Last-TLS-Session-Version: None

Sogo, plain text, signatur mit umlauten, footer mit umlauten

–=20
signature

plain text
=C3=B6=C3=A4=C3=BC=C3=9F

plain footer
=C3=B6=C3=A4=C3=BC=C3=9F

#######
DKIM Information:

DKIM Signature

Message contains this DKIM Signature:
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=z.de; s=dkim;
t=1700221435;
h=from:reply-to:subject:date:message-id:to:mime-version:content-type:
content-transfer-encoding; bh=lZj4ZrwRyEiLyclvgIilptURN6KHsH85iH83mFdpfT0=;
b=WOezYpMCyhYml9qghfu2Iikoe9pM7Rbdk1G3L0aTEsLcknLE13xr2kza07cc6y+/cQTBDz
vLwzHs2pUqdohgDrFvuvUnGzKa8SD/3rLDDImyor37oWIYCe2f/GTjr6H18Gfq3mzOc/wq
Fd45OF0rbl/ACT7bmNfpWFOuBLyIpdhadysT6ggYT+dSaXYrFDTnCGC/Ux4tYaD+Uuy/r2
A09tU6tqTNFCk3OKc6QdelA1D5663ZQI9y58nF1v8MZ2BCl1MgtaPdaEmPQKFsjNeR9plG
GBokcfLamSFqHDajTTphfw4QGAWMC4RavRs508MPfd5Hl0YFhjobYFZ7xbsLug==

Signature Information:
v= Version: 1
a= Algorithm: rsa-sha256
c= Method: relaxed/relaxed
d= Domain: z.de
s= Selector: dkim
q= Protocol:
bh= lZj4ZrwRyEiLyclvgIilptURN6KHsH85iH83mFdpfT0=
h= Signed Headers: from:reply-to:subject:date:message-id:to:mime-version:content-type:
content-transfer-encoding
b= Data: WOezYpMCyhYml9qghfu2Iikoe9pM7Rbdk1G3L0aTEsLcknLE13xr2kza07cc6y+/cQTBDz
vLwzHs2pUqdohgDrFvuvUnGzKa8SD/3rLDDImyor37oWIYCe2f/GTjr6H18Gfq3mzOc/wq
Fd45OF0rbl/ACT7bmNfpWFOuBLyIpdhadysT6ggYT+dSaXYrFDTnCGC/Ux4tYaD+Uuy/r2
A09tU6tqTNFCk3OKc6QdelA1D5663ZQI9y58nF1v8MZ2BCl1MgtaPdaEmPQKFsjNeR9plG
GBokcfLamSFqHDajTTphfw4QGAWMC4RavRs508MPfd5Hl0YFhjobYFZ7xbsLug==
Public Key DNS Lookup

Building DNS Query for dkim._domainkey.z.de
Retrieved this publickey from DNS: v=DKIM1;k=rsa;t=s;s=email;p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAtHkJ1zfpaCrbEr5y4riJc82jyNtNheQrREUuH1dhKOwfhyIeqHtAWPir5sdkn418FJ8j4Zu1N7g0xqQqvceXwllO2xik+tLAsWMYk2t7XvD7IWM9D1awaC9QgTPXk7v9mGEjh1HSvrxyBr7Fa8cJP56Ujhda7xpCw05AZTJL7Nu3hgnc6dEAotF1qEIpof6XJ5XW0zzd3cxvyN5TE12ewSYE6GblgtQjTNYyGaW2l4o8Kxpw6Qha1XowoDq/Eyv2PFyPbUg8i3QXLxBaGJQ0U+j8Tk0T1iay1AukZAdCvnPa8UCrc9CkKQ73TG+nd9OL4zZdwSVWWYTV8MzjwgABowIDAQAB
Validating Signature

result = fail
Details: message has been altered

#######
SPF Information:

Using this information that I obtained from the headers

Helo Address = mail.z.de
From Address = x.x@z.de
From IP = 109.192.q.q
SPF Record Lookup

Looking up TXT SPF record for z.de
Found the following namesevers for z.de: ns5.kasserver.com ns6.kasserver.com
Retrieved this SPF Record: zone updated 20210630 (TTL = 167)
using authoritative server (ns5.kasserver.com) directly for SPF Check
Result: pass (Mechanism ‘mx’ matched)

Result code: pass
Local Explanation: z.de: 109.192.q.q is authorized to use ‘x.y@z.de’ in ‘mfrom’ identity (mechanism ‘mx’ matched)
spf_header = Received-SPF: pass (z.de: 109.192.q.q is authorized to use ‘x.y@z.de’ in ‘mfrom’ identity (mechanism ‘mx’ matched)) receiver=ip-172-31-52-154.ec2.internal; identity=mailfrom; envelope-from=“x.y@z.de”; helo=mail.z.de; client-ip=109.192.q.q

Same test, but as html text in sogo:

#######
From: “x y” x.y@z.de
Content-Type: multipart/alternative; boundary=“—-==-OpenGroupware_org_NGMime-65-1700222345.461701-0——”
Reply-To: x.y@z.de
Date: Fri, 17 Nov 2023 12:59:05 +0100
To: iBsUWO3nrKugbZ@dkimvalidator.com
MIME-Version: 1.0
Message-ID: <41-65575580-5-677bcc00@203231989>
Subject: =?utf-8?q?T=C3=84SCHT?=
User-Agent: SOGoMail 5.9.0
X-Last-TLS-Session-Version: None

——==-OpenGroupware_org_NGMime-65-1700222345.461701-0——
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

Sogo, html text, signatur mit umlauten, footer mit umlauten

–=C2=A0
signature plain text =C3=B6=C3=A4=C3=BC=C3=9F

plain footer
=C3=B6=C3=A4=C3=BC=C3=9F

——==-OpenGroupware_org_NGMime-65-1700222345.461701-0——
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: quoted-printable

<html>Sogo, html text, signatur mit umlauten, footer mit umlauten<br /><br />–=
 <br />signature plain text =C3=B6=C3=A4=C3=BC=C3=9F</html>

html footer
=C3=B6=C3=A4=C3=BC=C3=9F

——==-OpenGroupware_org_NGMime-65-1700222345.461701-0——–

#######
Public Key DNS Lookup

result = pass
Details:

But - bad format with umlaute.

Anyway - we need as email-client thunderbird.

As I already wrote, if it helps I have other linux mailservers in my LAN(s) (also with sogo, not used), I can provide configs and versions. The used programs are AFAIK pretty much the same.

If there’s interest to track this down, I’d suggest to do this not in this forum. It’s better done via email.

esackbauer

Well, since yesterday I have one user who also has encoding troubles. User is on iPad with ActiveSync configured. User can send via SOGo without problems, but from Apples integrated mail client outgoing mails are totally unreadable.
Had no time to investigate further yet.

DocFraggle

esackbauer OK, I just checked this on my iPad and my ActiveSync account, works as usual with iPadOS 16.7.2 and after updating to iPadOS 17.1.1 in my case

esackbauer

Have deleted and reinstalled the Activesync account with my Ipad user. Problem persists. But only with that iPad user.
Installed the mobileconfig IMAP profile and everything as back to normal.

DocFraggle

Details: message has been altered

Strange, is there anything in-between your mailcow and your local gateway to the internet?
The validator had no problems on my side with plain text Umlaut mails from Sogo…

esackbauer

DocFraggle trange, is there anything in-between your mailcow and your local gateway to the internet

That is also my suspicion. Some transparent filters on the firewall I would guess.

« Previous Page Next Page »