Good point, thx! I really thought I was safe with my normal mailcow backups until now. But fiddlesticks, the mails in the backups are all encrypted! I’ll have a look at your document and backup my mailboxes locally unencrypted as well. I’ll wait until arm64 is stable, only then will I dare to update from my current version. Thank you all for your efforts!
EnglishARM64 Version
- Edited
@DocFraggle yeah… i’m afraid it don’t work.
It still cannot decrypt the mails. Even with lz4 enabled. So there is something different, as your Debian 12 Setup seems to work. I’ll try that doveadm command now to see if there is something different.
DerLinkman could you please update the nightly branch with the new dovecot image? Then I can have a look
DocFraggle Is updated!
DerLinkman OK, I really don’t get it… the installed dovecot-core version on my Ubuntu 22.04 machine is 1:2.3.16+dfsg1-3ubuntu2.2 and is perfectly able to decrypt the files with LZ4.
The dovecot version inside the container is 2.3.21 (47349e2482) and can’t decrypt it…
I guess this is your build configuration for dovecot?
> cat /usr/lib/dovecot/dovecot-config
DOVECOT_INSTALLED=yes
DOVECOT_CFLAGS="-std=gnu99 -Os -fstack-clash-protection -Wformat -Werror=format-security -fstack-protector-strong -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 "
DOVECOT_LIBS=""
DOVECOT_SSL_LIBS="-lssl -lcrypto"
DOVECOT_SQL_LIBS=""
DOVECOT_COMPRESS_LIBS=" -lz -lbz2 -llzma -llz4 -lzstd"
DOVECOT_LUA_LIBS="-L/usr/lib/lua5.3 -llua -lm"
DOVECOT_LUA_CFLAGS="-I/usr/include/lua5.3"
DOVECOT_BINARY_CFLAGS="-fPIE -DPIE"
DOVECOT_BINARY_LDFLAGS="-pie -Wl,-z -Wl,relro -Wl,-z -Wl,now"
LIBDOVECOT='-L/usr/lib/dovecot -ldovecot'
LIBDOVECOT_LOGIN='-ldovecot-login -lssl -lcrypto'
LIBDOVECOT_SQL=-ldovecot-sql
LIBDOVECOT_COMPRESS=-ldovecot-compression
LIBDOVECOT_LDA=-ldovecot-lda
LIBDOVECOT_STORAGE='-ldovecot-storage '
LIBDOVECOT_DSYNC=-ldovecot-dsync
LIBDOVECOT_LIBFTS=-ldovecot-fts
LIBDOVECOT_LUA=-ldovecot-lua
LIBDOVECOT_INCLUDE=-I/usr/include/dovecot
dovecot_pkgincludedir=/usr/include/dovecot
dovecot_pkglibdir=/usr/lib/dovecot
dovecot_pkglibexecdir=/usr/libexec/dovecot
dovecot_docdir=/usr/share/doc/dovecot
dovecot_moduledir=/usr/lib/dovecot
dovecot_statedir=/var/lib/dovecotDo you know eventually how to compare the build options with the Ubuntu package? Maybe there is something still missing?
We use the Alpine Builds they build so that is not my build exactly 
However i try dovecot on Debian 12 which had the same issue in the past (lz4 aside ).
- Edited
Debug output inside the container:
doveadm -D fs get compress lz4:1:crypt:private_key_path=/mail_crypt/ecprivkey.pem:public_key_path=/mail_crypt/ecpubkey.pem:posix:prefix=/ /var/vmail//var/vmail/asdökfjlaskdjfla/Maildir/.INBOX.! asdfasdfasdf/cur/1649346412.M610300P59299.fb225c99c952\,S\=1948\,W\=2008\:2\,S
Debug: Loading modules from directory: /usr/lib/dovecot/doveadm
Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: Error relocating /usr/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so: acl_rights_update_import: symbol not found (this is usually intentional, so just ignore this message)
Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: Error relocating /usr/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so: quota_get_resource: symbol not found (this is usually intentional, so just ignore this message)
Debug: Module loaded: /usr/lib/dovecot/doveadm/lib10_doveadm_sieve_plugin.so
Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: Error relocating /usr/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so: fts_backend_rescan: symbol not found (this is usually intentional, so just ignore this message)
Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: Error relocating /usr/lib/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so: mail_crypt_box_get_public_key: symbol not found (this is usually intentional, so just ignore this message)
2023-10-16 11:18:33 Debug: Loading modules from directory: /usr/lib/dovecot
2023-10-16 11:18:33 Debug: Module loaded: /usr/lib/dovecot/lib01_acl_plugin.so
2023-10-16 11:18:33 Debug: Module loaded: /usr/lib/dovecot/lib05_mail_crypt_acl_plugin.so
2023-10-16 11:18:33 Debug: Module loaded: /usr/lib/dovecot/lib10_mail_crypt_plugin.so
2023-10-16 11:18:33 Debug: Module loaded: /usr/lib/dovecot/lib10_quota_plugin.so
2023-10-16 11:18:33 Debug: Module loaded: /usr/lib/dovecot/lib15_notify_plugin.so
2023-10-16 11:18:33 Debug: Module loaded: /usr/lib/dovecot/lib20_fts_plugin.so
2023-10-16 11:18:33 Debug: Module loaded: /usr/lib/dovecot/lib20_listescape_plugin.so
2023-10-16 11:18:33 Debug: Module loaded: /usr/lib/dovecot/lib20_mail_log_plugin.so
2023-10-16 11:18:33 Debug: Module loaded: /usr/lib/dovecot/lib20_replication_plugin.so
2023-10-16 11:18:33 Debug: Module loaded: /usr/lib/dovecot/lib20_zlib_plugin.so
2023-10-16 11:18:33 Debug: Module loaded: /usr/lib/dovecot/lib21_fts_solr_plugin.so
2023-10-16 11:18:33 Debug: Loading modules from directory: /usr/lib/dovecot/doveadm
2023-10-16 11:18:33 Debug: Module loaded: /usr/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so
2023-10-16 11:18:33 Debug: Module loaded: /usr/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so
2023-10-16 11:18:33 Debug: Module loaded: /usr/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so
2023-10-16 11:18:33 Debug: Module loaded: /usr/lib/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so
2023-10-16 11:18:33 Error: read(/var/vmail/asdökfjlaskdjfla/Maildir/.INBOX.! asdfasdfasdf/cur/1649346412.M610300P59299.fb225c99c952,S=1948,W=2008:2,S) failed: Decryption error: no private key availablePerformed on the Ubuntu host system:
root@ubuntu-16gb-fsn1-1:/var/lib/docker/volumes/mailcowdockerized_vmail-vol-1/_data# doveadm -D fs get compress lz4:1:crypt:private_key_path=/var/lib/docker/volumes/mailcowdockerized_crypt-vol-1/_data/ecprivkey.pem:public_key_path=/var/lib/docker/volumes/mailcowdockerized_crypt-vol-1/_data/ecpubkey.pem:posix:prefix=./ asdfasdfasdf/asdfasdfasdf/Maildir/.INBOX.\!\ \ asdfasdfasdf/cur/1649346412.M610300P59299.fb225c99c952\,S\=1948\,W\=2008\:2\,S
Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm
Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_user_module (this is usually intentional, so just ignore this message)
Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: quota_user_module (this is usually intentional, so just ignore this message)
Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore this message)
Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_user_get_language_list (this is usually intentional, so just ignore this message)
Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol: mail_crypt_box_get_pvt_digests (this is usually intentional, so just ignore this message)
Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm
Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_user_module (this is usually intentional, so just ignore this message)
Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: quota_user_module (this is usually intentional, so just ignore this message)
Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore this message)
Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_user_get_language_list (this is usually intentional, so just ignore this message)
Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol: mail_crypt_box_get_pvt_digests (this is usually intentional, so just ignore this message)
Oct 16 09:19:41 Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm
Oct 16 09:19:41 Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_user_module (this is usually intentional, so just ignore this message)
Oct 16 09:19:41 Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: quota_user_module (this is usually intentional, so just ignore this message)
Oct 16 09:19:41 Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore this message)
Oct 16 09:19:41 Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_user_get_language_list (this is usually intentional, so just ignore this message)
Oct 16 09:19:41 Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol: mail_crypt_box_get_pvt_digests (this is usually intentional, so just ignore this message)
From: "asdfasdf asdfasdfasdf" <asdfasdf@web.de>
To: "adsfasdf asdfasdf"
Subject: Muffins
...[unknown] My home server is running on Debian 12 as well, with dovecot-core 1:2.3.19.1+dfsg1-2.1 it’s working as well for me
Okay… very interesting. I’ve just switched the Dovecot Image to Debian Bookworm as a base and now it can decrypt the mails even on ARM64 without issues. If you want to test @DocFraggle the image is called mailcow/dovecot:nightly-devel to confirm my changes.
Yes, I can confirm that doveadm can decompress and decrypt the mails now. Sogo isn’t able to login to dovecot anymore, but maybe something’s missing regarding with the bookworm image.
DocFraggle I’ve opened up a ticket at Alpines so let’s see what they say.
However i think we might compile dovecot ourself then on Debian 11/12 to ensure it’s still working as expected.
- Edited
BTW, while debugging I read in the dovecot documentation that it is no problem to have Maildir files with different compression types, it will work out of the box. So no problem to switch the compression from LZ4 to zstd in the future, the problem was just the odd alpine dovecot image…
When this plugin is loaded Dovecot can read both compressed and uncompressed files from Maildir. The files within a Maildir can use any supported compression algorithm (e.g., some can be compressed using gzip, while others are compressed using zstd). The algorithm is detected by reading the first few bytes from the file and figuring out if it’s a valid gzip or bzip2 header. The file name doesn’t matter.
But, of course, this should be tested thoroughly
DocFraggle Yeah exactly i remembered this so i switched it back then as Alpine had no lz4 support during the inital tests. Only zstd got added after my request first.
However it is related something which has changed from Alpine 3.16 to 3.18.
I just found something very, very, very, very interesting:
debian/patches/Support-openssl-3.0.patch · master · Debian / dovecot · GitLab GitLab
This is a patch to support openssl 3.0 from last year. At this time Debian 12 was in development which ships OpenSSL3.0 and a running Repo Installation of Dovecot.
- Edited
DerLinkman but it seems that it was fixed in the 2.3.19 sources according to
#996273 - dovecot: FTBFS with OpenSSL 3.0 - Debian Bug report logs bugs.debian.org
And the version in Alpine is 2.3.21
Ah it was fixed in the DEBIAN source package, not the dovecot sources. Yes, worth a try!
DocFraggle Yes but only for debian! Debian creates their own patches.
You can actually see that the Dovecot devs are/were working at this too here:
History for src/lib-dcrypt/dcrypt-openssl3.c - dovecot/core GitHub
In the current stable versions of Dovecot there is only one dcrypt-openssl.c file not two like in the main (dev) branch.
(See core/src/lib-dcrypt at release-2.3.21 · dovecot/core and GitHub core/src/lib-dcrypt at main · dovecot/core) GitHub
Ubuntu also make their own Patches so that is also the reason why it is working with the Ubuntu Repo packages.
Well, I thought I could quickly try to build an alpine dovecot image with the dovecot main branch, but I would need an Alpine Linux OS to do that 
that’s too much for the time I have to spare currently 
Anyways, is there a specific reason why you want to switch dovecot from the Debian image over to the Alpine image?
DocFraggle Yes indeed. Alpine is supporting ARM64 better then Debian at least for Dovecot. Yes their repo version is also ARM64 compatible but it makes the implementation for newer features harder as they don’t the latest versions.
With Dovecot 2.4 (or the current master state in git) there will change a lot which is causing mailcow to not even boot up. So yeah that’s a bit shitty. It has to be done in the future…
6 days later
- Edited
Guys! Good news! My approach to Alpine has been submitted, they merged the patch!
main/dovecot: fix openssl 3 support (815bb154) · Commits · alpine / aports · GitLab GitLab
Now it only have to work 
It works indeed! mailcow Team 
Alpine Team
DerLinkman Great news 
I just deleted my ARM VM yesterday 
but you already tested it I assume from your last line above 
Yep it works. When the packages will be installed without a quirky stuff inside the docker image then i’ll probably push the updated Image so anybody can test.
a month later
Does this mean that a migration from x86 to Arm should now be possible?
13 days later
I really don’t want to be a nuisance, but is there anything new yet?
If you want to use it in a production environment I would definitely wait until @DerLinkman published a stable version
Ganzjahresgriller I’m waiting on Alpine to release 3.19 stable which includes the Dovecot fixes.
I won’t merge anything which is using Edge packages like the current nightly Version of Dovecot does. It needs to be stable at least OS wise!
DerLinkman Which got released today \o/
No one is typing