MichaelThwaite

Good point, thx! I really thought I was safe with my normal mailcow backups until now. But fiddlesticks, the mails in the backups are all encrypted! I’ll have a look at your document and backup my mailboxes locally unencrypted as well. I’ll wait until arm64 is stable, only then will I dare to update from my current version. Thank you all for your efforts!

@DocFraggle yeah… i’m afraid it don’t work.

It still cannot decrypt the mails. Even with lz4 enabled. So there is something different, as your Debian 12 Setup seems to work. I’ll try that doveadm command now to see if there is something different.

    DerLinkman could you please update the nightly branch with the new dovecot image? Then I can have a look

      DerLinkman OK, I really don’t get it… the installed dovecot-core version on my Ubuntu 22.04 machine is 1:2.3.16+dfsg1-3ubuntu2.2 and is perfectly able to decrypt the files with LZ4.
      The dovecot version inside the container is 2.3.21 (47349e2482) and can’t decrypt it…

      I guess this is your build configuration for dovecot?

      > cat /usr/lib/dovecot/dovecot-config 
      DOVECOT_INSTALLED=yes
      DOVECOT_CFLAGS="-std=gnu99 -Os -fstack-clash-protection -Wformat -Werror=format-security -fstack-protector-strong -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2  "
      DOVECOT_LIBS=""
      DOVECOT_SSL_LIBS="-lssl -lcrypto"
      DOVECOT_SQL_LIBS=""
      DOVECOT_COMPRESS_LIBS=" -lz -lbz2 -llzma -llz4 -lzstd"
      DOVECOT_LUA_LIBS="-L/usr/lib/lua5.3 -llua -lm"
      DOVECOT_LUA_CFLAGS="-I/usr/include/lua5.3"
      DOVECOT_BINARY_CFLAGS="-fPIE -DPIE"
      DOVECOT_BINARY_LDFLAGS="-pie -Wl,-z -Wl,relro -Wl,-z -Wl,now"
      
      LIBDOVECOT='-L/usr/lib/dovecot -ldovecot'
      LIBDOVECOT_LOGIN='-ldovecot-login -lssl -lcrypto'
      LIBDOVECOT_SQL=-ldovecot-sql
      LIBDOVECOT_COMPRESS=-ldovecot-compression
      LIBDOVECOT_LDA=-ldovecot-lda
      LIBDOVECOT_STORAGE='-ldovecot-storage '
      LIBDOVECOT_DSYNC=-ldovecot-dsync
      LIBDOVECOT_LIBFTS=-ldovecot-fts
      LIBDOVECOT_LUA=-ldovecot-lua
      
      
      LIBDOVECOT_INCLUDE=-I/usr/include/dovecot
      
      dovecot_pkgincludedir=/usr/include/dovecot
      dovecot_pkglibdir=/usr/lib/dovecot
      dovecot_pkglibexecdir=/usr/libexec/dovecot
      dovecot_docdir=/usr/share/doc/dovecot
      dovecot_moduledir=/usr/lib/dovecot
      dovecot_statedir=/var/lib/dovecot

      Do you know eventually how to compare the build options with the Ubuntu package? Maybe there is something still missing?

      We use the Alpine Builds they build so that is not my build exactly 😃

      However i try dovecot on Debian 12 which had the same issue in the past (lz4 aside 😃).

      Debug output inside the container:

      doveadm -D fs get compress lz4:1:crypt:private_key_path=/mail_crypt/ecprivkey.pem:public_key_path=/mail_crypt/ecpubkey.pem:posix:prefix=/ /var/vmail//var/vmail/asdökfjlaskdjfla/Maildir/.INBOX.!  asdfasdfasdf/cur/1649346412.M610300P59299.fb225c99c952\,S\=1948\,W\=2008\:2\,S 
      Debug: Loading modules from directory: /usr/lib/dovecot/doveadm
      Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: Error relocating /usr/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so: acl_rights_update_import: symbol not found (this is usually intentional, so just ignore this message)
      Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: Error relocating /usr/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so: quota_get_resource: symbol not found (this is usually intentional, so just ignore this message)
      Debug: Module loaded: /usr/lib/dovecot/doveadm/lib10_doveadm_sieve_plugin.so
      Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: Error relocating /usr/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so: fts_backend_rescan: symbol not found (this is usually intentional, so just ignore this message)
      Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: Error relocating /usr/lib/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so: mail_crypt_box_get_public_key: symbol not found (this is usually intentional, so just ignore this message)
      2023-10-16 11:18:33 Debug: Loading modules from directory: /usr/lib/dovecot
      2023-10-16 11:18:33 Debug: Module loaded: /usr/lib/dovecot/lib01_acl_plugin.so
      2023-10-16 11:18:33 Debug: Module loaded: /usr/lib/dovecot/lib05_mail_crypt_acl_plugin.so
      2023-10-16 11:18:33 Debug: Module loaded: /usr/lib/dovecot/lib10_mail_crypt_plugin.so
      2023-10-16 11:18:33 Debug: Module loaded: /usr/lib/dovecot/lib10_quota_plugin.so
      2023-10-16 11:18:33 Debug: Module loaded: /usr/lib/dovecot/lib15_notify_plugin.so
      2023-10-16 11:18:33 Debug: Module loaded: /usr/lib/dovecot/lib20_fts_plugin.so
      2023-10-16 11:18:33 Debug: Module loaded: /usr/lib/dovecot/lib20_listescape_plugin.so
      2023-10-16 11:18:33 Debug: Module loaded: /usr/lib/dovecot/lib20_mail_log_plugin.so
      2023-10-16 11:18:33 Debug: Module loaded: /usr/lib/dovecot/lib20_replication_plugin.so
      2023-10-16 11:18:33 Debug: Module loaded: /usr/lib/dovecot/lib20_zlib_plugin.so
      2023-10-16 11:18:33 Debug: Module loaded: /usr/lib/dovecot/lib21_fts_solr_plugin.so
      2023-10-16 11:18:33 Debug: Loading modules from directory: /usr/lib/dovecot/doveadm
      2023-10-16 11:18:33 Debug: Module loaded: /usr/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so
      2023-10-16 11:18:33 Debug: Module loaded: /usr/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so
      2023-10-16 11:18:33 Debug: Module loaded: /usr/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so
      2023-10-16 11:18:33 Debug: Module loaded: /usr/lib/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so
      2023-10-16 11:18:33 Error: read(/var/vmail/asdökfjlaskdjfla/Maildir/.INBOX.!  asdfasdfasdf/cur/1649346412.M610300P59299.fb225c99c952,S=1948,W=2008:2,S) failed: Decryption error: no private key available

      Performed on the Ubuntu host system:

      root@ubuntu-16gb-fsn1-1:/var/lib/docker/volumes/mailcowdockerized_vmail-vol-1/_data# doveadm -D fs get compress lz4:1:crypt:private_key_path=/var/lib/docker/volumes/mailcowdockerized_crypt-vol-1/_data/ecprivkey.pem:public_key_path=/var/lib/docker/volumes/mailcowdockerized_crypt-vol-1/_data/ecpubkey.pem:posix:prefix=./ asdfasdfasdf/asdfasdfasdf/Maildir/.INBOX.\!\ \ asdfasdfasdf/cur/1649346412.M610300P59299.fb225c99c952\,S\=1948\,W\=2008\:2\,S
      Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm
      Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_user_module (this is usually intentional, so just ignore this message)
      Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: quota_user_module (this is usually intentional, so just ignore this message)
      Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore this message)
      Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_user_get_language_list (this is usually intentional, so just ignore this message)
      Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol: mail_crypt_box_get_pvt_digests (this is usually intentional, so just ignore this message)
      Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm
      Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_user_module (this is usually intentional, so just ignore this message)
      Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: quota_user_module (this is usually intentional, so just ignore this message)
      Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore this message)
      Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_user_get_language_list (this is usually intentional, so just ignore this message)
      Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol: mail_crypt_box_get_pvt_digests (this is usually intentional, so just ignore this message)
      Oct 16 09:19:41 Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm
      Oct 16 09:19:41 Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_user_module (this is usually intentional, so just ignore this message)
      Oct 16 09:19:41 Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: quota_user_module (this is usually intentional, so just ignore this message)
      Oct 16 09:19:41 Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore this message)
      Oct 16 09:19:41 Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_user_get_language_list (this is usually intentional, so just ignore this message)
      Oct 16 09:19:41 Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol: mail_crypt_box_get_pvt_digests (this is usually intentional, so just ignore this message)
      From: "asdfasdf asdfasdfasdf" <asdfasdf@web.de>
      To: "adsfasdf asdfasdf"
      Subject: Muffins
      ...

      [unknown] My home server is running on Debian 12 as well, with dovecot-core 1:2.3.19.1+dfsg1-2.1 it’s working as well for me

      Okay… very interesting. I’ve just switched the Dovecot Image to Debian Bookworm as a base and now it can decrypt the mails even on ARM64 without issues. If you want to test @DocFraggle the image is called mailcow/dovecot:nightly-devel to confirm my changes.

      Yes, I can confirm that doveadm can decompress and decrypt the mails now. Sogo isn’t able to login to dovecot anymore, but maybe something’s missing regarding with the bookworm image.

        DocFraggle I’ve opened up a ticket at Alpines so let’s see what they say.

        However i think we might compile dovecot ourself then on Debian 11/12 to ensure it’s still working as expected.

        BTW, while debugging I read in the dovecot documentation that it is no problem to have Maildir files with different compression types, it will work out of the box. So no problem to switch the compression from LZ4 to zstd in the future, the problem was just the odd alpine dovecot image…

        When this plugin is loaded Dovecot can read both compressed and uncompressed files from Maildir. The files within a Maildir can use any supported compression algorithm (e.g., some can be compressed using gzip, while others are compressed using zstd). The algorithm is detected by reading the first few bytes from the file and figuring out if it’s a valid gzip or bzip2 header. The file name doesn’t matter.

        Zlib plugin — Dovecot documentation

        But, of course, this should be tested thoroughly 😃

          DocFraggle Yeah exactly i remembered this so i switched it back then as Alpine had no lz4 support during the inital tests. Only zstd got added after my request first.

          However it is related something which has changed from Alpine 3.16 to 3.18.

          I just found something very, very, very, very interesting:

          GitLab Icon debian/patches/Support-openssl-3.0.patch · master · Debian / dovecot · GitLab

          This is a patch to support openssl 3.0 from last year. At this time Debian 12 was in development which ships OpenSSL3.0 and a running Repo Installation of Dovecot.

            DocFraggle Yes but only for debian! Debian creates their own patches.

            You can actually see that the Dovecot devs are/were working at this too here:

            GitHub Icon History for src/lib-dcrypt/dcrypt-openssl3.c - dovecot/core

            In the current stable versions of Dovecot there is only one dcrypt-openssl.c file not two like in the main (dev) branch.

            (See GitHub Icon core/src/lib-dcrypt at release-2.3.21 · dovecot/core

            and GitHub Icon core/src/lib-dcrypt at main · dovecot/core
            GitHub Icon GitHub
            core/src/lib-dcrypt at main · dovecot/core
            Dovecot mail server. Contribute to dovecot/core development by creating an account on GitHub.
            Dovecot mail server. Contribute to dovecot/core development by creating an account on GitHub.
            )

            Ubuntu also make their own Patches so that is also the reason why it is working with the Ubuntu Repo packages.

            Well, I thought I could quickly try to build an alpine dovecot image with the dovecot main branch, but I would need an Alpine Linux OS to do that 😄 that’s too much for the time I have to spare currently 🙂

            Anyways, is there a specific reason why you want to switch dovecot from the Debian image over to the Alpine image?

              DocFraggle Yes indeed. Alpine is supporting ARM64 better then Debian at least for Dovecot. Yes their repo version is also ARM64 compatible but it makes the implementation for newer features harder as they don’t the latest versions.

              With Dovecot 2.4 (or the current master state in git) there will change a lot which is causing mailcow to not even boot up. So yeah that’s a bit shitty. It has to be done in the future…

              6 days later

              Guys! Good news! My approach to Alpine has been submitted, they merged the patch!

              GitLab Icon main/dovecot: fix openssl 3 support (815bb154) · Commits · alpine / aports · GitLab

              Now it only have to work 🤞

              It works indeed! mailcow Team 🤜 🤛 Alpine Team

                DerLinkman Great news 🥳 I just deleted my ARM VM yesterday 🙃 but you already tested it I assume from your last line above 😀

                Yep it works. When the packages will be installed without a quirky stuff inside the docker image then i’ll probably push the updated Image so anybody can test.

                a month later

                Does this mean that a migration from x86 to Arm should now be possible?

                13 days later

                If you want to use it in a production environment I would definitely wait until @DerLinkman published a stable version

                Ganzjahresgriller I’m waiting on Alpine to release 3.19 stable which includes the Dovecot fixes.

                I won’t merge anything which is using Edge packages like the current nightly Version of Dovecot does. It needs to be stable at least OS wise!

                  No one is typing