Restore results in duplicated and unread messages

felixoi · Dec 22, 2022

Hey there,

I’m currently working on migrating my emails to a self-hosted mailcow instance running in the Hetzner Cloud. I tried to use Ansible as much as I can, which works really good ( GitHub - felixoi/mailcow-hetzner-ansible: Ansible playbook to automatically set up a mailcow on Hetzner Cloud, including provisioning of the server itself, with optional backups to a Hetzner Storage Box (or other providers with available ssh access).

GitHub

GitHub - felixoi/mailcow-hetzner-ansible: Ansible playbook to automatically set up a mailcow on Hetzner Cloud, including provisioning of the server itself, with optional backups to a Hetzner Storage Box (or other providers with available ssh access).

Ansible playbook to automatically set up a mailcow on Hetzner Cloud, including provisioning of the server itself, with optional backups to a Hetzner Storage Box (or other providers with available s...

). Now I wanted to test backup/restore before I start the production migration and got some problems with it. I’m using Borgmatic as described in the

docs

docs.mailcow.email

Borgmatic Backup - mailcow: dockerized documentation

None

. Backups are working good and running one an hour but when I restore I’m getting duplicated and unread messages in the testing account.
Starting from these mails in the Inbox:

Restoring a backup from earlier this day:
Removed readonly for the mounted borgmatic volumes.

docker-compose exec borgmatic-mailcow borgmatic restore --archive mailcow-2022-12-22T12:14:08.710457
docker-compose exec borgmatic-mailcow borgmatic extract --path mnt/source --archive mailcow-2022-12-22T09:32:48.442065
docker-compose down && docker-compose up -d

Results in something like this:

As you can see some emails are duplicated with a read and a new unread version.
This is not what I expected. Someone has an idea what’s going on maybe?

Thanks,
Felix

felixoi · Dec 23, 2022

Fixed it with docker-compose exec dovecot-mailcow doveadm -D deduplicate -m -u '*' mailbox INBOX but still would like to know why and when this happens.

felixoi · Jan 25, 2023

Ok, bumping this once again as I found the acutal underlaying issue.
Seems like docker-compose exec borgmatic-mailcow borgmatic restore --archive latest only merges the folder extracted the backup into the current folder state. This is why after a restore the maildirs end up in a weird situation. New mails in the maildir that were not existent at the time of the backup, will stay there. If you moved a mail then you are going to have duplicates as the restore will copy the old file into the old folder.

Is this intended behavior? How does backup_and_restore.sh work in this case? Is it totally resetting to the backup state? I can reproduce this for the Borgmatic setup described in the docs.