I am using Sieve filter to redirect incoming emails to my gmail account. In some cases I am seeing messages bounced from gmail with a duplicate header error. Investigating further it looks like there is a duplicate Message-ID header being added by Sieve when it believes that the original Message-ID header is invalid/malformed.

In some cases this is reasonable, e.g. the message ID value does not contain the “@” character. However in some cases the original message ID value does not include the “<” and “>” around the value but is otherwise OK. From what I have been reading it looks like the relevant RFC says that the “<” and “>” are not actually part of the message ID itself, so this seems like a bug with Sieve in this case.

Is there any way to suppress this behaviour in Sieve and ignore malformed message ID headers in the redirected messages?

7 days later

Hi,

I’ve discovered this exact same issue.

It looks like dovecot’s pigeonhole handling sieve is adding a Message-ID when there is no Message-ID in the original mail.
However it also seems it somehow considers “bogus” Message-ID like there was no Message-ID at all.

In this situation, it results in double Message-ID header in the forwarded mail.

I’m trying to discuss the issue at the dovecot mailing list:
Pigeonhole redirect is adding a message-id header when it already exists

dovecot.org
Pigeonhole redirect is adding a message-id header when it already exists

Have something to say?

Join the community by quickly registering to participate in this discussion. We'd like to see you joining our great moo-community!

4 months later

Hey @r00tsh3ll , I followed your conversation over at dovecot.org but felt like it trailed off. Did you manage to find a fix for the bogus Message-ID? Or @cjwalsh, How did you get on in the end?

I haven’t had the chance to pick this back up again, I’m mostly just adding filters to avoid forwarding messages from senders that use “invalid” Message ID formats and picking them up eventually in Gmail via POP3 checking.

Hello @Untapped9992,

Yeah unfortunately it seems this the interest for this issue on the dovecot mailing list went to /dev/null 🙁 Don’t hesitate to rise it again there though.

I have no solution yet to workaround this yet. I thought about doing some headers check in postfix before the mail is sent out, so that if there are duplicate Message-ID, it only keeps the first one and remove the Message-ID added by pigeonhole.
Maybe using this: Postfix manual - header_checks(5)

postfix.org
Postfix manual - header_checks(5)

But i’m not even sure it’s actually possible to add this kind of logic.

Kind regards

Cool, thank you both. I feel we’re all trying to treat the symptom rather than the cause. We ’shouldn’t’ have to fix this kind of thing as the issue (in my scenario, anyway) is the external sender.

I’ll keep having a play around and report back if I find any solutions!

a year later

I’ve encountered the same issue… Reading around I found this commit changelog: dovecot/pigeonholeblob/main/NEWS

Notice “v0.4.13 18-03-2016” – “redirect action: Made mail loop detection more robust by forcibly adding a Message-ID header if it is missing.”. This feels like the time when the bug was introduced.

However, I also found this changelog: Pigeonhole: ChangeLog | Fossies

fossies.org
Pigeonhole: ChangeLog | Fossies
fossies.orgfossies.orgfossies.org
and source here: dovecot/pigeonhole87ea4d7

Notice “2023-05-09 07:29:46 +0000” line – “lib-sieve: act_redirect_execute() - Pick Message-id even if invalid. This avoids adding another Message-id if one is already present but of bad quality.”

The latter very much feels like a fix for the issue discussed here and in the dovecot mailing lists? Anyone had any workarounds (other than manually creating filter rules) or fixes for this, otherwise?

7 days later

As of this message Mailcow currently ships with:
docker compose exec dovecot-mailcow doveconf | head
| 2.3.21 (47349e2482): /etc/dovecot/dovecot.conf
| Pigeonhole version 0.5.19 (4eae2f79)

Dovecot is the latest release, but Pigeonhole needs version 0.5.21.

a month later

We are having a fair amount of duplicate Message-id problems from alibaba / aliexpress. I would like to get this solved. I am happy to pay for pigeonhole to be updated / sponsor this fix. I tried to sign up for support and the website hangs. Does anyone have any suggestions on how to help move this along?

I can’t help with your problem, but I was wondering why mailcow contained an older version of Pigeonhole, especially since mailcow simply installs the Alpine Linux package of dovecot-pigeonhole-plugin when GitHub Icon building the Dovecot Docker image

GitHub Icon GitHub
mailcow-dockerized/data/Dockerfiles/dovecot/Dockerfile at 70126e1f0cb0f78d99446453521b8d665fe07db9 · mailcow/mailcow-dockerized
mailcow: dockerized - 🐮 + 🐋 = 💕. Contribute to mailcow/mailcow-dockerized development by creating an account on GitHub.
mailcow: dockerized - 🐮 + 🐋 = 💕. Contribute to mailcow/mailcow-dockerized development by creating an account on GitHub.
.

Also it is the lastest package available (see the .21):

root@mailcow# docker compose exec dovecot-mailcow apk info dovecot-pigeonhole-plugin
dovecot-pigeonhole-plugin-2.3.21-r17 description:
Sieve and managesieve plugin for Dovecot

But looking at the respective git.alpinelinux.org Icon Alpine package souces

git.alpinelinux.org Icon git.alpinelinux.org
APKBUILD « dovecot « main - aports - Alpine packages build scripts
cgit logo
, they are explicitly using dovecot-pigeonhole-0.5.19.tar.gz to build the Pigeonhole sub-package for Dovecot.

According to the dovecot.org Icon 0.5.21 release notes

dovecot.org Icon dovecot.org
Pigeonhole v0.5.21 released - Dovecot-news - dovecot.org
Loading...HyperKitty
, Dovecot indeed fixed a problem with Message-ID headers.

0.5.20 (and even 0.5.19) were just GitHub Icon releases to keep the version numbers synced

GitHub Icon GitHub
pigeonhole/NEWS at main · dovecot/pigeonhole
Pigeonhole project: Sieve support for Dovecot. Contribute to dovecot/pigeonhole development by creating an account on GitHub.
Pigeonhole project: Sieve support for Dovecot. Contribute to dovecot/pigeonhole development by creating an account on GitHub.
between Dovecot and Pigeonhole.

Long story short: If this fix in 0.5.21 is the solution to your problems, either Alpine must update their package sources to .21 (which would also be less misleading regarding the version number) or mailcow should use a different way to install the current version of Pigeonhole.

    accolon Thank you for all of that explanation. This makes a lot of sense as to why the older version is happening.

    Ideally, it would be great to get pigeonhole updated so the fix is elegant. I see there is lobbying of other groups that may be necessary.

    Less ideally here is the scenario:

    The original email comes in with this message-id:
    Message-ID: 9833819030155984$ba6dfc7f00d0457e96aaa5b6ecb7a02e

    Because it is malformed a new message-id is added to the headers like
    Message-ID: <dovecot-sieve-1720341929-837352-1@72f9e7376ada>

    Then reputable mail servers reject the forward on the basis that there are multiple message-ids. Any work around hack idea where I can remove one of the message-ids in a way that I can comment out later when the fix ultimately makes it out?

      5 days later

      I’ve added the following at the end of

      data/conf/postfix/anonymize_headers.pcre

      as a VERY hacky solution for the moment, that solved the issue:

      /^(?i)Message-ID:\s+<dovecot-sieve.*?@.*?>/ IGNORE

      Don’t forget to reload postfix with:

      docker compose exec postfix-mailcow postfix reload

      Mechanic - I love you. It works perfectly until a proper solution is implemented.

      Thank you.

      Thanks for the heads up. I requested they update the version and I also filed a support ticket with mailcow who said they would request it as well. Happy to see everyone working together.

      6 days later

      It would be great to see this finally resolved, looks like there might be some light at the end of the tunnel!

      6 months later

      • esackbauer

        • Community Hero
        Moolevel 347
      • Post #19

      mechanic
      Please bump the issue on github (or open an issue)

        mechanic

        Confirmed with Dev team that this is scheduled next update.

        “we plan the integration of this update for the february 2025-02 update.”

          No one is typing