Attachment issue after updating to Mailcow 2025-03a

DocFraggle

sOliver The problem here is 100% reproducable:
1) create new email and write some text
2) make a screenshot and paste that after the text (now we’ve a inline graphic)
3) write some more text (after the pasted graphic)
4) add attachements (we had PDFs, not big at all 60 KB)
5) send eMail

0% reproducable for me…

I added a plain text footer along with the HTML footer from above. I created the mail exactly as you wrote and added a 200KB PDF. I sent the mail to GMX, Hotmail and my work email. All 3 emails were OK, attachment included.

Sent via SoGo
Received with Outlook

Did you send the mail via SoGo, too?

sOliver

DocFraggle
Not sure if i had tested that as well, sorry.
I’ll try to replicate, but need to wait until evening.

I myself (and that client where it happens) use ‘em Client’ as eMail Client.
Need to add, it’s the first time, that i tried to establish the email footer with any mailcow installation.

DocFraggle

OK: I added the mailbox via IMAP to Outlook, this time the attachment was stripped, too

sOliver

DocFraggle
Ok, you’re right, i had not told how the clients connect to Mailcow.
Always IMAP (no POP here).

But interesting, that you had it too (now with IMAP).

DocFraggle

Well, SoGo connects via IMAP as well, strange… I will check the mail’s source code later

BTW, SoGo displays the attachment of the mail sent via Outlook when looking at it in the Sent folder, even stranger

sOliver

DocFraggle
In my tests SOGo didn’t display them. Just another data point.

Thanks for looking at the source code (no coder here, so sadly can’t help).

shaneodoo

I am somehow drawn to ClamAV data/conf/rspamd/antivirus.conf

clamav { type = "clamav"; symbol = "CLAM_VIRUS"; servers = "127.0.0.1:3310"; scan_mime_parts = true; scan_text_mime = true; scan_image_mime = true; timeout = 15s; }

Scan_mime_parts = true, before sending ClamAV breaks down the email and scans each part separately before sending now if it takes a while to scan, postfix might not be waiting for ClamAV to finish the scan and send the email not 100% intact therefore missing off an attachment. Setting ‘scan_mime_parts = false’ I believe allows ClamAV to scan the email as a whole before postfix gets its hands on it to send. Any thoughts on this…?

DocFraggle

OK, I debugged a little further. I sent the email via Outlook from my test domain to my main mailbox in a different domain (same mailcow instance).
I then decrypted the mail file in the “Sent” folder of my test mailbox and the received mail file in the inbox of my main mailbox to have a look.
As I already told you, the mail in the “Sent” folder is ok, attachment visible. The mail in my inbox shows the attachment icon, but the attachment isn’t visible.
I checked the source of the inbox mail file vs the sent file, and this seems to be the problem: the MIME part of the multipart message seems to be scrambled.

Here’s the code of the inline image and the attached file. It should look like this (sent mail file):

------=_NextPart_001_008F_01DC00A7.817FB140
Content-Type: image/jpeg;
        name="image001.jpg"
Content-Transfer-Encoding: base64
Content-ID: <image001.jpg@01DC007C.7283FEB0>

/9j/4AAQSkZJRgABAQEAYABgAAD/4QBmRXhpZgAATU0AKgAAAAgABAEaAAUAAAABAAAAPgEbAAUA
AAABAAAARgEoAAMAAAABAAMAAAExAAIAAAAQAAAATgAAAAAAAJOjAAAD6AAAk6MAAAPoUGFpbnQu
TkVUIDUuMS44AP/bAEMAAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEB
AQEBAQEBAQEBAQEBAQEBAQEBAQEBAf/bAEMBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEB
AQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAf/AABEIAAEAAQMBEgACEQEDEQH/xAAf
AAABBQEBAQEBAQAAAAAAAAAAAQIDBAUGBwgJCgv/xAC1EAACAQMDAgQDBQUEBAAAAX0BAgMABBEF
EiExQQYTUWEHInEUMoGRoQgjQrHBFVLR8CQzYnKCCQoWFxgZGiUmJygpKjQ1Njc4OTpDREVGR0hJ
SlNUVVZXWFlaY2RlZmdoaWpzdHV2d3h5eoOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3
uLm6wsPExcbHyMnK0tPU1dbX2Nna4eLj5OXm5+jp6vHy8/T19vf4+fr/xAAfAQADAQEBAQEBAQEB
AAAAAAAAAQIDBAUGBwgJCgv/xAC1EQACAQIEBAMEBwUEBAABAncAAQIDEQQFITEGEkFRB2FxEyIy
gQgUQpGhscEJIzNS8BVictEKFiQ04SXxFxgZGiYnKCkqNTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNk
ZWZnaGlqc3R1dnd4eXqCg4SFhoeIiYqSk5SVlpeYmZqio6Slpqeoqaqys7S1tre4ubrCw8TFxsfI
ycrS09TV1tfY2dri4+Tl5ufo6ery8/T19vf4+fr/2gAMAwEAAhEDEQA/AP7+KKAP/9k=

------=_NextPart_001_008F_01DC00A7.817FB140--

------=_NextPart_000_008E_01DC00A7.817F8A30
Content-Type: text/plain;
        name="0.txt"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
        filename="0.txt"

0
------=_NextPart_000_008E_01DC00A7.817F8A30--

You can see the start and end of the attachments indicated by the “NextPart” lines.

Start of the inline image:
------=_NextPart_001_008F_01DC00A7.817FB140
End of the inline image:
------=_NextPart_001_008F_01DC00A7.817FB140--

Start of the attached file:
------=_NextPart_000_008E_01DC00A7.817F8A30
End of the attached file:
------=_NextPart_000_008E_01DC00A7.817F8A30--

Now here’s the received mail’s source:

------=_NextPart_002_008A_01DC00A7.7FB837B0--

------=_NextPart_001_0089_01DC00A7.7FB837B0
Content-Type: image/jpeg;
        name="image001.jpg"
Content-Transfer-Encoding: base64
Content-ID: <image001.jpg@01DC007C.7283FEB0>

/9j/4AAQSkZJRgABAQEAYABgAAD/4QBmRXhpZgAATU0AKgAAAAgABAEaAAUAAAABAAAAPgEbAAUA
AAABAAAARgEoAAMAAAABAAMAAAExAAIAAAAQAAAATgAAAAAAAJOjAAAD6AAAk6MAAAPoUGFpbnQu
TkVUIDUuMS44AP/bAEMAAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEB
AQEBAQEBAQEBAQEBAQEBAQEBAQEBAf/bAEMBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEB
AQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAf/AABEIAAEAAQMBEgACEQEDEQH/xAAf
AAABBQEBAQEBAQAAAAAAAAAAAQIDBAUGBwgJCgv/xAC1EAACAQMDAgQDBQUEBAAAAX0BAgMABBEF
EiExQQYTUWEHInEUMoGRoQgjQrHBFVLR8CQzYnKCCQoWFxgZGiUmJygpKjQ1Njc4OTpDREVGR0hJ
SlNUVVZXWFlaY2RlZmdoaWpzdHV2d3h5eoOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3
uLm6wsPExcbHyMnK0tPU1dbX2Nna4eLj5OXm5+jp6vHy8/T19vf4+fr/xAAfAQADAQEBAQEBAQEB
AAAAAAAAAQIDBAUGBwgJCgv/xAC1EQACAQIEBAMEBwUEBAABAncAAQIDEQQFITEGEkFRB2FxEyIy
gQgUQpGhscEJIzNS8BVictEKFiQ04SXxFxgZGiYnKCkqNTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNk
ZWZnaGlqc3R1dnd4eXqCg4SFhoeIiYqSk5SVlpeYmZqio6Slpqeoqaqys7S1tre4ubrCw8TFxsfI
ycrS09TV1tfY2dri4+Tl5ufo6ery8/T19vf4+fr/2gAMAwEAAhEDEQA/AP7+KKAP/9k=

------=_NextPart_001_0089_01DC00A7.7FB837B0--

------=_NextPart_000_0088_01DC00A7.7FB810A0--

------=_NextPart_000_0088_01DC00A7.7FB810A0
Content-Type: text/plain;
        name="0.txt"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
        filename="0.txt"

0

As you can see, the lines marking the end of the attachments are ABOVE the starting lines, which breaks the mail.

Now we have to find out which component of mailcow breaks the mail… maybe it’s RSpamd, maybe Clamd used by RSpamd

DocFraggle

Ah… @FreddleSpl0it found this RSpamd bug back in February and opened a Github issue:

rspamd/rspamd5334

It’s already fixed, merged and the fix was released with RSpamd 3.11.1. This version was added with the mailcow release 2025-03. But it seems the problem still exists, even with RSpamd 3.11.1

shaneodoo

It might have something to do with the message_size_limit if you ensure that Postfix, ClamAV and Rspamd size limits are between 25mb and 50mb, when sending emails with attachments they are not being chopped off at the end. I have set mine to 50mb even though I will never send that much data.

I can confirm that attachments are getting through now with out issues. Unless my client sends malformed filenames, which I have notified them of this.

sOliver

shaneodoo
Well, here it never was a file size matter in the first place (60-100 KB).
Even though, i raised the limits at the installation time to 100 MB anyways.

Like @DocFraggle wrote, i also had the suspision, that the encode of the eMail with inline parts and attachements gets messed up, so that the receiver side simply can’t decode and so ignores/strips those then.

But, the problem here only happens, when the automatic footer is active (or no inline graphic is in the body section of the email). So that should help in determining, which component is responsible for it – and i guess it’s RSpamd (if i remember the talks back then about implementing [the domain wide footer] it).

DocFraggle

sOliver So that should help in determining, which component is responsible for it – and i guess it’s RSpamd

It’s already determined, it’s this RSpamd lua script:

FreddleSpl0it/rspamdblob/master/lualib/lua_mime.lua

sOliver

DocFraggle
Ok, can it be fixed?

DocFraggle

sOliver Not by me 🙂 @FreddleSpl0it could have a look I guess

sOliver

DocFraggle
Oh that would be perfect!

I had to replaced an (aging) Exchange installation for one client and ran into this bug. Not the most pleasant experience. Only option was to disable the footer and do alot of “add the complete signature at every client machine” – puh!

shaneodoo

I have been doing a bit of digging in the lua.. One of them in particular looks like it could be breaking MIME formatting. Your thoughts on this would be helpfull.

1. Make the fallback MIME boundary unique (lines 184-185)

`- boundaries[#boundaries + 1] = boundary or '--XXX'          -- line 184
+ boundaries[#boundaries + 1] = boundary or '--' .. rspamd_util.random_hex(8)  -- line 184`

Using a hardcoded boundary like –XXX isn’t safe. Swapping in a random hex suffix keeps things unique and avoids accidental collisions.

2. Likely Bug: Incorrect string.sub when checking footer (lines 198-199)

- if content:sub(-(nlen), nlen + 1) == ending then           -- line 198
+ if content:sub(-nlen) == ending then                       -- line 198

The original mixed a negative and positive index in string.sub, which doesn’t work properly. Fixing this should ensure the footer presence check works correctly and doesn’t break MIME structure.

3. Cleaner fallback for missing content (lines 205-208)

- local content = tp:get_content()                           -- line 205
- if not content then                                         -- line 206
-   content = rspamd_text.fromstring('')                     -- lines 207-208
- end
+ local content = rspamd_text.fromstring(tp:get_content() or '')  -- line 205

Simplifies fallback to always get a rspamd_text object, even if content is empty.

4. Fix broken regex for IPv4 redaction (lines 247-248)

- ret = ret:gsub('[%d+%.%d+%.%d+%.%d+]', 'XXX.XXX.XXX.XXX') -- line 247
+ ret = ret:gsub('%d+%.%d+%.%d+%.%d+', 'XXX.XXX.XXX.XXX')    -- line 247

Square brackets were turning the pattern into a char class; fixing it to a proper pattern matches IP addresses.

5. Optional tweak: make the HTML threshold configurable (lines 270-271)

- if part:is_html() and part:get_words_count() >= 10 then   -- line 270
+ local html_min_words = 10                                  -- new line before 270
+ if part:is_html() and part:get_words_count() >= html_min_words then  -- line 270

No functional change, just makes the word-count threshold easier to adjust.

Of all the bugs I could see from above #2 looks like the little blighter that could be braking MIME when domain footers are added, which could also cause attachments to be dropped as it seems to be messing around with the multipart structure.

Your thoughts on this would be great.

sOliver

shaneodoo
Thanks for digging into this…

Well, my only programming skills were assembly in the days when the Motorola 68000 CPUs were kings.
I try to help were i can.

1) MIME boundary
Sounds good to me. I understood, you replaced the “XXX” (which would always be the same for every MIME, right?) with an random hex string, so every MIME part is unique and couldn’t be mismatched.
But your wrote “fallback MIME boundary” – does this imply, there’s another MIME boundary check before that?

2) Checking for footer
Your reasoning sounds good to me and surely (if code was wrongly mixing -/+) then could cause our trouble.

3) Fallback missing content
When could content ever be missing?
Could an empty text object inject another point of trouble later on?

4) Fix broken regex
The square brackets have a special meaning inside a regex. But if the meaning got changed because of the lua language here, then it’s great, that you found it.

5) Optional tweak
This 10 word count was hardcoded. In which context was it used for? Do something only if at least 10 words are in the body part?
Anyway, i love to see adjustable settings.

If i could do anything else, please say so.

stefan21

@shaneodoo

I appreciate very much your jumping on this one. Did you open a bug on github? Or any reaction (patch) on this from the devs?

@devs: will this be patched?

shaneodoo

@stefan21

No just looking through the lua and think the code could be tided up a bit. It might be the reason for the missed attachments and MIME generation. Thought better to take about it in the community before submitting an git changes.

stefan21

For a test I changed inside the rspamd-container (docker compose exec rspamd-mailcow /bin/bash) the /usr/share/rspamd/lualib/lua_mime.lua file. Specifically for the suggestions:

Make the fallback MIME boundary unique (lines 184-185)
Likely Bug: Incorrect string.sub when checking footer (lines 198-199)
Cleaner fallback for missing content (lines 205-208)

Could not find no. 4 and no. 5 inside the file?

Did a restart of the container. Several test sending emails with/without attchments are showing no errors.

While sending emails with an enabled domain wide footer - and this issue

https://community.mailcow.email/d/4230-strange-characters-in-email-from-tb-to-mailcow/6

seems also to be solved.

How can this made persist?

« Previous Page Next Page »