rspamd bad_extensions mod

gsloop

I’ve tried to do this on my own, but can’t really make headway…
I’d like to document this for the wiki, so others can start from a better place than I am. So, I’m trying to figure out the “right” way to do this…rather than just hack something together that only kinda works, but is fragile and prone to break easily…

—
Let me explain what I’m trying to do, and you can suggest ways to do this most efficiently.
I’ve also posted on the telegram group, but really didn’t get anything useful there either.

I’ve got a hosted MC from you, and I’m also wanting to do this for an internal MC server at a client. (I’m pointing this out, since if I’m documenting this, I’d like to allow hosted MC users to do similar stuff using non SSH tools, if possible - since hosted MC doesn’t have access to SSH - even though I do.)

Here’s what I’m trying to do.
I want to essentially block all attachments that could potentially be hostile. (I’m trying to fine-tune that list currently, but it’s still something of a moving target…)

But obviously we’ll have some senders and recipients that we do want to allow attachments through.

So, we’ll then we’ll identify some sources we trust for attachments.
(I currently do this with Amavis in a non MC install, by identifying a triad of conditions -

1) The source server (or source email address - yeah, I know this is forge-able, but works for us where something that isn’t forge-able isn’t available.)
2) The destination address.
3) The attachment type. (say zip file, or word doc, or…)

If all three elements line up. (here’s an example)
From: gregs@schmoo.net
To: gregs@schmoodle.com
Type: zip file.

Then allow the attachment.

So, I want to do something similar in MC.

—
Several potential issues come up.
(Lets only worry about redefining bad_extensions here - not matching and allowing any through. If I can figure out how to best apply the block for the extensions I want, I’ll tackle the “allow” portion next, independent from this problem.)

Since MC is running in docker, we can’t just edit rspam config files in any context we want (at least I think so), since a MC update in docker will likely overwrite some files, and our customization will be lost. (This is different than if rspam was running on a bare-metal install. So, the interaction between MC/Docker/rspam matters.)

If I understand rspam correctly, we can’t use .local files to redefine things like “bad_extensions” where they are previously defined. We have to remove the definitions in the base config, and define them the way we want them defined in the .local config.

I haven’t tried setting up the triad of elements in the “allow” portion of the system yet, but lets not sweat that now. Lets just try to get the bad_extensions working the way I need first, then I’ll continue to work the problem to conclusion. (Plus I think we’ll avoid config over-writes doing the multi-maps for sender/recipient/file-type - since these aren’t previously defined items - so it will be less complicated, trying to mash two configurations together.)

gsloop

I’ll also post what I started trying - but it didn’t work. (I think I understand why, but I’m not at all certain.)

I used the rspamd web UI (hosted MC)

Edit the var/lib/rspamd/mime_types.inc.local file.
Add something like

bad_extensions = {
  7z = 40, 
  zip = 50,
};

Then use the scan/learn tool/tab to scan an email with, in this case, a 7z. (This is the easiest fastest way to test, I think.)

However the score stays the same at 1.6 for a 7z extension, no matter what values I use in the config.
7z = 40 or 7z = 1.

I restarted the rspamd container (though I’m not sure that matters.) That didn’t seem to change things either.

As I read the rspam docs, and details about override and local files - it seems like you can’t redefine items in the local/override that were defined previously - it appears that the rspam docs say that the re-definition will simply be ignored. And this appears to bear out in the test I did above. They state you have to remove the initial def and define it in a local/override file.

So, I think that’s going to mean that I have to remove the def in the base mailcow install.
If so, that might mean that I’ll be modifying a file that will get overwritten on update.

But now I’m way out of my knowledge base.

Can you shed some light on this?

diekuh

You can edit the files locally.

As long as we didn’t change the same code, your changes will be kept.

It’s a managed mailcow, just let us know what you want to have changed (via ticket, please). 🙂

gsloop

But it IS changing the same code.
You have “bad_extensions = { };” defined in the default install - here:
/opt/mailcow-dockerized/data/conf/rspamd/local.d/mime_types.conf

So, I have to edit that file directly to modify the multipliers for each extension type, or to add new extensions, since I can’t, I don’t think, just re-define bad_extensions in a .local file/directory.

Is editing /opt/mailcow-dockerized/data/conf/rspamd/local.d/mime_types.conf permissible and avoids overwrite by an update?

diekuh

Oh, sorry, that’s a misunderstanding.

I was talking about the local.d files. They can be edited and will be kept when A we didn’t change the same line or B when mailcow is updated with the --ours switch.

You want that, right? =>

  7z = 40, 
  zip = 50,

gsloop

I most want to know how best to do this myself.
Like I said, I’d like to add this to the wiki, and I have a corporate mail-server I’m trying to stand up as well, where I need to make these kinds of mods.

So, I really need to know how to go about it without screwing stuff up, and allowing updates with the least amount of fuss.

diekuh

Yes, as I explained above. 🙂

gsloop

Can we be a little less terse? 🙂

So, to summarize;
I can edit: /opt/mailcow-dockerized/data/conf/rspamd/local.d/mime_types.conf
But it may get updated/overwritten when one runs the “update” script.
(Will that only happen if the file was changed, or will it overwrite every time, regardless of change from version to version?)

Using the –ours switch in the update script will keep the/my change. But it might cause other issues if there’s something in the update that impacts the file you edited.

Do I have that all right?

If so, then does the –ours flag simply/blindly ignore any updates from the master?
Is there a diff output to show you the differences?
Is there a way to get both versions so you can do your own diff on it?

—
Lastly, going back to the original post;
Is it permissible to use the rspamd UI to edit the files listed on the configuration tab? [Like:
/var/lib/rspamd/mime_types.inc.local ]

Or will that cause issues? (It won’t help in the specific case I’m tinkering with, but it would be nice to know for general use guidelines.)

gsloop

@diekuh
Can you please answer these definitively?

diekuh

I explained it a few times in this thread already. 🙂 Have not checked back since I thought it was all clear now, sorry.

If you changed a line in a any file that WE changed in an update, too, then your change will be overwritten. If you use “–ours” as parameter for the update script, your change has a higher prio and will overwrite our change.

So you can change any file, always. But every change YOU do might break mailcow in future versions. So we always prefer to overwrite user changes.

For files like mime_types it is perfectly fine, there will not be breaking changes.

gsloop

@diekuh
From previous posts:
Can a user use the rspam UI to change the files listed on the config tab? (As noted, I’m not changing these, but they’re offered as editable. And if it’s not safe to edit these, or precautions need to be taken, it ought to be documented, IMO. So that’s why I’m asking.)

As of right now, it looks like I’ll need to make changes to:
mime_types.conf
multimap.conf
composites.conf

I’m likely to do this, this way.
Include a line like:
.include(try=true,priority=10) “${CURDIR}/multimap-mod.conf” at the top of the file. (Obviously the name will be different for each include.)

Then include all the changes or additional code/defs in that “included” file.
This means (I think) that for an update, I can simply accept the changes from master, (i.e. not use –ours) and then re-edit the relevant files to re-add the includes.
My added files shouldn’t get over-written, right?

In the case of multi-map, the include should occur before the rest of the defs, so it will take priority, right?

Is there any flaw you can see in this approach that I should be wary of?

—
In case, you’re wondering what I’m trying to do again, see the first post. I think it explains it fairly well. If not, I’m glad to expand on anything as needed.

Thanks!

diekuh

You can change scores in the Rspamd UI. It will NOT change the files in local.d in this case. It will create override-files in the rspamd-vol-1 volume. This volume persists. If you want to use the Rspamd UI, that’s the best way to modify scores. But as mentioned before, you could also edit the local.d files.

Any changes to multimap and composites are very, very likely to break things we implement at a later point. If you know what you are doing, go ahead and change rules etc. - but we cannot provide official support then of course. When you change multimap or composites, you should use update.sh --ours, because we change these files quite often and may break your changes.

Regarding maps and orders of executions (like prios), please see the Rspamd docs or ask in their Telegram channel.

gsloop

You can change scores in the Rspamd UI. It will NOT change the files in local.d in this case. It will create override-files in the rspamd-vol-1 volume. This volume persists. If you want to use the Rspamd UI, that’s the best way to modify scores. But as mentioned before, you could also edit the local.d files.

I think we have a misunderstanding…
Scores, I think, get changed on the “Symbols” tab in the rspam UI.
I’m asking about the file edits you can make on the “Configuration” tab.

When you change multimap or composites, you should use update.sh –ours, because we change these files quite often and may break your changes.

But if I’m putting my changes in an “include” file - that’s separate from composites and multimap - wouldn’t I NOT use “–ours” and let the update pull in from master, then add the include back in?

Doesn’t that make it easier to keep each of our changes separate and handle updates more cleanly? (If something doesn’t work right, remove the include and see if it does. If it does, then you know something in the include borked things up.)