AI-Powered Spam Filtering

VulcanTom

Dear Mailcow-Team and Community

First of all a huge thanks to you all for your contribution to an easy to install and maintainable full featured email server solution on open source base to use for everybody! 💪

I’m mailcow user since a year and impressed how good and simple the solution works for daily usage. My goal would be to stay long therm here. Unfortunately is there a point which increases doubts. The spam detection rate is terrible bad! My email boxes are getting literally flooded on a daily base. Only minor amount keeps hanging in the junk box. That tarnishes the pleasure to run the mailcow solution long term.

Is there a plan and timeline to address this point for instance with AI powered technology? I would help and/or donate for such a plan to make/keep mailcow a good choice for the future.
All I found on this topic is this mailcow post => “Rspamd GPT Plugin”

Wish list:

Data Privacy: Option to use Local AI
Green resource-saving design: allowing embedded system to run e.g. trigger AI filter through rspamd threshold (Rspamd GPT Plugin) and using distilled models
Continuous Improvement: Open and transparent community way to share filter experience
System Protection: Blacklist/block connections for repeat offender through help of AI

Are there any planned efforts in this directions? Maybe an adapted version of DeepSeek could be an easy option for now?

pkernstock

Honestly, I’m not having issues with spam detection. The rspamd fuzzy was down for some days, but was recently fixed. Maybe that caused some minor bump in spam raise on your side.

You can use the GPT plugin (see rspamd docs) nowadays without any issues, if you wish to. It does integrate transparently with mailcow. (It also has options to run it with your own ollama installation, if you happen to have one)

What also helps to some degree is the url_redirector rspamd plugin (docs). I do have it enabled, works well. It isn’t enabled by default, because it can potentially cause quite some HTTP/S calls for resolving redirections. (which not all users want)

Hope this helps

VulcanTom

@pkernstock Thank you, I saw the GPT plugin option. It’s new to me this option supports to run a local AI instance. GPT is famous for it’s enormous power consumption. My HW is on embedded level very moderate on power consumption and performance. I don’t want to kill my server. Do you have some experience to share?
Regarding the url_redirector, is there further documentation on what this exactly does? The rspam docs are too cryptic to me only showing the config coding.

pkernstock

VulcanTom It’s new to me this option supports to run a local AI instance.

It was added in a quite recent rspamd commit: rspamd/rspamdb33cbea

VulcanTom My HW is on embedded level very moderate on power consumption and performance. I don’t want to kill my server. Do you have some experience to share?

On a VM with approx. 20 cores @ 2 GHz a quick “Hello, how are you” takes approx. 15 minutes to reply and around 20 GB of memory or so. On a RTX 3080 Ti, it takes around 5 seconds.

VulcanTom Regarding the url_redirector, is there further documentation on what this exactly does? The rspam docs are too cryptic to me only showing the config coding.

In a nutshell, it follows redirects of links in an email and then scores the links if they are known bad. In other words, it “resolves” URL shorteners such as j.mp

VulcanTom

@pkernstock Thanks again for your active response. Okay, so the local GPT is not an option at all. I’m looking for something power consumption below 10 watts average during daylong. It’s my home server and I do not want to waist power. Maximum possible to use a notebook CPU powered server with peak consumption of 55 watts.
Regarding the redirects, this might be an option. I checked most of my spam really has a link in there. Here is an overview of example spam I get. The spam looks very focused coming from a single source. Unfortunately are the source IP addresses alternating otherwise I would simply block them.

DocFraggle

VulcanTom I had the same problem with these exact SPAM mails. I changed some RSpamd symbols, see this thread:

https://community.mailcow.email/d/4411-too-much-spam-going-through-mailcow/2

I changed the following:

BAYES_SPAM = 6 (default 4.5)
IP_REPUTATION_SPAM = 6 (default 4.5)
BAD_REP_POLICIES = 3 (default 2)

This decreased the above mails by ~ 90%

I have to add: you have to move all of these mails to your Junk folder in order to train the Bayes algorithm, otherwise you won’t get the BAYES_SPAM symbol to fire. I think with the current configuration you need 200(?) learned mails

pkernstock

The hint from @DocFraggle in regards to Junk folder is a crucial one. I’ve been hit with same emails lately, but barely getting these through now (without adjusting outlined symbols). So BAYES seems to be properly trained now.

VulcanTom

Dear @DocFraggle and @pkernstock

Thanks for your prompt reply and support. My server just got updated from Nov-24 release to latest. The default values after the mailcow upgrade where:
BAYES_SPAM = 4.5
IP_REPUTATION_SPAM = 4.0
BAD_REP_POLICIES = 2.0

I changed them now as @DocFraggle recommended. I’m curious to see how much spam filtering gets improved and share my feedback after a while of field testing.

VulcanTom

Dear @DocFraggle and @pkernstock

The changed rspam settings are online since almost a week. There is much less spam in my inbox. However, the detection rate is still a kind of poor. As that type of spam emails have so a simple email subject pattern. I would expect a much aggressive learning curve to block them entirely after a short while and not even seeing them in the junk folder anymore. This is unfortunately not the case. Here is an example from 2 days ago:

A while ago some manual log analysis showed me altering IP addresses as source of this spam terror from same IP subnet of a dutch ISP. Any idea to directly firewall block such IP’s after certain re-occurrence of spam terror? E.g. kind of Regex log filter via Fail2Ban? Or similar?

maybl8

VulcanTom Any idea to directly firewall block such IP’s after certain re-occurrence of spam terror?

I block subnets via iptables but it is not recommended here.

pkernstock

But as per your screenshot, it is in the junk folder. So it’s fine? Just ignore it in the spam folder.

VulcanTom

Well, the score is extremely low. Some spam of same type do still snake through. Additionally, I don’t want think how much “brute force” traffic only this type of spam is causing on my email server. They should become simply blocked, not just nicely archived. On top of that, I’m not a fan to see how intensive my junk/quarantine is getting flooded. Good luck to find a false negative in that spam chaos.

smares

How do you enable the GPT plugin? I tried editing data/conf/rspamd/rspamd.conf.local with

gpt {
  enabled = true;
  api_key = "my API key";
  model = "gpt-4o-mini";
}

but the plugin is not loaded and the OpenAI dashboard shows that the key was never used.

w64

smares

The instructions in mailcow docs are about new modules, not included in rspamd. GPT plugin is included already in rspamd-mailcow. So basically, it’s needed to be created a new config file: data/conf/rspamd/local.d/gpt.conf
Open the file and fill the configuration parameters:

enabled = true; # Enable the plugin
type = “openai”; # Supported types: openai, ollama
api_key = “xxx”; # Your key to access the API
model = “gpt-4o-mini”; # Model name
max_tokens = 1000; # Maximum tokens to generate
temperature = 0.0; # Temperature for sampling
timeout = 10s; # Timeout for requests
prompt = “xxx”; # Prompt for the model (use default if not set)
condition = “xxx”; # Custom condition (lua function)
autolearn = true; # Autolearn if gpt classified
reply_conversion = “xxx”; # Reply conversion (lua code)
url = “https://api.openai.com/v1/chat/completions”; # URL for the API
allow_passthrough = false; # Check messages with passthrough result
allow_ham = false; # Check messages that are apparent ham (no action and negative score)
reason_header = “X-GPT-Reason”; # Add header with reason (null to disable)
json = false; # Use JSON format for response

Restarting rspamd-mailcow will load the gpt.lua plugin and if loading is successfull, you can view this messages in the rspamd-mailcow logs:
rspamd-mailcow-1 | 2025-02-28 23:41:30 #1(main) <1n5m8f>; lua; lua_redis.lua:605: use default Redis settings for gpt rspamd-mailcow-1 | 2025-02-28 23:41:30 #1(main) <1n5m8f>; cfg; rspamd_init_lua_filters: init lua module gpt from /usr/share/rspamd/plugins/gpt.lua; digest: bfd7f51817

I am in testing a ollama model, but i get errors like this one:
call to (GPT_CHECK) failed (2): /usr/share/rspamd/plugins/gpt.lua:192: attempt to call field 'get_displayed_text_part' (a nil value); trace: [1]:{/usr/share/rspamd/plugins/gpt.lua:192 - condition [Lua]}; [2]:{/usr/share/rspamd/plugins/gpt.lua:854 - <unknown> [Lua]};

…and can’t understand if gpt plugin for now support only openai, or ollama support is also implemented completely, but my configuration of the config file is wrong somehow?

smares

Thanks, works fine so far with OpenAI.