Outgoing Spam without Accounts/Mailboxes?

RogerThat

Hello everyone.
I’m using mailcow since a while and never had any problem, so thanks for the great product!
But since a yesterday, my mailcow instance is sending Spam-Messages to @qq.com addresses. The addresses are “random” numbers, but many of them are going through and are getting delivered.
The problem is that I don’t really understand why.

Here’s an example out of the postfix log:

01/04/2024, 02:19:16 PM	info	94F7847D74: client=unknown[172.22.1.1]

01/04/2024, 02:19:17 PM	info	94F7847D74: message-id=<id@mydomain.tld>

01/04/2024, 02:19:20 PM	info	94F7847D74: from=<address@mydomain.tld>, size=51105, nrcpt=1 (queue active)

01/04/2024, 02:19:20 PM	info	94F7847D74: to=<address@targetmail.tld>, relay=none, delay=4.5, delays=4.5/0.01/0/0, dsn=4.4.1, status=deferred (delivery temporarily suspended: connect to mx1.qq.com[162.62.116.184]:25: Connection timed out)

01/04/2024, 02:19:20 PM	info	disconnect from unknown[172.22.1.1] ehlo=1 mail=1 rcpt=1 data=1 quit=1 commands=5

I can’t really see which specific other log entries are related, but some of these are probably:

01/04/2024, 02:19:14 PM	info	connect from unknown[172.22.1.1]
01/04/2024, 02:19:14 PM	info	WHITELISTED [172.22.1.1]:39742
01/04/2024, 02:19:14 PM	info	CONNECT from [172.22.1.1]:39742 to [172.22.1.253]:25
01/04/2024, 02:19:14 PM	info	connect from unknown[172.22.1.1]
01/04/2024, 02:19:14 PM	info	WHITELISTED [172.22.1.1]:39726
01/04/2024, 02:19:14 PM	info	CONNECT from [172.22.1.1]:39726 to [172.22.1.253]:25

Anonymous TLS connection established from unknown[185.36.81.40]: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)

The sending IP Addresses are from Lithuania or Columbia. The sending Accounts/mailboxes do not exist, nor do they exist as alias addresses.

These are my Firewall Rules:

root@mx:~# iptables -S
# Warning: iptables-legacy tables present, use iptables-legacy to see them
-P INPUT ACCEPT
-P FORWARD DROP
-P OUTPUT ACCEPT
-N DOCKER
-N DOCKER-ISOLATION-STAGE-1
-N DOCKER-ISOLATION-STAGE-2
-N DOCKER-USER
-N MAILCOW
-N f2b-sshd
-A INPUT -m comment --comment mailcow -j MAILCOW
-A INPUT -p tcp -m multiport --dports 22 -j f2b-sshd
-A FORWARD -m comment --comment mailcow -j MAILCOW
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A FORWARD -o br-mailcow -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o br-mailcow -j DOCKER
-A FORWARD -i br-mailcow ! -o br-mailcow -j ACCEPT
-A FORWARD -i br-mailcow -o br-mailcow -j ACCEPT
-A FORWARD -o br-cff850d7011f -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o br-cff850d7011f -j DOCKER
-A FORWARD -i br-cff850d7011f ! -o br-cff850d7011f -j ACCEPT
-A FORWARD -i br-cff850d7011f -o br-cff850d7011f -j ACCEPT
-A DOCKER -d 172.22.1.7/32 ! -i br-mailcow -o br-mailcow -p tcp -m tcp --dport 443 -j ACCEPT
-A DOCKER -d 172.22.1.253/32 ! -i br-mailcow -o br-mailcow -p tcp -m tcp --dport 587 -j ACCEPT
-A DOCKER -d 172.22.1.7/32 ! -i br-mailcow -o br-mailcow -p tcp -m tcp --dport 80 -j ACCEPT
-A DOCKER -d 172.22.1.10/32 ! -i br-mailcow -o br-mailcow -p tcp -m tcp --dport 8983 -j ACCEPT
-A DOCKER -d 172.22.1.253/32 ! -i br-mailcow -o br-mailcow -p tcp -m tcp --dport 465 -j ACCEPT
-A DOCKER -d 172.22.1.250/32 ! -i br-mailcow -o br-mailcow -p tcp -m tcp --dport 12345 -j ACCEPT
-A DOCKER -d 172.22.1.250/32 ! -i br-mailcow -o br-mailcow -p tcp -m tcp --dport 4190 -j ACCEPT
-A DOCKER -d 172.22.1.249/32 ! -i br-mailcow -o br-mailcow -p tcp -m tcp --dport 6379 -j ACCEPT
-A DOCKER -d 172.22.1.13/32 ! -i br-mailcow -o br-mailcow -p tcp -m tcp --dport 3306 -j ACCEPT
-A DOCKER -d 172.22.1.253/32 ! -i br-mailcow -o br-mailcow -p tcp -m tcp --dport 25 -j ACCEPT
-A DOCKER -d 172.22.1.250/32 ! -i br-mailcow -o br-mailcow -p tcp -m tcp --dport 995 -j ACCEPT
-A DOCKER -d 172.22.1.250/32 ! -i br-mailcow -o br-mailcow -p tcp -m tcp --dport 993 -j ACCEPT
-A DOCKER -d 172.22.1.250/32 ! -i br-mailcow -o br-mailcow -p tcp -m tcp --dport 143 -j ACCEPT
-A DOCKER -d 172.22.1.250/32 ! -i br-mailcow -o br-mailcow -p tcp -m tcp --dport 110 -j ACCEPT
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -i br-mailcow ! -o br-mailcow -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -i br-cff850d7011f ! -o br-cff850d7011f -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -j RETURN
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -o br-mailcow -j DROP
-A DOCKER-ISOLATION-STAGE-2 -o br-cff850d7011f -j DROP
-A DOCKER-ISOLATION-STAGE-2 -j RETURN
-A DOCKER-USER -j RETURN
-A MAILCOW -s <fail2ban>/32 -j DROP
-A MAILCOW -s <spamsubnet>/24 -j DROP
-A MAILCOW -s <spamsubnet>/24 -j DROP
-A f2b-sshd -s <fail2ban>/32 -j REJECT --reject-with icmp-port-unreachable
-A f2b-sshd -j RETURN

I reboot the server daily at night, so the iptables settings are always like these at the moment, nothing was changed manually by myself, just 2 bans from fail2ban included + the 2 subnets I mentioned.

I set a rate-limit for now to stop it + I blocked some of the IP ranges over netfilter.

Any ideas how I can fix that? 🙁 I don’t really have a clue at this point. The only thing I could explain is a contact form from a friend that could be used, but that form runs on another server, so the servers IP should be inside the log, which isnt, and also the mailbox doesn’t have alias addresses, so I would guess that it can’t send under other addresses from the same domain?

esackbauer

You did not use the search function of the forum?
Have a look here:
https://community.mailcow.email/d/2493-server-sending-spam-to-qqcom-once-in-a-while-from-internal-ip-address/9

RogerThat

I did, but I didn’t found that specific topic to be honest…
I’ll give it a try, it is indeed correct that its a while ago that I updated docker-compose.
But even then I don’t understand how this could be possible just through docker compose.

I’ll report back tomorrow if the problem persist.

RogerThat

Alright, faster than I expected.
I did the update of docker-compose, also a update from Mailcow itself (But no changes were made, so it was up2date) and also a reboot to make sure, everything is actually using the new versions.
Sadly, the problem persists. I deleted the rate limit and the outgoing messages continues, so I set the limit again for now.

01/04/2024, 04:03:50 PM	info	5EA5A47C9E: from=<random@mydomain.tld>, size=155247, nrcpt=1 (queue active)
01/04/2024, 04:03:46 PM	info	5EA5A47C9E: message-id=<id@mydomain.tld>
01/04/2024, 04:03:45 PM	info	5EA5A47C9E: client=unknown[172.22.1.1]
01/04/2024, 04:03:44 PM	info	connect from unknown[172.22.1.1]
01/04/2024, 04:03:44 PM	info	WHITELISTED [172.22.1.1]:57070
01/04/2024, 04:03:44 PM	info	CONNECT from [172.22.1.1]:57070 to [172.22.1.253]:25

There are no directly coherent messages that fit to this one. The last successful connection before was uptimerobot, which makes sense, because I use it to check the availability of the Mailserver.
The one before that was:

01/04/2024, 04:02:46 PM	warning	warning: unknown[<IP from Lithuania>]: SASL LOGIN authentication failed: UGFzc3dvcmQ6, sasl_username=monitor@mydomain.tld
01/04/2024, 04:02:44 PM	info	connect from unknown[<IP from Lithuania>]
01/04/2024, 04:02:44 PM	warning	warning: hostname hostname_of_sending_server.tld does not resolve to address <IP from Lithuania>: Name or service not known
01/04/2024, 04:02:14 PM	info	Anonymous TLS connection established from unknown[<IP from Lithuania>]: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
01/04/2024, 04:02:09 PM	info	disconnect from unknown[<IP from Lithuania>] ehlo=1 auth=0/1 rset=1 commands=2/3
01/04/2024, 04:02:09 PM	info	lost connection after AUTH from unknown[<IP from Lithuania>]
01/04/2024, 04:02:04 PM	warning	warning: unknown[<IP from Lithuania>]: SASL LOGIN authentication failed: UGFzc3dvcmQ6, sasl_username=demo@mydomain.tld
01/04/2024, 04:02:02 PM	info	connect from unknown[<IP from Lithuania>]
01/04/2024, 04:02:02 PM	warning	warning: hostname hostname_of_sending_server.tld does not resolve to address <IP from Lithuania>: Name or service not known
01/04/2024, 04:01:32 PM	info	Anonymous TLS connection established from unknown[<IP from Lithuania>]: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
01/04/2024, 04:01:29 PM	info	disconnect from unknown[<IP from Lithuania>] ehlo=1 auth=0/1 rset=1 commands=2/3
01/04/2024, 04:01:29 PM	info	lost connection after AUTH from unknown[<IP from Lithuania>]
01/04/2024, 04:01:23 PM	warning	warning: unknown[<IP from Lithuania>]: SASL LOGIN authentication failed: UGFzc3dvcmQ6, sasl_username=azureuser@mydomain.tld

esackbauer

RogerThat also a reboot to make sure, everything is actually using the new versions.

That is not enough. You have to do docker compose up -d to make changes active.
Also check the link I posted for the network things.

RogerThat

esackbauer Should’ve been done when I made the update of Mailcow.
The order of updating was: docker-compose -> mailcow -> reboot
When updating mailcow, the containers are shutting down and then getting started again, aren’t they?
But I did it right now, just to make sure. Will update again if I see (or don’t see) anything.

Also, because I read it in another thread: I disabled IPv6 long time ago, that shouldn’t be a problem too.

esackbauer

RogerThat when updating mailcow, the containers are shutting down and then getting started again, aren’t they?

I guess not when nothing changed.

RogerThat

Okay, so after restarting mailcow with docker compose up -d the problem still persits.
Even more curious: I set a rate limit, which seems to be ignored. I set it to 0 earlier and noticed that 0 = disabled. Now I set it to 1, which is also displayed correctly, but it still sent multiple e-mails.
Blocked the ip range a bit wider for now which seems to help, but the problem itself is still there.

Edit: Fixed it for now with a regex filter for GLOBAL_RCPT_BL

Still, the problems persits.

Jan  4 19:17:15 a55be90510dc postfix/postscreen[1336]: CONNECT from [172.22.1.1]:51164 to [172.22.1.253]:25
Jan  4 19:17:15 a55be90510dc postfix/postscreen[1336]: WHITELISTED [172.22.1.1]:51164
Jan  4 19:17:15 a55be90510dc postfix/smtpd[1339]: connect from unknown[172.22.1.1]
Jan  4 19:17:15 a55be90510dc postfix/postscreen[1336]: CONNECT from [172.22.1.1]:51170 to [172.22.1.253]:25
Jan  4 19:17:15 a55be90510dc postfix/postscreen[1336]: WHITELISTED [172.22.1.1]:51170
Jan  4 19:17:15 a55be90510dc postfix/smtpd[1341]: connect from unknown[172.22.1.1]
Jan  4 19:17:16 a55be90510dc postfix/smtpd[1339]: 6D03E47C42: client=unknown[172.22.1.1]
Jan  4 19:17:16 a55be90510dc postfix/smtpd[1341]: 8C83347C44: client=unknown[172.22.1.1]
Jan  4 19:17:17 a55be90510dc postfix/cleanup[1343]: 6D03E47C42: message-id=<id@mydomain.tld>
Jan  4 19:17:17 a55be90510dc postfix/cleanup[1344]: 8C83347C44: message-id=<id@mydomain.tld>
Jan  4 19:17:18 a55be90510dc postfix/cleanup[1344]: 8C83347C44: milter-reject: END-OF-MESSAGE from unknown[172.22.1.1]: 5.7.1 Matched map: GLOBAL_RCPT_BL; from=<name@mydomain.tld> to=<name@qq.com> proto=ESMTP helo=<reunionisland.cool>
Jan  4 19:17:18 a55be90510dc postfix/cleanup[1343]: 6D03E47C42: milter-reject: END-OF-MESSAGE from unknown[172.22.1.1]: 5.7.1 Matched map: GLOBAL_RCPT_BL; from=<name@mydomain.tld> to=<name@qq.com> proto=ESMTP helo=<flavie-delangle.com>
Jan  4 19:17:19 a55be90510dc postfix/smtpd[1341]: disconnect from unknown[172.22.1.1] ehlo=1 mail=1 rcpt=1 data=0/1 commands=3/4
Jan  4 19:17:19 a55be90510dc postfix/smtpd[1339]: disconnect from unknown[172.22.1.1] ehlo=1 mail=1 rcpt=1 data=0/1 commands=3/4

There is no additional Software installed on the Server, and there are no SSH logins.
I’ll continue with tracing the traffic with tcpdump and take a look into it later…

Edit 2:
I took a look into the tcpdump file and noticed that there is still a lot of IPv6 traffic, which shouldn’t be there at all. I followed the documentation on how to disable ipv6 but it still somehow worked in the background. I disabled it on the host machine via kernel parameter + disabled it in the docker daemon config.
I’ll see if that works.