Sysctl for redis breaks on LXC/LCD

LinusCDE

Hi,
I know that LXC is not officially supported, but I still would like a to point out a change that causes problems on it.

I updated today (dockerized setup; ./update.sh), enabled Ipv6 and got this error when the stack was starting (this is a specific attempt, but the error was the same):

# docker-compose up -d redis-mailcow
Creating mailcow_redis-mailcow_1 ... error

ERROR: for mailcow_redis-mailcow_1  Cannot start service redis-mailcow: OCI runtime create failed: container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: write sysctl key net.core.somaxconn: open /proc/sys/net/core/somaxconn: no such file or directory: unknown

ERROR: for redis-mailcow  Cannot start service redis-mailcow: OCI runtime create failed: container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: write sysctl key net.core.somaxconn: open /proc/sys/net/core/somaxconn: no such file or directory: unknown
ERROR: Encountered errors while bringing up the project.

At first I thought that this is a fault of enabling ipv6 (it’s experimental BUT recommended?!?). After a lot of digging I found the culprit: The docker-compose now contains a setting for a sysctl:

    redis-mailcow:
      image: redis:6-alpine
      volumes:
        - redis-vol-1:/data/:Z
      restart: always
      ports:
        - "${REDIS_PORT:-127.0.0.1:7654}:6379"
      environment:
        - TZ=${TZ}
+      sysctls:
+        - net.core.somaxconn=4096
      networks:
        mailcow-network:
          ipv4_address: ${IPV4_NETWORK:-172.22.1}.249
          aliases:
            - redis

The problem is, that my host is running in an lxd container and this file isn’t available from within (the whole /proc/sys/net/core folder is actually empty). I had to comment this out for the time being.

The host has this already set to 4096 (but the lxd container can’t see that).

I’m running mailcow for about 1,5 years now (the server was even migrated 3 times and is now running almost a year in an lxc instance). So even if LXC/LXD isn’t officially supported. I’m happy to now start editing the docker-compose.yml after each update, but I just wanted to note that this change probably breaks some setups.

LinusCDE

Found the commit that added this: mailcow/mailcow-dockerized737b40a

LinusCDE

Just found the PR 4176.

From pquan’s reply to PR 4176:

It does not look nice to introduce a breaking change and limit the possible use cases of mailcow in general. And this change will likely break existing installations on update.

He seems to have predicted my excact problem. andryyy’s reply already stated that LXC seems to be unwelcome (and I didn’t know that LXD was unsupported until today). I personally find the decision to introduce a breaking change with this argument a bit weak. It also makes my dear docker look bad, since I should preserve software from those very problems. But well, he seemed to have ended this discussion already.

I’ll probably have to keep modifying the docker-compose.yml now for each update. I guess I’ll have now happy problems after each update, should I forget to change that.

EDIT: Is there some trivia as to why LXC/LCD isn’t supported (or even kinda hated it seems)? It seemed to run fine until that point.

ojsef39

Thank you very much! That’s the answer I googled for 🤣
Even though it’s sad, I can’t use LXC.

LinusCDE

Yeah. It’s an unfortunate issue. Still, I just removed it for every update up until a month ago. Now I have a new server and me and some friends who host it, don’t use lxc ourselves anymore anyway. So that is not an issue for me anymore.

Just remove the comments for the lines before an update, run ./update.sh --skip-start, comment the lines out again, and then finally start the containers up with docker-compose up -d.

Pointing that out, or point it out to a user on update (can be for example detected like so in shell scripts: [ "$(systemd-detect-virt)" = "lxc" ]) would surely be a helpful addition for users. Though that may probably cause issues with lxc officially not supported. But at least that could be pointed out with it, since I used mailcow on a lxc container for 2 years without noticing any issues.

ojsef39

Thank you very much for specifying what to do!

I found something about changing the container settings (in my case on proxmox) from an unprivileged container to a privileged one (on container creation uncheck “unprivileged”) and activating “nesting” afterwards in options.
Source: here.

I’m currently testing this solution, I can give an update here if it works 🙂

seidler2547

The easy way to work around it is to remove the sysctls in your docker-compose.override.yml:

version: '2.1'
services:
  redis-mailcow:
    sysctls: !reset []