I solved the problem on my own and thought it would be useful to share here with you.
So first of all, I forgot about the fact that I am running a nginx reverse proxy outside mailcow-dockerized natively on my host system. This is because I have other services on my system that should also be accessible over HTTPS. In order to serve multiple websites listening on different domains/servernames, a centralized nginx reverse proxy is necessary.
Now to the point with the autoconfiguration of Thunderbird. Wenn you add a new mailbox to Thunderbird, it tries to get the server configuration via HTTPS from a webserver listening on the domain of the mailbox you want to add to Thunderbird. If you want to add mail@second.org, Thunderbird tries to get https://second.org/.well-known/autoconfig/mail/config-v1.1.xml. This file contains the parameters for protocols, ports and the mailserver Thunderbird should use. You can see all web requests from Thunderbird in the error console of it.
However, if you only have a nginx reverse proxy configuration exactly like this: https://docs.mailcow.email/post_installation/firststeps-rp/#nginx with just one SSL certificate for the mailserver domain, this would require the SSL certificate to contain all hosted mail domains as SAN, because it would be only valid for the mailserver domain (in this case example.org). The autoconfiguration would only work for the mailserver domain with no exception in Thunderbird, because the SSL certificate would be only valid for it. For the mailbox mail@example.org the resource https://example.org/.well-known/autoconfig/mail/config-v1.1.xml would be requested for autoconfiguration and the certificate would be valid, because its issued for (*.)example.org.
Solution 1: don’t use a reverse proxy but the nginx that comes shipped with mailcow-dockerized instead. In addition with the acme-container, it will request separate certificates for each domain for access to Mailcow-UI, SOGo, autoconfiguration files, calender paths, etc. [Haven’t tried this myself yet]
Solution 2: setup Traefik like stated here https://docs.mailcow.email/post_installation/firststeps-rp/#traefik-v2-community-supported. [Haven’t tried this myself yet]
Solution 3: serve the xml-file separately and statically with the nginx reverse proxy for each domain you added to your mailserver with corresponding servernames and different certificates. This means one nginx site for example.org with a SSL certificate issued for example.org, one nginx site for second.org with a SSL certificate issued for second.org etc.
The third solution is what I came up with, because it limits HTTPS access to only the xml-file needed for Thunderbird autoconfiguration, which I find convenient. I added access for SOGo with another nginx site listening under a specific servername and Mailcow-UI access with yet another nginx site, which is only accessible over a secured VPN tunnel.
If you have questions, remarks or you found this helpful, feel free to leave a reply here.