I understand that the question is old, but I had the same query. And since I have found the answer, I will post it here to benefit others.
All custom symbol values are kept in the file /var/lib/rspamd/rspamd_dynamic which can be found INSIDE the mailcow-rspamd container. You can erase the contents of the file (or truncate it), so that the default symbol values are applied.
Besides logging in the mailcow-rspamd container, you can also access this file via the mailcow_rspamd-vol-1 volume. In my case, it is located in the following path at the Docker HOST: /var/lib/docker/volumes/mailcow_rspamd-vol-1/_data