I am sorry, I mixed that up, you are right, you cannot switch both hosts.
Anyways, it syncs both hosts, and therefore it syncs in both directions by default.
So there is no need to switch the 2 hosts.
Thats why its called imapsync, and not imapcopy 😉