I managed to get it working with IPv6 mangle.
I seem to need 3x mangle rules, preroute mark connection, preroute mark routing, and forward mark routing. I also needed to disable fasttrack for marked connections.
The downside is that there seems to be a performance hit, as mangle is (I guess?) more resource intensive than simple routing rules.
ETA: It seems the mark routing preroute wants an interface (in my case, a VLAN), although my use case is a little funky as I am routing this traffic via WireGuard running on an AP, as the AP CPU is way beefier than my router.