in order to program it over SWD you can't have it hooked up normally, so you need to have the SWD header provide vref, but that causes a backfeed to the rest of the board on the 3v3 net :v
So what i've done is use some FETs to make a little dynamic switch to prevent backfeed as a normal diode would drop too much voltage
Could be worse, but still, it's annoying that you can't do SWD normally with this part.