You'd be better off creating two host DRS groups (One for each building) so that you could create vm\host and vm\vm affinity\anti-affinity rules keeping them together or apart depending on what you need.
Read these two, they'll help you wrap your head around affinity and Fault Tolerance.
Affinity rules are easy to setup but you need to make sure they're setup properly, Affinity rules will let you break things like HA, depending on how you've got them setup your hosts won't evacuate when you put a host into maintenance mode, etc and rules aren't checked against each other.
