This is entirely based on an assumption they cost too much and take too much time to rebuild.
Suppose it takes say, 5-10 minutes to flip via whatever the flip mechanic is (Destruction, kill all the guards, etc). Once you control the area it takes 1 minute and 200 wood to repair it, turning it...