Honestly if it comes down to it just being able to determine the layout of your city, the usuage of terrain, having the banks there, being able to log in reinforcements, being able to use NPC guards, use stables to replenish pets are ALL big advantages to the defender. Without any walls in that...