Yah I see your point.
Same time you gotta take in to account, right now, the server holds PvP, PvE, and crafters.
With a non-pvp server, human nature tells me all the non-pvp players will migrate off the pvp server.
And that's the biggest drawback for me.