I think the real problem isn't with the pragmatic aspects of scaling, but with sociocultural and interpersonal issues.
What do you do in a small commune when you eventually have 2 people who can't stand each other, but haven't committed any offenses that would justify removing one of them, and neither is willing to voluntarily give up the home they've built and leave? And what happens when that problem begins to spread?
Minority Report here we come...