True, I don’t think scripture really tries to define the size of a congregation, but it does very strongly imply order, structure and unity. I don’t see how that can be accomplished with simply two or three. For sure, where ever two or three believers are, the church is there, and, as a matter of fact, where ever one believer is, the church is there, but, like the bodies we dwell in, so the church dwells in a physical body which is the institution or organization. 

Just as we live in bodies damaged by the fall, so the institutions in which we find the church are imperfect and marred by sin. And just as some wish to disown their flesh put on a spiritual body now, others wish to disown the structure of the church, the earthly Jerusalem, and create a Heavenly one now. But that is not going to happen until Christ returns. Thus we remain pilgrims awaiting our King’s return. In the mean time we need to put aside petty issues and try to find a place where we can grow spiritually and be a blessing. 

If someone feels a call from God to start a new church, I am not going to fault that. Neither am I going to fault someone who’s conscience is troubled by the sorts of evil things their church might be engaged in to the point that they leave and join another congregation. But the constant desire to escape from “organized religion” troubles me greatly because the pattern set in the New Testament itself shows order, authority, complexity, all the traits that these people are trying to avoid in the name of finding true fellowship. I think they are fooling themselves in this regard.