If it's laggy after 4 people, you probably have a bad upload speed. Goto
http://www.dslreports.com and test your speed. With that upload number, you can determine what your maxrate is, which could make your server stable for about 6 people.
Goto dslreports, get your approximate upload speed from their tests there, and post it here, then i'll tell you how to set your maxrate.