50 down 10 up will be fine. If you're running addons with people, you need RAM. Lots of it. We are dedicating 2 gigs of ram i think 1333mhz per server and that works GREAT if you don't have mods running. If we stress the server. Like if we place 6000/10000 blocks at once (and yes, we are doing that now) it will fill up that RAM and dump the server's population.
Note that is VERY high stress for a server. But I would dedicate 6-8gigs per server. So get 8x2 or 4x4 1600mhz. That way you have enough for your server + OS + w/e else you want to do on your server.


Edit: the 50 down and 10 up will be more then enough, just remember, any personal downloads(game updates ect) will effect your server. So just schedule them wisly.