Don't think it has anything to do with my modem or router. I have a very fast internet. I've asked a friend of mine to join the server with me as well. He said the downloads are taking unusually long for him as well.

Thanks for all the help though! I really appreciate it