Hey,
Okay so here are the details.

Server specs:
2.4 Quad Core Intel
8 GB RAM
7200RPM HD I believe (though it seems slow!)
100Mbit port (FDCServers)
Windows 2003 64Bit
Apache 2.2.x, PHP 5.x, MySQL 5.x

Web site info:
Was just a vBulletin forum and had a ChatBox with a refresh rate of 30 seconds, on one page (index) only. Thats it, nothing more.

Problem:
When I had my site hosted on this dedicated box, every time we got a lot of visitors, say ~300ish, we would start having connection problems to the web server. It was like a "lock out" on some people. The site would suddenly stop working, you would be at the "waiting for server response".

Once the number of people visiting the site reduced, the website would be fine again and running amazingly smooth.

So I had moved to a professionally hosted (shared hosting) server. However, we are still having issues there now, except this time its MySQL reaching the maximum, so I'm planning on going back or switching hosts or something.

Anyhow, in the mean while, I'm hosting downloads on the old dedicated box, however Apache doesn't seem to be able to handle it. Everytime the uploads speed reach 5MB/s or something or around there, Apache will clog up. Sometimes it even happens at only 1.5 or 2.5MB/s.

To fix this I have to just stop/start Apache and its done.
It is only Apache that is affected because everything else runs smoothly, even at 5MB/s upload, i can freely go around the server, play on a FPS server as well with no lag. Apache just clogs up

I need to know how to fix this. I need to know how to setup Apache correctly so I can avoid this. Right now on the dedicated box all I have is downloads, and its still taking place.