Web Server Upgrade, Part 3

Much has been happening since I last wrote to you about the web server upgrade. If you want to read up on what's been happening, here's Part 1 and Part 2.

A quick recap

Due to an increased in the amount of visitors hitting the infopackets web site, I had to upgrade the web server. I am now leasing a lightening fast Pentium 4 1.7 GHz Apache Web Server with 512 Meg DDR RAM a 60 gig hard drive. My web server package includes a "T1 Connection" which allots a continuous flow of 1.5 megabits of bandwidth per second at any given moment.

The saga continues!

Since the server has been upgraded, the infopackets web site has hit a few bumps along the way -- nothing major. At least, not until the last 72 hours, anyway.

I just received an email today from my hosting company which stated that some customers (myself included) have been experiencing "substantial packet loss". In order to alleviate this problem, the web hosting company opted to introduce another OC3 Fiber Optic connection, capable of 155 megabits per second of bandwidth.

That's a lot of bandwidth.

The implementation of the OC3 required major hardware re-configuration changes and optimizations. For the last 3 days, service for the infopackets web site has been interrupted while the hosting company made their move.

When the OC3 connection was put in place, the hosting company also added a new switch*. To further alleviate network congestion, the switch was configured to limit bandwidth which could flow to and from the infopackets web server. A simple representation of the setup looks something like this:

The infopackets web server -> switch -> OC3 connection (Internet)

Side note: A switch is very similar to a hub, but offers greater control over how information is delivered over a network. For example, a network hub shares it's bandwidth equally among all ports, whereas a switch can give exclusive bandwidth to a single port. For a graphical representation of the two major differences between the two, click here.

Recall that the infopackets web server is capable of 1.5 megabits per second bandwidth at any given moment.

In theory, the amount of bandwidth that the web server could transfer in one month (non-stop) would be calculated like this: 1.5 megabits per second * 60 seconds in a minute * 60 minutes in an hour * 24 hours in a day * 31 days in a month 474.609 gigabytes of information per month, non-stop.

In actuality, the web server is connected to a local area network (LAN) which operates at 100 megabits per second; the LAN is then connected to the OC3 connection, which is connected to the rest of the Internet. Speed wise, the setup looks something like this:Web server (100 meg per second) -> switch (100 meg per second) -> OC3 (155 meg per second)

So, where does the 1.5 megabit per second cap come into play?

From my understanding, the 1.5 megabit per second limit which has been allotted to my server was recently being capped at the switch.

The apache web server which controls the infopackets web site was configured to run at LAN network speed. Requests for file transfers and information from the infopackets web site were being delivered at a full 100 megabits per second, but then were capped at 1.5 megabits per second before reaching the OC3 gateway to the Internet. Effectively, it was causing a major traffic jam and caused a huge lag in server response today.

The good news

Ian Wilson, a computer technician/programmer at the server company, wrote his own program which splits file transfers and page requests by the number of users who are requesting information from the infopackets web server. This computation is done at the server level (via the Apache web server software) and divides requests and bandwidth evenly, before it reaches the limit set by the switch.

Ian's algorithm might look something like this: Server bandwidth limit = 1.5 megabits per second. 1 user connects to web server --> give this user 1.5 megabits per second for file transfers and page requests; 2 users connected to the web server --> divide bandwidth by two and deliver file transfers and page requests; when transfers are complete, allot full bandwidth to next request, or continue to divide transfers.

... and so on.

Very cool

I'm not a hard-core networking guru, but that sure impressed me when Ian Wilson told me about his program. While I was talking on the phone, I mentioned to Ian that I also enjoy writing my own programs to automate the web site, including my latest feat which automates purchases @ the infopackets web site and rewards users 25% for purchases over $30.00.

He was impressed.

Long story short

If you had trouble accessing the web site in the few days, click here to catch up on some past issues of the newsletter. For anyone who made a purchase online and was not able to enter an unlock code or download their purchases, you can safely do so now (at least, that's what Ian tells me).