As some may have seen over the past few weeks, ServeTheHome has gotten relatively hammered. As a result, the site which was running perfectly well in June 2011 was seeing stability issues in late August. I decided to take the opportunity to see if I could speed things up a bit.
Fixing the Problem – Using More Resources
The first big change was moving to a new Virtual Private Server (VPS) with more CPU/ RAM. I noticed some spiking memory usage which was giving users sporadic 500 and 404 errors. I will note that although the change has been made, I am still thinking strongly of just hosting this in Amazon EC2 so scaling will not be an issue going forward.
As many note, unlike many tech sites that try increasing page views by splitting one or two pages of information into six to twelve, I like keeping pages a bit larger but requiring just scrolling and no clicking. Maybe it is using my MacBook Air or my iPad regularly both of which do great scrolling but where clicking is not as precise. One thing a lot of users have noted is that page times, especially with the pictures I use tend to suffer. This was a bit more complex to fix.
My first thought was bigger, better VPS would mean better performance. It did to some extent, but WordPress started to regularly run away with memory usage and start crashing. After about a week of nightly Google searches, I had a few fixes in-place that fixed the issues (hopefully but it seems to be working.) The additional server resources meant anywhere from 5-15% faster page loads.
My next thought is that locating my site in California, USA and also being in California (albeit about 400 miles away from the servers), meant that I was probably seeing close to a best-case load times, and they were not all that great. The next step was moving a lot of media into Amazon’s cloud and making available from more locations. You will see things like media.servethehome.com in the status bar when images are being served from the cloud. In initial testing, this is making a much bigger impact than the bigger virtual machine.
Next Steps
Now that this is all working decently I am going to request that anyone that sees images not loading or something wrong either use the contact forum to submit a bug or feel free to post on this forum thread.
I think the next big infrastructure improvement will be moving the remainder of the site to an Amazon EC2 instance. I really do not want to deal with scaling issues in the near future so I can get back to making more content.
Fun Facts
- According to current projections, the faster page loads will save users approximately 200,000 seconds each month! That is well over two days worth of time savings each month.
- ServeTheHome.com’s predecessor was hosted on a Windows Home Server (32-bit) WAMP installation. Total duration of this endeavor was under 36 hours.
- STH was originally hosted on a shared server. This worked fairly well, but as the site grew throughout 2010 and into 2011 (2009 growth was minor at best), this did not work.
- I have now tested no fewer than fifteen virtual private server providers for the site and am marginally happy with the current setup.
- Approximately every 60-90 days I build out a dedicated server for STH. In fact, when I bought the Supermicro SC216E1 chassis that I eventually reviewed, the goal was to turn that into the enclosure for the STH web server hardware. At the end of the day, I always come to the conclusion that I do not want to be responsible for changing hardware in a remote location in the middle of the night.
Thanks everyone for making this site a bunch of fun to maintain and a useful resource to many! I will still look into making the site faster, since I know STH is still far from great in terms of loading speed, but hopefully the latest re-configuration will help a lot.
Your web site is inspiring, and informative. It has become my favorite site lately. It is always up to date with new and interesting articles. Thanks for you effort and dedication. We may not say much, but you effort is noticed and appreciated.
I feel your pain. Virtual is now the way to go.
I put my first domain online in 1995. It was hosted on a glorified desktop that I snuck into the rack at the colo where my employer has its equipment.
It lived there until I decided to host my first viral video… the original South Park Christmas video. It was something like 50mb and this was pre-YouTube. After I linked the video in a Slashdot comment thread about the video, the entire ISP – which was served by a just pair of T1s – ground to a halt.
Hands slapped, I bought my own hosting. For $75 a month, I was online. That was back in the days before a rack-mount server was required. Most of the servers in the original colo were not rack-mount.
In 2001, I had to give up the desktop because the ISP was moving facilities and I would be charged per U instead of a flat rate per connection. I bought a Dell Poweredge 350 – Celeron 850mHz, 256MB of RAM and 40GB of RAID storage. A couple years later, I doubled the RAM to 512MB. That little server is still online. It moves more than half a million messages a month, hosts DNS and a few static web pages. I added a beefy 2U server for a few dozen WordPress blogs but the core of my infrastructure and web empire is that ten-year-old, underpowered Dell.
Last month, I dumped the 2U server in favor of hosted virtual server on the XenServer platform. I get massive amounts of both bandwidth and hardware at roughly the same price as I was paying for the 2U server. Best of all, since I’m using XenServer, I can pack up my virtual machine and pretty much move it anywhere (or everywhere) fairly easily.
While I’m still hesitant to move my mail and DNS into the cloud – and lose a server that has been rock solid for more than a decade – I must say that the hosted web sites are doing fine. (You can click the above link to see a picture of the server if you’re into that sort of thing.)
For a site that gets as much traffic as yours, unless building and maintaining servers is your full-time job and you like the round-trip to the colo, you are much better going into the cloud. I find the cloud products from Amazon and Rackspace intimidating and cost-prohibitive for my needs so I went with a local company that offers virtual hosting. Being able to call an engineer for advice and the occasionally hand-holding is a feature of the local guy that I can’t get from a national company at my price point.
Good luck!
Cheers,
Matt
Have you looked into using Nginx instead of Apache? It handles static content much better than Apache, as well it’s lighter on ram. Bundle that with php-fpm and you have a knockout combo.
That is something that I am looking into when swapping to Amazon S3 in the near future. Have any good resources on it that you like? I have basically used IIS and Apache over the past decade or so.
The original script for easy Nginx + PHP + MySQL setup was here:
http://www.lowendbox.com/blog/wordpress-cheap-vps-lowendscript/
An updated script with newer software is here:
https://github.com/Xeoncross/lowendscript
It uses dotdeb repos (or allows you to). Dotdeb brings backported webserver stuff to stable Debian easy and quick without fuss.
Here is another modified script:
https://github.com/maxexcloo/Server
You can check out my blog as well. It is using nginx too.