I had my first major run-in with social media in June 2005. Slashdot had picked up a controversial story I had just written and it was flung onto the front page.

Though it was a huge success in terms of getting attention and new readers to my site, it was a complete disaster for my host. My site, then hosted on a regular shared hosting with Westhost, crumbled. The thousands of people Slashdot was sending to my site were greeted with a not-so-friendly error page, and if it hadn’t been for one user posting the article in the comments, most Slashdotters would never have been able to read it.

Eventually I was able to work with Westhost and get the site restored while some of the flood was still ongoing. They told me that, if I saw the potential for another spike in traffic to warn them and they would help. I agreed to let the incident go and simply waited for the next opportunity.

However, I didn’t have to wait long.

My First Digg Homepage

In December of the same year I noticed that I was getting a growing amount of traffic from Digg. One of my stories was making its way through the upcoming stories and seemed likely to break through to the front page. Having warning, I called Westhost to have them prepare. They upped the RAM my account could use and did everything they said was possible but warned me it probably wouldn’t be enough.

An hour later, the story hit the front page and my site promptly went down.

This time there was nothing that my host or I could do about it. The traffic was too great and, even using a caching plugin, the server could not handle the load. All I could do was wait for the traffic to slack off enough to the point that my site would come back online.

Which is precisely what it did, the next day.

Take Two

It was at this point I left Westhost. I realized that, if I wanted to survive sudden traffic spikes I needed to change strategies. Lured by the promise of a resolution, I jumped ship to Media Temple and their grid hosting service.

The service got a major test in May of 2007 when yet another article of mine began to tick up to the front page of Digg. This happened to hit right at the same time as the Digg user revolt so there was a lot of attention on the site.

When the article hit, Media Temple did its job. The site stayed online (after bursting out) and there was no downtime reported. However, several people did complain that the site moved slower during this period of increased traffic and the spike did end up adding a small amount to my bill for the month. A second spike would have been very expensive.

So, while I was happy with Media Temple in every regard, I decided that I needed to seek out another solution, eventually settling on a VPS with Servint after several short-lived deals with other VPS hosts.

However, the new approach didn’t get a stress test for quite some time. Wary of sites like Digg and Reddit, I had shifted my marketing to other social news sites like Twitter and StumbleUpon. They send great deals of traffic, but over longer periods of time- that helps to avoid the spikes and build longer-term readers. This strategy worked remarkably well but very recently I became very glad for my change to a full VPS service.

Third Time’s a Charm

Last week, much to my surprise, an article of mine was generating a lot of traffic from Digg. Having been submitted by Msaleem, a well-known power user, it seemed almost destined to make it to the front page.

As the Diggs ticked up, I battened down the hatches. I double checked my cache settings and ensured the server was in good working order. Though it was going to hit on a weekend, I wanted to make sure that we were prepared for the inevitable spike.

That afternoon, just before I was due to head out for a concert, the article reached 111 Diggs and made it to the front page.

The site was none the worse for the wear. Response time was still very quick. Sever load increased, but never shot up to more than .65 and RAM usage never went critical. The site was responding, receiving comments, sending email and handling all of the other domains on the server without any trouble or interruption. A regular reader, not there because of the Digg, probably would not have known anything was going on.

As the site’s traffic waned over the next 24 hours, the server continued to hold steadfast. The only difference was that Pingdom reported a slight improvement in the site’s response time as the traffic tapered off, but the difference was small enough as to be minimal.

All in all, it wasn’t a very large Digg, about 4,000 visitors in the course of a few hours, but it could have been enough to shut down many hosting accounts, especially when combined with several mainstream media mentions of my story. My VPS, however, simply laughed off the rush.

Why VPS Survived the Digg Effect

The reason my VPS was able to survive so easily is because, quite simply, it could handle the load. Though it only has 512 MB of RAM, that is more than enough to weather a modest Digg storm (or any traffic spike) without any special techniques. As long as the server isn’t overloaded before the spike hits, it can likely handle a bump in traffic.

This was combined with the use of proper caching techniques. At the time I was using WP Super Cache (not W3 Total Cache as I was doing a benchmarking run with the former) and that certainly seemed to help. Though I didn’t engage lockdown, the site still loaded very quickly server load/memory usage remained well within tolerances. This ensured that end users were unaware that I was running any form of caching and kept all of my page’s dynamic elements working.

All in all though, there was no “secret” to surviving the Digg effect. The most important thing is to simply have a hosting account that can take the strain without buckling. Though caching and optimization can help, they are no substitute for having the right amount of power at your fingertips.

Bottom Line

Now that I’m certain my server can survive the load of a Digg effect, as well as other social media rushes, I’m debating readjusting my strategy to encourage the use of those sites more. However, I, like many others, am still uneasy about the long-term benefits of sites like Digg as they have almost no lasting effect on traffic or subscribers.

Still, it is always flattering to be Dugg and the traffic is also very welcome, especially for sites that run advertisements. So, it is best to be prepared for these spikes and be ready to greet them with open arms, regardless if it is in your strategy.

On that front, a VPS might be more expensive than a regular shared hosting account or even a grid one, but having real power without any gimmicks can make all of the difference in the world when social news discovers your front door.

Tags:   digg • Media Temple • reddit • servint • slashdot • social news • traffic spikes • vps • westhost