Results 1 to 1 of 1
  1. #1
    Brian Livingston
    We've had a fantastic ride for the past two weeks. That's when the Lounge started getting visitors from the Windows Secrets Newsletter and search engines like Google.

    For years, the Lounge banned all search engines. That's because constant indexing by search crawlers like the Googlebot could slow the Lounge to, well, a crawl.

    On Dec. 2, 2009, we started allowing search engines to gradually include Lounge pages in search results. And we started notifying the 400,000 subscribers of the Windows Secrets Newsletter of the Lounge's existence. Every subscriber received an e-mail invitation between Dec. 2 and 16.

    We had just a smidge over 20,000 members on Dec. 1. By Dec. 16, that had doubled to more than 40,000 members.

    As of this moment, I see that almost 50,000 people are now registered members of the Lounge, and it's still growing. That means we'll certainly have tripled our membership within the next few weeks. You can always see the number for yourself on the Lounge Lobby page. Look under near the bottom of the page for the Total members count.

    This growth is good, and we implemented on Dec. 22 faster server resources to handle the numbers that are coming.

    Starting with the Windows Secrets Newsletter on Jan. 7, 2010, and every Thursday after that, the newsletter in each issue will promote the Lounge to its entire list of 400,000 subscribers. There's no way to spread across an entire week the newsletter's delivery. Every subscriber wants it (and deserves to receive it) on Thursday. Our site, not counting the Lounge, receives six times more traffic on Thursdays than on other days of the week.

    To handle the peak in traffic without making the Lounge extra-slow on Thursdays, we converted our virtual server to a scalable system on Dec. 22. Like most Web sites, our server during most of 2009 was just a single machine. Now, whenever we publish a newsletter — or any time there's a surge of traffic from or whatever — our new, scalable server will duplicate itself. This means the first server will spawn two, three, four or more identical machines. When the traffic goes down, the extra instances will go away.

    Please see the image below (courtesy of, which makes this scalability possible). Our new server cluster is similar to (but not identical to) the configuration shown. In the diagram, App Server 1 is the machine that will run seven days a week. On Thursdays, it will spawn App Server 2, App Server 3, and so forth.


    App Server 1, the master application server, will hold the Lounge's HTML code, uploads, and other material that seldom changes. Our master database server, running the MySQL database, will hold Lounge posts and other content that changes from moment to moment.

    Placing the database server on a different virtual instance from the master app server gives the Lounge better performance. Scaling the app server to spin up multiple instances will keep the Lounge's performance acceptable even during the heaviest peaks.

    Why wouldn't we create 2, 3, or more database servers during times of peak demand? The reason is that a database server cannot easily be duplicated in real time. Entire books have been written about the synchronization problems of running multiple database servers that must be in constant communication. For this reason, network engineers recommend that you get the largest machine you'll ever need for the master database server. That server transmits each transaction to a slave database server, called a replication server, which records every detail. Finally, both servers back themselves up to Amazon S3 (the Simple Storage Service).

    Our replication server resides in a different so-called Availability Zone than the master database server. Amazon Web Services provides different data centers (Availability Zones), each of which has its own power supplies and other resources. If our master app server and master database server are in a data center that goes down — and it's happened before to AWS — our replication server can immediately be promoted from slave to master. New app servers will also spin up, and our load balancers will direct visitors to the second, operating data center. Visitors should experience delays only for a few minutes, not hours or days as with previous data center outages we've experienced.

    To design the architecture and monitor the resulting virtual server cluser around the clock, Windows Secrets contracted with This firm specializes in 24-hour operations for companies with major Web presences. Its network engineers have designed solutions for,, the Huffington Post, and many other well-known sites.

    How can we afford all of this service? Easy. Windows Secrets has been paying up to $1,500 per month for a colocated server at a data center in Seattle. By contrast, the cost for a "medium" virtual application server (on a 3-year reserved contract at AWS) is only $44 per month. A "large" instance to run the database server is only $88 per month. There are also storage and bandwidth charges, but these are a tiny fraction of what we've been paying. AWS currently charges $0.10 per GB of storage per month plus $0.17 per GB of outbound data transfer. After scalability is working for the Lounge, the parent site of itself will soon move to the cloud as well. The savings will be substantial.

    To manage the load on the servers even better, we streamlined some of the Lounge's operations. We reduced or minimized extra-large images and extra-long user signatures (sigs can now be up to 4 lines instead of unlimited). But you will always be able to maximize your viewing of user-provided personal information. You can see any member's collection of images, signatures, and interests by visiting the member's Profile page. To do this, click the member's display name atop any post. You can also control the display of images when you're signed in to the Lounge by clicking the Settings menu and selecting the Forums tab. Under the heading Lounge display settings, configure your display of posts any way you like them.

    I hope you'll enjoy using the Lounge on our new virtual server cluster. The best part is, you don't need to understand any of the above! The Lounge should work the way it did before, and continue to work well, even during the heaviest times of the week.

    Please don't reply to this post, which is only an announcement. If you have any concerns or information to share with us, please notify us using the "Contact an administrator" link at the bottom of the page. Thanks!
    Attached Images Attached Images

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts