As you may have noticed, we’re working on an all new version of Basecamp called Basecamp 3. We want to build the fastest version of Basecamp yet, so we’ve been hard at work on optimizing performance. One of the things we’ll be doing to improve both performance and reliability is to serve Basecamp 3 from two datacenters: one in Chicago, IL, where our other applications are all located, and our second site in Ashburn, VA that we’ve brought online over the past couple of years. We recently optimized how traffic reaches these datacenters by using Anycast BGP.
Here’s how it works in some more detail.
When you make a request for a page in Basecamp 3, that request leaves your computer, goes through your home or office Internet router, and ends up in your internet service provider’s network, whether that’s Comcast, Google Fiber, or your mobile phone carrier. In most cases, your ISP will send that request along to another upstream ISP (though a process called peering), eventually reaching one of the networks that connect to our datacenters (XO and ServerCentral).
Once it reaches our providers, the ✨magic ✨ of anycast starts to happen. Because we advertise the IP address for Basecamp 3 as being served from either datacenter, the provider can choose to send that traffic in either direction, so it looks for the lowest “cost” (fastest) path to get that traffic delivered. When it picks the shortest route, you win: you get to the datacenter that’s closest to you in Internet terms, which may or may not actually be physically closer to you.
We previously utilized a DNS service to send traffic to both data centers for Basecamp 3 and our other multiple-datacenter applications. Below you can see the traffic for one of our other applications over the last two days. Initially traffic was split evenly between Chicago and Ashburn. After the change was pushed yesterday, Chicago started seeing progressively more traffic as the day went on and our traffic mix shifted further to customers on the west coast. This morning, Ashburn has seen the bulk of the traffic since it is closer to customers in Europe and the east coast.
Anycast routing provides some other benefits as well. In contrast with modifying DNS records (even with very low TTLs) changes to anycast routes result in a much shorter “long tail” of traffic hitting the datacenter that has been removed from production service.
Compare the above graph of a DNS based change with the graph below showing anycast route changes :
You can see that traffic shifts happen much faster using anycast.
David
on 11 Sep 15Base camp
No love for Europe or Asia in having a data center there?
On a similar topic, how are you keeping data in synch between the data center?
Dan
on 11 Sep 15Off topic: Eron, private pilot, eh? What do you fly? (I fly an RV-10, great stuff).
Taylor
on 11 Sep 15@David Stay tuned. We’re doing everything we can reasonably do to deliver the best Basecamp experience no matter where you are in the world.
We’ll make a separate post about how we are keeping the data in sync, but it’s nothing magic. Just replicating data stores and do a little bit of massaging to avoid situations where an update at one dc could cause a stale data read or write at the other.
Eron
on 12 Sep 15@Dan I mostly fly a Piper Cherokee, occasionally a Cessna 172. I’d love to get into an RV some day, great planes!
This discussion is closed.