You’re reading Signal v. Noise, a publication about the web by Basecamp since 1999. Happy .

Signal vs. Noise: Programming

Our Most Recent Posts on Programming

Automating with convention: Introducing sub

Nick wrote this on 21 comments

When I started my on-call shifts, we had pretty little in the way of automation for day-to-day issues. Tasks like SSH’ing into our cluster, starting a Rails console, or doing a deep search through our gigantic mail directories, were either shelved away in someone’s bashrc, history log, or just ingrained into someone’s memory. This pain was also felt by a few other of my fellow programmers, and we started cobbling together a Git repo simply named “37s shell scripts”.

This repo started very innocently: a little Ruby script named console that mapped a product name (basecamp) to a server name inside of our cluster (jobs-03), SSH’d in, and then ran a production Rails console. Several more bash and Ruby scripts started to trickle in as we started to share more of our personal code that we used when on-call. Eventually Sam laid down a foundation of bash scripts and directories borrowed from rbenv, and dubbed it “37”.


Running beta in production

David wrote this on 22 comments

We’re constantly working on new features, improvements, and technical upgrades for Basecamp. Many of these changes need to be experienced in the wild to guide their evolution. We need to live with them in our daily use of Basecamp to see whether what seemed like a good idea is actually a good idea.

To this end, we run six different beta servers that all point to the same production database. We’ve found it impossible to accurately evaluate a feature unless it’s being used in anger with real data that actually matters. Evaluating changes against a staging server that’s running an old copy of the database just doesn’t cut it.

The way we book a beta server for use with a certain branch is simple: It’s just in the title of the BCX Campfire room.

When you’re done with a beta server, you change the title to “available”. When you need a beta server, you change it to the name of the running branch.

To select a given beta server, we’ve added a drop-down to all 37signals’ staff accounts within Basecamp itself.

You just pick the server you want to run on and it’ll route you to the beta environment running the branch you’re looking to try out.

Long-term exposure to upcoming features is a great way to get them just right. But it’s also a great way to realize that what you thought was a great idea just wasn’t good enough to ship. We’ve killed many features and changes to Basecamp after living with them for a few weeks. Often times, the most important features are the ones you don’t ship.

Clarity over brevity in variable and method names

David wrote this on 66 comments

Many programmers have a natural preference for short variable and method names. I doubt many would recognize this preference as a trade of brevity for clarity, but that’s often exactly the result. This is especially true if you subscribe to the ridiculous Church of 80-character Lines.

It need not be that way. Writing terse code can be a joy even if you spell things out in abundant detail. Modern programming languages are expressive enough that what you save in laborious boilerplate can be spent on clarity — and you’ll still have plenty of lines left over for a dance.

And most certainly, you’ll hardly ever need to abbreviate anything. I cringe when I see ext for extension, cp for copy, or worse, application-specific abbreviations sure to be forgotten two months after you left the project.

At times being exceedingly clear will seem almost silly at first glance. The name of the method or variable can be longer than the operation being performed! But the silliness quickly dissipates the first time you return to a piece of code and know exactly what it does.

Here are a few examples of long method names from the new Basecamp code base:

def make_person_an_outside_subscriber_if_all_accesses_revoked
  person.update_attribute(:outside_subscriber, true) if person.reload.accesses.blank?

def shift_records_upward_starting_at(position)
  positioned_records.update_all "position = position - 1",
    ["position >= ?", position]

def someone_else_just_finished_writing?(document)
  if event = document.current_version_event
    !event.by_current_creator? and event.updated_at > 1.minute.ago

If you work hard at being clear in your naming, you’ll also rarely need to write comments for the code. Comments are generally only needed when you failed to be clear enough in naming. Treat them as a code smell.

“The UNIX System: Making Computers More Productive”, from 1982. Brian Kernighan, Dennis Ritchie, and Ken Thompson talk about what UNIX is, how to use it, and show off some live coding too. Thanks to Tony Arcieri for sharing this!

Go code spelunking with me in the all new Basecamp! You’ll see how we build Rails apps at 37signals and learn several exciting Ruby and JavaScript patterns that you can use. There’s also a writeup of some of the best chunks from the talk to check out.

Levels of aspiration

David wrote this on 31 comments

Debates over technology, technique, and process often go nowhere because the participants are arguing from different levels of aspiration.

You’re unlikely to convince someone they should switch to programming Ruby for its beauty, if they’re merely looking to make a living as a single consultant serving local businesses in Schaumburg, Illinois.

Questions such as “does this run on my existing web host?” or “will my clients want something their nephew web designer hasn’t even heard of?” matter far more. Their aspirations are local, finding something that (sorta) works, and getting paid.

It’s easy to snub your nose at that if you have grander aspirations, but there’s a lot of software to be written in this world, and we need all sizes and shapes to get it done.

The third largest landscaper in Schaumburg is probably operating from the same level of aspiration as our single consultant, so there’s a great fit.

You can think of these levels of aspiration in terms of both geographical and technical hierarchies. Here’s a take on what a hierarchy of technical aspirations could look like for programming:

You can imagine a similar hierarchy for the geographical angle with the levels being Local, City, Regional, National, Global.

The key to a fruitful debate is to first establish at what level the person you’re talking to resides, and then use arguments from the next level up. But if they’re entirely content with where they are, then it’s probably better to live and let live.

To help someone move up the hierarchies, they have to have an intrinsic desire to do so. Arguments like “but it works” or “it gets the job done” are tell-tale signs of someone happy at the lowest level of the technical hierarchy and your cue to just quietly back out of the debate.

Announcing Pow 0.4.0 with support

Sam Stephenson
Sam Stephenson wrote this on 34 comments

We’ve just released version 0.4.0 of Pow, our zero-configuration web server for Rails development on OS X.

There are several new features in this release, including port proxying and better support for zsh users, but my favorite is a tiny addition that makes a huge difference when testing your apps on mobile devices.

Pow has always made it easy to access Rails apps on your computer with its built-in .dev domain. Just symlink your app into the ~/.pow directory and visit in your browser.

But what about testing your apps on mobile devices, or in IE? Pow’s .dev domain only works on your local machine.

Until now, testing on other computers required modifying /etc/hosts or setting up a custom DNS server on your router. Now we’ve fixed that, too.

Introducing, the magic domain name

Pow 0.4.0 has built-in support for, a free service from 37signals that provides wildcard DNS for any IP address.

With you can access your Rails apps from devices on your local network, like iPads, iPhones, Windows VMs, and other computers. No configuration required.

Say your development computer’s LAN IP address is With the new version of Pow, you can now access your app at And supports wildcard DNS, so any and all subdomains of resolve too.

Read more about at or check out the full source code on GitHub.

Installing and upgrading

See the full 0.4.0 release notes and install or upgrade with one simple command from your terminal:

curl | sh

As always, the user’s manual and annotated source code are available for your perusal.

Keynote by David from RailsConf 2012 on progress, curiosity, fear, and the danger of easy-bake ovens.

Code Spelunking in the all new Basecamp

Nick wrote this on 14 comments

Over the past few months I’ve learned an immense amount from the all new Basecamp codebase. This year for RailsConf, I was able to show off some new (and old!) patterns in the codebase along with some neat libraries that we’ve put to serious use for our customers. The slides are over at SpeakerDeck if you’d like to see them all, and hopefully a video of the presentation will be up soon!

Here’s a few choice sections from the talk, and plenty of great links to check out in the meantime:


The On-Call Programmer

Nick wrote this on 39 comments

My dad is a firefighter and fire captain in Niagara Falls, NY. When I told him I had on-call duty for my new job, he was beyond excited. After relaying stories of waking in the middle of the night to head into the hall and getting overtime from his buddies that didn’t want to wake up for work were sick, I had to explain that it’s a different type of action for me. I face issues of a less life-threatening variety, but there’s still plenty of virtual fires to extinguish. Here’s a primer for what being an on-call programmer means at 37signals.

The routine

On-call programmers are rotated around every 2 weeks, and all 8 programmers currently get their fair share of customer support and interaction. Usually we have 2 to 3 programmers on-call on a given week. Support issues from email and Twitter are routed through Assistly, and handled by our awesome support team.

If there’s an issue they can’t figure out, we get a ping in our On Call room in Campfire to check out a certain issue:

These issues range in severity, and they might look like:


Testing like the TSA

David wrote this on 113 comments

When developers first discover the wonders of test-driven development, it’s like gaining entrance to a new and better world with less stress and insecurity. It truly is a wonderful experience well worth celebrating. But internalizing the benefits of testing is only the first step to enlightenment. Knowing what not to test is the harder part of the lesson.

While as a beginner you shouldn’t worry much about what not to test on day one, you better start picking it up by day two. Humans are creatures of habit, and if you start forming bad habits of over-testing early on, it will be hard to shake later. And shake them you must.

“But what’s the harm in over-testing, Phil, don’t you want your code to be safe? If we catch just one bug from entering production, isn’t it worth it?”. Fuck no it ain’t, and don’t call me Phil. This line of argument is how we got the TSA, and how they squandered billions fondling balls and confiscating nail clippers.

Tests aren’t free (they cost a buck o’five)

Every line of code you write has a cost. It takes time to write it, it takes time to update it, and it takes time to read and understand it. Thus it follows that the benefit derived must be greater than the cost to make it. In the case of over-testing, that’s by definition not the case.

Think of it like this: What’s the cost to prevent a bug? If it takes you 1,000 lines of validation testing to catch the one time Bob accidentally removed the validates_presence_of :name declaration, was it worth it? Of course not (yes, yes, if you were working on an airport control system for launching rockets to Mars and the rockets would hit the White House if they weren’t scheduled with a name, you can test it—but you aren’t, so forget it).

The problem with calling out over-testing is that it’s hard to boil down to a catchy phrase. There’s nothing succinct like test-first, red-green, or other sexy terms that helped propel test-driven development to its rightful place on the center stage. Testing just what’s useful takes nuance, experience, and dozens of fine-grained heuristics.

Seven don’ts of testing

But while all that nuance might have a place in a two-hour dinner conversation with enlightened participants, not so much in a blog post. So let me firebomb the debate with the following list of nuance-less opinions about testing your typical Rails application:

  1. Don’t aim for 100% coverage.
  2. Code-to-test ratios above 1:2 is a smell, above 1:3 is a stink.
  3. You’re probably doing it wrong if testing is taking more than 1/3 of your time. You’re definitely doing it wrong if it’s taking up more than half.
  4. Don’t test standard Active Record associations, validations, or scopes.
  5. Reserve integration testing for issues arising from the integration of separate elements (aka don’t integration test things that can be unit tested instead).
  6. Don’t use Cucumber unless you live in the magic kingdom of non-programmers-writing-tests (and send me a bottle of fairy dust if you’re there!)
  7. Don’t force yourself to test-first every controller, model, and view (my ratio is typically 20% test-first, 80% test-after).

Given all the hundreds of books we’ve seen on how to get started on test-driven development, I wish there’d be just one or two that’d focus on how to tame the beast. There’s a lot of subtlety in figuring out what’s worth testing that’s lost when everyone is focusing on the same bowling or bacon examples of how to test.

But first things first. We must collectively decide that the TSA-style of testing, the coverage theater of quality, is discredited before we can move forward. Very few applications operate at a level of criticality that warrant testing everything.

In the wise words of Kent Beck, the man who deserves the most credit for popularizing test-driven development:

I get paid for code that works, not for tests, so my philosophy is to test as little as possible to reach a given level of confidence (I suspect this level of confidence is high compared to industry standards, but that could just be hubris). If I don’t typically make a kind of mistake (like setting the wrong variables in a constructor), I don’t test for it.