You’re reading Signal v. Noise, a publication about the web by Basecamp since 1999. Happy !

Signal v. Noise: Programming

Our Most Recent Posts on Programming

Humans need to know why

David wrote this on 17 comments

Just doing work without an eye on the big picture rarely satisfies anyone. We want to know what we’re doing matters and that means knowing why we’re doing it. This is true in all walks of life and certainly so in software development.

If you simply ask a programmer to implement a feature, but fail to share the purpose or the justification for said feature, all you get is a code monkey. Code monkeys merely translate requirements into code verbatim. There’s no room to trade concessions with the designers if I don’t know what you’re trying to do.

So not only is it demotivating not to know why, it’s also bad for productivity. Figuring out what the problem really is is how good programmers judo the problem: “Yes, what you’re proposing is one way to solve it, and that’ll take 50 hours. But we can solve the same underlying problem by doing this instead and that’ll only take 8 hours. How about it?”

Sharing the context shares ownership. Now we’re all working on solving the problem the best way we can. Not just laying the bricks in the order the big man tells us to.

Winning is the worst thing that can happen in Vegas

David wrote this on 30 comments

There’s nothing more appealing than thinking you’re just that good. Our brains are optimized for fondly remembering our successes and quietly downplaying our failures.

The casinos in Vegas are primed for this by making it relatively likely you’ll win something early on. By the time you realize that the slight edge of the house means over the long run it’ll win everything back and more, well, you’ve probably spent more than your first ATM stop and then some.

Future coding is a lot like playing the roulette. If you can guess where the requirements of tomorrow will land today, you’ve just scored the ultimate programmer’s prize: looking like a wizard. You saw the future and you were right! High five, brain!

That’s the almost irresistible draw of “flexibility”—often just a euphemism for building half or whole features before you know how they’re supposed to work or whether you need them. Just like in Vegas, the worst thing that can happen is to be right about this once in a while.

Because like the casinos, the house of program complexity will take all your little bets on the future and add them up until you just don’t know how this damn program got so deep into technical debt!

Running up the debt on your code is not just about the quick hacks and dirty commits you know you really should clean up (but just don’t). No, the far more insidious kind of debt is that acquired in the name of “flexibility”.

Every little premature extraction or abstraction, every little structure meant for “when we have more of this kind”, every little setting and configuration point for “things we might want to tweak in the future”. It all adds up, and at a dangerous rate.

The road to programming hell is paved with “best practices” applied too early. Very few abstractions, extractions, or architectures are inherently bad (although I’ll contend that a few are). But most of them are downright evil if applied before they’re called for.

To take the scare quotes off flexibility, you have to be willing to write no code at all. The “Simplest Thing That Could Possibly Work” and “You’re Just Not Going To Need It” are both sayings to help you get there.

This really does mean not a single parameter to a method that has not yet found a use. No needless public methods that might get reused later (not even for testing!)

The hardest—and yet most important—thing in the world of design is the conviction to say no.

The five programming books that meant most to me

David wrote this on 28 comments

There are so many programming books out there, but most focus on specific technologies and their half-life is incredibly short. Others focus on process or culture. Very few focus on the timeless principles of writing good code, period. The following five books had the biggest influence on my programming style and development:

  • Smalltalk Best Practice Patterns: Incredibly practical advice for what constitutes good OO code. It’s done in Smalltalk, but the principles are mostly universal. Probably my favorite nuts’n’bolts of programming design book. Very granular.
  • Refactoring: The fundamental before/after book. Here’s some code that could be better, here’s how to make it better. The trick to reading this book is to carefully read through every single refactoring pattern and then try to apply it on your code base (you don’t have to commit if it doesn’t fix things). You can’t just blow through it or you won’t really learn it. And you can’t just say “oh, I’ll look up a refactoring when I need it” — because then you don’t know what to look for.
  • Patterns of Enterprise Application Architecture: Great inventory of many of the patterns that underpin Rails itself, as well as descriptions of many of the “new” approaches that people advocate today (like transaction scripts and service layers). You won’t necessarily implement most of these patterns yourself, but it’s an invaluable resource to understanding the differences in architectures and why framework work the way they do. (Funny anecdote: before I created Rails, I redrew many of the diagrams in OmniGraffle for Martin Fowler because I liked the book so much.)
  • Domain-Driven Design: This is probably the least readable book of the bunch. It’s a slug to work through, but the ideas are worth it. It’s a great primer on how to turn a problem space into a beautiful OO domain model. What should your models be called? What logic goes where? How do we reproduce reality into an object model.
  • Are Your Lights On?: This isn’t technically a programming book, but it deals with the biggest problem facing developers none the less: What is the problem we’re trying to solve? Is it the right problem? Could we solve a different problem instead and that would be just as good? Nothing has increased my programming productivity more than being able to restate hard problems as simple ones.

If you consider programming to be a subset of writing, and I certainly do, then you would also do well to read Elements of Style and On Writing Well. I’ve found reading those made me a better programmer as well.

Reading these five to seven books will give your programming chops more vitamins and nutritional value than a couple of year’s worth of blog posts and tutorial.

Getting hyper about hypermedia APIs

David wrote this on 56 comments

Using URLs instead of ID references in your APIs is a nice idea. You should do that. It makes it marginally more convenient when writing a client wrapper because you don’t have to embed URL templates. So you can do client.get(response[:person][:url]) instead of client.get("/people/#{response[:person][:id]}"). But that’s about it.

The recurrent hoopla over hypermedia APIs is completely overblown. Embedding URLs instead of IDs is not going to guard you from breakage, it’s not going to do anything materially useful for standardizing API clients, and it doesn’t do much for discoverability.

Preventing breakage
According to hypermedia lore, you will be able to willy nilly change your URLs without needing to update any clients. But that’s based on the huge assumption that every API call is going to go through the front door every time and navigate to the page they need. That’s just not how things work.

If I want to request a message off a project in Basecamp, I would have to do something like this GET /projects, GET /projects/1, GET /projects/1/messages, GET /projects/1/messages/2. That’s great for the first fumbling in the dark discovery, but it doesn’t work as soon as I bookmark that last URL because I want to send comments to it later.

Just like bookmarks in the browser break if you change the URL, so will any client that’s stored a URL for later use.

Because breaking URLs is such a bad idea, people tend not do it. If you look at the successful APIs on the web, they’ve stayed remarkably stable because that’s the best way to prevent breakage. Like the W3C says: Cool URIs don’t change. Which means this isn’t much of a problem in the wild and even if it was, hypermedia APIs would still have big holes with direct links break.

Enabling discoverability
Good API docs explain what all the possible attributes of a resource are. They explain the possible values of those attributes. The options available and so forth. Thinking that we can meaningfully derive all that by just telling people to GET / and then fumble around to discover all the options on their own just doesn’t gel with me.

How do I know that the data I happen to be stumbling across includes everything that’s possible to do? What if the project I request doesn’t have any documents attached with it? How do I know how to find those and how to add new ones?

Standardizing API clients
The idea that you can write one client to access multiple different APIs in any meaningful way disregards the idea that different apps do different things. Just because there’s a standard way to follow a resource to a sub-resource doesn’t mean that you can just write one generic client that then automatically knows how to work with any API.

Any generic API will not know what to do with the things it get. It won’t know the difference between Flickr photos and Basecamp projects. The assistance of being able to follow a link to go from photo to comments and project to messages is nice, but as explained in the beginning, not exactly earth shattering.

Every single application is still going to need a custom API client. That client needs to know what attributes are available on each resource and what to do with them.

In summary, here’s a low-fi solution to these three problems that doesn’t require a spec or involving the IETF:

  1. Don’t change your API URLs.
  2. Document your API.
  3. Provide a custom client wrapper (you’ll have to write one anyways).

On top of that, be a chap and use URLs in IDs because convenience is nice and it doesn’t take too much effort if you’re using something like jbuilder anyway. But don’t go thinking that you’ve magically solved all these problems just because you did.

We’ve been down this path to over-standardization of APIs before. It lead to the construction of the WS-deathstar. Let’s not repeat the same mistakes twice. Some times fewer standards and less ceremony is exactly what’s called for.

Put chubby models on a diet with concerns

David wrote this on 47 comments

Different models in your Rails application will often share a set of cross-cutting concerns. In Basecamp, we have almost forty such concerns with names like Trashable, Searchable, Visible, Movable, Taggable.

These concerns encapsulate both data access and domain logic about a certain slice of responsibility. Here’s a simplified version of the taggable concern:

module Taggable
  extend ActiveSupport::Concern

  included do
    has_many :taggings, as: :taggable, dependent: :destroy
    has_many :tags, through: :taggings 

  def tag_names

This concern can then be mixed into all the models that are taggable and you’ll have a single place to update the logic and reason about it.

Here’s a similar concern where all we add is a single class method:

# current_account.posts.visible_to(current_user)
module Visible
  extend ActiveSupport::Concern

  module ClassMethods
    def visible_to(person)
      where \
        "(#{table_name}.bucket_id IN (?) AND
          #{table_name}.bucket_type = 'Project') OR
         (#{table_name}.bucket_id IN (?) AND
          #{table_name}.bucket_type = 'Calendar')",

Concerns are also a helpful way of extracting a slice of model that doesn’t seem part of its essence (what is and isn’t in the essence of a model is a fuzzy line and a longer discussion) without going full-bore Single Responsibility Principle and running the risk of ballooning your object inventory.

Here’s a Dropboxed concern that we mix into just the Person model, which allows us to later to route incoming emails to be from the right person:

module Dropboxed
  extend ActiveSupport::Concern

  included do
    before_create :generate_dropbox_key

  def rekey_dropbox

    def generate_dropbox_key
      self.dropbox_key = SignalId::Token.unique(24) do |key| 

Now this is certainly not the only way to slice up chubby models. For Visible concern, you could have Viewer.visible(current_account.posts, to: current_user) and encapsulate the query in a stand-alone object. For Dropboxed, you could have a Dropbox stand-alone class.

But I find that concerns are often just the right amount of abstraction and that they often result in a friendlier API. I far prefer current_account.posts.visible_to(current_user) to involving a third query object. And of course things like Taggable that needs to add associations to the target object are hard to do in any other way.

It’s true that this will lead to a proliferation of methods on some objects, but that has never bothered me. I care about how I interact with my code base through the source. That concerns happen to mix it all together into a big model under the hood is irrelevant to the understanding of the domain model.

We’ve been using this notion of extracting concerns from chubby models in all the applications at 37signals for years. It’s resulted in a domain model that’s simple and easy to understand without needless ceremony. Basecamp Classic’s domain model is 8+ years old now and still going strong with the use of concerns.

This approach to breaking up domain logic into concerns is similar in some ways to the DCI notion of Roles. It doesn’t have the run-time mixin acrobatics nor does it have the “thy models shall be completely devoid of logic themselves” prescription, but other than that, it’ll often result in similar logic extracted using similar names.

In Rails 4, we’re going to invite programmers to use concerns with the default app/models/concerns and app/controllers/concerns directories that are automatically part of the load path. Together with the ActiveSupport::Concern wrapper, it’s just enough support to make this light-weight factoring mechanism shine. But you can start using this approach with any Rails app today.


Open Source Guilt & Passion

Nick wrote this on 6 comments

I was on In Beta a while ago, and we talked about the “guilt” that comes with maintaining open source projects. I feel both intensely passionate and exceptionally guilty with how I contribute to open source. I love getting patches into new projects, and the needs of existing projects are demanding.

An open source contributor doesn’t feel guilt as in a crime, but guilt as in time: What is best to spend your yours on? Steve Klabnik has a strategy for dealing with a limited amount of time:

The basic idea is this: you try to minimize the things that are bad, and maximize those that are good.

My strategy has built on Steve’s: minimize the guilt of open source, maximize the passion from open source. I’ve been thinking a lot about what creates passion, and what builds up guilt. It’s also worth considering what destroys passion, and what can tear down guilt.


Learning Rails at The Starter League

Jason Fried
Jason Fried wrote this on 10 comments

This fall I’ve been taking the Rails for Designers class at The Starter League here in Chicago.

My classmates come from as far as South America and as close as a few desks down (fellow 37signals designers Jamie, Mig, Jonas, John, and Shaun are also taking the class). One student, not in our class, came over from Hong Kong.

My classmates also come from a variety of backgrounds. Some are designers with no programming experience. Others are programmers who work in languages other than Ruby. There’s also a lawyer and a couple Chicago Public School teachers, too. There’s a good 30 year age range spread as well. It’s a diverse and dedicated group. They’re inspiring people.

Class ends in a few weeks, and applications for the next quarter close in a few days, so I thought it would be a good time to reflect on the experience.

Was it worth it? Absolutely. Would I do it again? Absolutely. Would I recommend it to someone else? Absolutely.

I’ve been around Rails since the beginning. At 37signals, designers and programmers work together on the same codebase, so every 37signals designer has seen plenty of Rails in their time. And I’ve tried to pick it up on my own over the years by reading books or getting a few crash courses from co-workers. But it never clicked like it does now after taking this Starter League class.

I think the magic is in how Jeff and Raghu, the two teachers, understand how to teach absolute beginners. Teaching beginners is a unique skill. It requires a ton of patience and a truckload of empathy. You really have to start right at the beginning and assume nothing about what people may or may not know. You have to think like a beginner again. That’s really hard.

Am I a fantastic programmer now? No way. Would I hire myself as a programmer at 37signals? No, I wouldn’t. But that wasn’t my personal goal.

However, after just a few months I have a solid basic understanding of Ruby, Rails, and what programming is all about. I can build a database-backed web app on my own. I can pull in data from external APIs, manipulate it, and return it formatted the way I want. I can read and understand a bunch of code in the Basecamp code base that was complete greek to me before. I know what this code does, why it is where it is, how to manipulate it enough not to break the basics, and how to make changes without having to ask for help. That’s a huge leap forward for me, and it means fewer “hey, can you do this for me?” questions for my co-workers.

I can’t tell you how liberating it is to be able to find my way around our codebase now. It makes me a better designer, too. I can quickly prototype new ideas without having to get someone else involved. Big win. Further, I know where to go from here if I want to dedicate myself to getting better.

And on top of all of this, I feel like I’ve gained an invaluable skill: The ability to see problems from a new angle. Learning how to program has introduced me to a new perspective on problem solving, a new way of thinking. That doesn’t come around often and I’m thrilled to have found it at The Starter League.

If you’re interested in learning Rails – even if you don’t have a single bit of experience – check out The Starter League’s Web Development Class. Just want to learn HTML/CSS? Check out the HTML/CSS class (and there’s an advanced HTML/CSS class, too). There’s even a User Experience Design class. Want to learn visual design or how to improve your current design skills? Our very own Mig Reyes is teaching the Visual Design class.

Applications for The Starter League Winter session are due by Sunday. If you’re on the fence, hop off and apply. You will not regret it.

(Disclosure: 37signals is a minority investor in The Starter League)

Pattern vision

David wrote this on 16 comments

Design and programming patterns are wonderful ways of disseminating knowledge. It’s immensely satisfying to bring solution to a tough problem by applying a perfect-fit pattern.

But success with pattern applications is like taking that first innocent bump. You quickly want more of what made you feel this good. And buoyed by the high of a perfect fit, it’s easy to develop pattern vision where code, like Tetris shapes, become mere blocks for fitting into the pattern holes.

With this infliction, you will invariably drift away from the original intent of making specific and immediate pain go away. Where the perfect and legitimate fit will remove a thorn that’s obviously hurting the code right now, the speculative fit pretends to do the same for future hurts.

Now that’s an honorable intention. A pinch of prevention over a pound of cure, right? Only, it rarely works out like that. Speculative pattern applications to avoid future, possible ails is a form of fortune telling. It’s the reason YAGNI was coined.

The further down this rabbit hole you go, the farther away from practical improvement you’ll end up. Go deep enough and the pattern vision turns into articles of faith. Inheritance is always bad, only composition can be our true savior! Only standalone command objects can give us the reusability this application needs!

Turning patterns into articles of faith takes them out of the realm of rational discussion. Code either adheres to the holy patterns or it doesn’t. If it does not, the programmer must first repent, say his 25 rose-delegations, and then swear never to let an opportunity for a policy object extraction pass by him again!

The naming of many design principles supports this theological world view that patterns also at times inhibit. It’s obvious that you’re a sinner if you break The Law of Demeter and of weak will if you forgo the Single Responsibility Principle.

Patterns are best thought of like helpful guidelines and suggestions, not laws, not imperatives. It’s a written account of “hey, if you have this problem, you can try this thing to make it better”. Their core value is transforming some code to better code in a way that’s immediately obvious to the writer.

Evaluating such improvement is easy. You look at the code you had before applying the pattern and after it. Did it make the code simpler to understand? Clearer? Prettier? If you can’t easily see whether that’s so, you’ve been sold.

Automating with convention: Introducing sub

Nick wrote this on 21 comments

When I started my on-call shifts, we had pretty little in the way of automation for day-to-day issues. Tasks like SSH’ing into our cluster, starting a Rails console, or doing a deep search through our gigantic mail directories, were either shelved away in someone’s bashrc, history log, or just ingrained into someone’s memory. This pain was also felt by a few other of my fellow programmers, and we started cobbling together a Git repo simply named “37s shell scripts”.

This repo started very innocently: a little Ruby script named console that mapped a product name (basecamp) to a server name inside of our cluster (jobs-03), SSH’d in, and then ran a production Rails console. Several more bash and Ruby scripts started to trickle in as we started to share more of our personal code that we used when on-call. Eventually Sam laid down a foundation of bash scripts and directories borrowed from rbenv, and dubbed it “37”.


Running beta in production

David wrote this on 22 comments

We’re constantly working on new features, improvements, and technical upgrades for Basecamp. Many of these changes need to be experienced in the wild to guide their evolution. We need to live with them in our daily use of Basecamp to see whether what seemed like a good idea is actually a good idea.

To this end, we run six different beta servers that all point to the same production database. We’ve found it impossible to accurately evaluate a feature unless it’s being used in anger with real data that actually matters. Evaluating changes against a staging server that’s running an old copy of the database just doesn’t cut it.

The way we book a beta server for use with a certain branch is simple: It’s just in the title of the BCX Campfire room.

When you’re done with a beta server, you change the title to “available”. When you need a beta server, you change it to the name of the running branch.

To select a given beta server, we’ve added a drop-down to all 37signals’ staff accounts within Basecamp itself.

You just pick the server you want to run on and it’ll route you to the beta environment running the branch you’re looking to try out.

Long-term exposure to upcoming features is a great way to get them just right. But it’s also a great way to realize that what you thought was a great idea just wasn’t good enough to ship. We’ve killed many features and changes to Basecamp after living with them for a few weeks. Often times, the most important features are the ones you don’t ship.