The Big Rewrite, revisited

Of the many axioms in the world of software, few rise to the occasion of Thou Shall Not Rewrite Your Application. Spolsky called it the “single worst strategic mistake that any software company can make” in his seminal 2000 essay Things You Should Never Do.

The Big Rewrite has been likened to all sorts of terrible things, but perhaps the most damning is its association with declaring technical bankruptcy. Basically, the only time it’s reasonable to embark on the terribleness of the rewrite, is when you’ve been profligate cowboy coder. When your mountain of technical debt is crashing down upon you.

So it’s the white flag, then. It’s giving up. Capitulation! The error of your inadequacy has finally caught up and will be taking you to the cleaners. Who the hell wants to be that sorry sob of a programmer!?

No wonder programmers are loathe to question the wisdom of NEVER REWRITE. Either they’ll reveal their own morally deficient ways, or they’ll be seen as apologists for others traveling that deviant path.

Now, axioms develop for a reason. Many a rewrite has been started and gone astray for all the wrong reasons. Either it truly was a result of technical bankruptcy, or, perhaps even worse, it was a result of perceived technical bankruptcy by a new team uninterested in learning why things became the way they are. As Spolsky quips, such a rewrite is basically the same software with more and different bugs!

But there are other types of rewrites. The one most dear to me is the “Don’t Try To Turn A Chair Into A Table” rewrite. It’s the one we committed when we launched the new version of Basecamp a couple of years ago. A full, start-over, everything-is-reimplemented rewrite of Basecamp because we wanted it to do different things.

Yes, Basecamp Classic and the new Basecamp both juggle many of the same ingredients of project management and collaboration, but they’re mixed together in very different curations. So while we could have gotten from A to B one carefully tested refactoring and commit at the time, it would have been a fool’s errand.

Starting over allowed us to question the fundamentals of how the application worked and how it was implemented. We were able to make leaps, not just skips.

A chair can indeed be turned into a table with enough effort. Both have four legs, and all you need to do is chop off the back of the chair, and somehow refasten it to the base to extend the surface. Oh, and maybe some new wood to raise the legs up higher. It can be done, but why on earth would you?

So we decided to leave the chair alone. People were using the chair, and still are – years after the new table premiered! And we got to build a beautiful new table that’s been serving us so very well for the last couple of years.

That’s really the bottom line here: We rewrote Basecamp from scratch and it turned out great! Better than great, actually. We got to leave well enough alone for people who had adopted and grown accustomed to Basecamp Classic, we got to offer a brand-new product to new customers that was the very best we knew how to make, and we got a greenfield codebase to enjoy as well. Plus-plus, would do again!

So what am I saying here? Rewrite willy nilly whenever you get blue over seeing a few modes of progress made cumbersome because of poor past decisions? Of course not. Embarking on The Big Rewrite is not a choice to be made lightly, but a choice it remains. It should be one of the options on the table.

If you’ve accumulated enough fresh ideas about how your application can be radically different and better, The Big Rewrite may very well be just what the carpenter ordered.

David wrote this on Feb 11 2015 There are 15 comments.

LukeW

on 11 Feb 15

Is there a new Basecamp version in the pipeline?

DHH

Foot note: I did a podcast with @FullStackRadio touching on this issue as well.

Tim

Nailed it!

johnson

What was one of the most surprising challenges of the rewrite, technical or otherwise, that you found? (e.g. transitioning customers, keeping people motivated to work rewrite, maintenance for classic, etc)

- johnson

Johnson, the biggest break-through in our thinking was to give up on the idea of forcing people to migrate. We basically applied the Until The End of The Internet notion to past versions.

Otherwise, I’d say that everything has been far easier than the doomsday predictions of Spolsky and others. Business went way up with the new version, we’ve been able to keep Classic running and secure with a modest investment of dev resources, and we’ve been much happier running a Basecamp that’s closer to the idea of “the best we know how to build”.

Kas Thomas

Excellent post. I would add this: Once you have decided to do the rewrite, the first thing to do is conduct a post mortem on the previous product. What went right? What went wrong? What’s not working? Why do we know that now? Why didn’t we know it then? Work from baseline knowledge acquired during the post mortem.

Bartosz

I’m also a proponent of delaying rewriting software, but sometimes it’s inevitable.

We’ve been building our product for 13 years and many of the pieces are now 6-7 years old. Maintaining old software (such as SOAP API or PHP-driven payment gateway) is a bad idea. Our developers just don’t use these technologies anymore (in favor of node.js or Python), so there are fewer and fewer devs who can maintain the software. And that’s a good moment to rewrite.

Funny thing: 2 years ago, we’ve still been exchanging the data by sending a transparent GIF of a given width instead of using JSONP. It’s because it worked fine for 11 years. (You can read that story here). After all we rewrote that code (GIFs have some limitations compared to JSON, haha), but for 11 years it did the job.

Derick

@DHH

But you missed the entire assumption of Joel’s point.

The Big Rewrites that fail are the projects that force their customers to migrate to the new (rewriten) product.

You never did that. You allowed people to run the old Basecamp. So his argument doesn’t apply to you.

Essentially, you just created two seperates (while concurrently running) project management products.

Both products are independent.

That’s no different than Joel’s point. People could have stayed using the old browser. It was installed software, so not like it would disappear from their computers.

Alister Scott

on 12 Feb 15

Great article @DHH.

Earlier this week I wrote about a concept called ‘Intentionally Disposable Software’ which is basically what you’re talking about but doing ‘The Big Rewrite’ every six months or a year so you can make your software leaner and better each time.

It’s available to read here: http://watirmelon.com/2015/02/06/intentionally-disposable-software/

Mads Buch Stage

Good points David, and I would say that the most important is the fact, that you didn’t just rewrite the code to get rid of technical debt, increase performance etc. You rewrote it to fit with an entirely new user experience in mind, and for me at least, that almost always makes sense.

If you want your user experience to be drastically different, then there is a very big likelihood that your code would also benefit from being drastically different.

Andrew Miller

We’re deciding to rewrite our entire application not due to the fact that it’s just old. It’s old enough that it’s become a security concern due to the fact that it relies on old technologies that cannot be upgraded. Support from a community is non-existent due to the fact that no modern coding styles have been implemented, and overall application architecture was not ever properly considered during its initial creation. Maintanence is one thing, but due to the system design it’s nearly impossible to implement new features that the team now realizes we should have built to handle from the beginning. Could we manage most of these problems within the confines of our current project? Probably yes, but is it worth the time and effort? Probably not.

Juan P

how could the users use two versions? was there one database for each version? and a import export tool? or was it the same DB? or how?

Juan, they can have 3 projects on Basecamp Classic and 5 projects on the new Basecamp. Or they can migrate everything over, though that’s a somewhat lossy process, due to the apps having different features and implementations of the same features. Not the same DB (that wouldn’t be much of a rewrite).

GregT

on 13 Feb 15

Joel repeatedly refers to “throwing away the old code” which presumably means not supporting it anymore or acknowledging its existence. Since that’s NOT what 37S did with Basecamp, I think it’s apples and oranges, and you are misunderstanding Joel’s point entirely.