Over the past few months I’ve learned an immense amount from the all new Basecamp codebase. This year for RailsConf, I was able to show off some new (and old!) patterns in the codebase along with some neat libraries that we’ve put to serious use for our customers. The slides are over at SpeakerDeck if you’d like to see them all, and hopefully a video of the presentation will be up soon!

Here’s a few choice sections from the talk, and plenty of great links to check out in the meantime:

We’re looking for one or two more folks to join our support team. Specifically, we’re after some great people in the UK time zone (or close to it) to cover business hours in that part of the world. Launching the all-new Basecamp got a lot of people excited and we want to expand our team to cover business hours in more time zones.

You’ll be responsible for providing tremendous customer service via email for Basecamp, Basecamp Classic, Highrise, Backpack, and Campfire. You’ll also help us answer questions via Twitter, create and edit help documentation, and maybe run some online classes.

You’ll be expected to answer about 75 emails per day once you’re fully up to speed (2-3 months on-ramp). This is a significant volume, so be sure that you’re ready and able to deal with that kind of daily load.

We’re looking for some great writers who love helping our customers, so you should enjoy making complicated situations simple and painless and have a passion for our products.

If you live in the UK (or thereabouts) and want to join Ann, Chase, Emily, Kristin, Merissa, Michael and me in making our customers happy, please apply!

How to apply

Please submit a cover letter explaining:

1. Why you want to work in customer support.
2. Why you want to work at 37signals and not somewhere else.
3. A description of a great customer service/support experience you had recently, and what made it great.

Also, attach the following writing samples:

1. Explain in 3 paragraphs or less why a customer would pick Basecamp vs Highrise.
2. Respond to a customer asking for project categories in Basecamp that it’s not something we offer, and suggest she work with the project titles and stars to add organization.
3. A company using our job board failed to find to find a suitable candidate and wants a refund. Respond that we don’t offer refunds for job postings.

We offer heaps of lovely benefits, plus a progressive work environment. Starting salary is $45k USD, depending on experience.

Email everything to [email protected]. Include “Customer Support” in the subject line. If you’re attaching a resume, please send it as a PDF. Note: We look favorably on people who get creative with their applications.

We look forward to hearing from you!

We just launched a brand new feature in the all new Basecamp that is handy in a bunch of different ways. Now you can move to-do lists, discussions, text documents, files, and calendar items between projects or into a brand new project.

Moving items between projects is especially useful in a few specific cases (and probably many we’ve never thought of). For one, people put things in the wrong place from time to time. Now it’s no big deal – just move it to where it should be.

And then there are other times when you want to break a bigger project into a couple smaller projects. Now you can take a list or a message or anything from one project and start an entirely new project from that item. It’s great.

Here’s a video showing you how it works. We think you’re going to love it.

Thanks for using Basecamp! Wait, you aren’t using Basecamp? What are you waiting for?.

One of the things we’ve added to our applications in the last few months is a little gem that (among other things) adds a comment to each MySQL query that is generated by one of our applications.

Now, when we look at our Rails or slow query logs, our MySQL queries include the application, controller, and action that generated them:

Account Load (0.3ms)  SELECT `accounts`.* FROM `accounts`
WHERE `accounts`.`queenbee_id` = 1234567890
LIMIT 1
/*application:BCX,controller:project_imports,action:show*/

When we’re trying to improve a slow query, or identify a customer problem, we never have to go digging to understand where the query came from—it’s just right there. This comes in handy in development, support, and operations – we used it during a pre-launch review of unindexed queries in the brand new Basecamp which launched a couple months ago. If you combine this with something like pt-query-digest, you end up with a powerful understanding of how each Rails action interacts with MySQL.

It’s easy to add these comments to your Rails application in a relatively unintrusive way. We’ve released our approach that works in both Rails 2.3.x and Rails 3.x.x apps as a gem, marginalia.

marginalia (mar-gi-na-lia, pl. noun) — marginal notes or embellishments
Merriam-Webster

We’ve been using this in production on all of our apps now since December, ranging from Rails 2.3.5 to Rails master and Ruby 1.8.7 to 1.9.3. You should be able to have it running in your application in a matter of minutes.

It’s worth acknowledging that anytime you modify the internals of something outside your direct control there are risks, and that every function call adds some overhead. In our testing, these have both been well worth the tradeoff, but I absolutely encourage you to consider the tradeoff you’re making for yourself every time you instrument or log something. You may certainly have a different set of tradeoffs, and you should absolutely test on your own application.

Have a suggested improvement to our sample code or another way to do this? We’d love to hear it.

Thanks to Taylor for the original idea, and to Nick for helping to extract it into its own gem.

“Oh, that’s not my job,” is the sound of doom. Maybe not imminent doom, but doom indeed. It’s the magic inflection point when a company becomes too big (even if only psychologically) for any single employee to give a rat’s ass about job numero uno: Making shit work.

No profession is immune. You can have designers who oh-thats-not-my-job to get the JavaScript they wrote to work, programmers who cry for operations to make their slow code run on time, operations people who refuse to answer customer complaints from their network outage, and on and on. Once the mentality cements, everything is eventually someone else’s job, and they’re being a toad for inconveniencing you with it.

And besides, it’s easy to put it on somebody else, right? Everybody else’s job is easy!

Departmental hedges grow fast and tall if not trimmed with vigor. It’s the natural path unless you take steps to fight it.

That’s why, at 37signals, we all chip in when lots of customers have questions after a new product launch and customer support is overwhelmed. It’s why programmers will wake up in the middle of the night if a sql query tipped over and needs an urgent rewrite until faster servers can arrive.

Don’t let your company culture become one where certain people are too good to do the jobs that need doing. Making shit work is everyone’s job.

My dad is a firefighter and fire captain in Niagara Falls, NY. When I told him I had on-call duty for my new job, he was beyond excited. After relaying stories of waking in the middle of the night to head into the hall and getting overtime from his buddies that didn’t want to wake up for work were sick, I had to explain that it’s a different type of action for me. I face issues of a less life-threatening variety, but there’s still plenty of virtual fires to extinguish. Here’s a primer for what being an on-call programmer means at 37signals.

The routine

On-call programmers are rotated around every 2 weeks, and all 8 programmers currently get their fair share of customer support and interaction. Usually we have 2 to 3 programmers on-call on a given week. Support issues from email and Twitter are routed through Assistly, and handled by our awesome support team.

If there’s an issue they can’t figure out, we get a ping in our On Call room in Campfire to check out a certain issue:

These issues range in severity, and they might look like:

It’s always satisfying to announce the release of one of the top Basecamp feature requests. Today is one of those days. We’re glad to announce that to-dos with due dates now show up on the all new Basecamp calendar.

Prior to this update, only events that were directly added to the calendar showed up on the calendar. But as of today, any to-dos with due dates also show up on the calendar. And just like events, you can drag and drop to reschedule a to-do. Very handy.

Now you have a much better view of everything that’s due in a project. To-dos, events, meetings, project phases, etc. – now you can see them all at the same time on a single calendar.

We’re really happy with how it turned out. We hope you love it.

Here’s a quick video showing you how it works.

The signs are all here: There’s now an incubator on every corner, even your uncle is donning angel wings, and IPO expectations for Facebook are exceeding the hype for a new Twilight movie. But that’s all circumstantial evidence. What we needed was some public testimony to really put everyone in the right frame of mind.

Enter our trusted troubadour of bullshit, TechCrunch:

My best guess is that it is about to get crazy. And, only fools sit on the sidelines. Many strong and older entrepreneurs that I know are wealthy today because they made intelligent decisions during the dot-com bubble of the late ’90s. Success was not easy then, and it will not be easy now, either. But, the likelihood of a great outcome is much higher in a boom.

There are a lot of newly minted entrepreneurs that pursue their dream company in a halfhearted way. You may tinker with your idea while toiling at a day job. You may refuse to put in the work required to recruit the best talent. You might be afraid of launching an imperfect product. Or, you may put a mediocre effort into fundraising.

Let me translate: Dude, you’re going to miss riding this bull onto the bubble if you do not get on it RIGHT. FUCKING. NOW! Didn’t you see that someone just made A BILLION DOLLARS? Why wasn’t that you, lazy schmuck? Don’t answer that—there’s no time to look at the past, just quit your job, and come out here to the Valley post-haste. Sand Hill road just scored a fresh load of loot.

Now I get the basic psychology. Someone just won the billion dollar startup powerball and now everyone wants to make sure they bought a ticket for the next drawing. And why not? While the past season of tech IPOs has been full of duds, we still have the big baller Facebook coming up for a shot. And if red 32 hasn’t hit for the last few IPOs, it’s bound to do for this round. COME ON LUCKY 32!!!

But let’s calm down. Sooner or later, the market is going to sort these things out, and all will be right as rain. That’s evident with the Groupon fiasco. They’ve restated their accounting numbers endlessly and the stock has finally tanked. At the end of the day, the rules of accounting will blow through all the smoke and the mirror will show a face with no make-up.

Are you calm? Good. Now get ready to rage right back up. The new JOBS Act that was just passed with the help of a thousand VCs stomping it down the throat of Congress undoes all that. Matt Taibbi from Rolling Stones reports:

Ostensibly, the law makes it easier for startup companies (particularly tech companies, whose lobbyists were a driving force behind passage of this law) attract capital by, among other things, exempting them from independent accounting requirements for up to five years after they first begin selling shares in the stock market.

The law also rolls back rules designed to prevent bank analysts from talking up a stock just to win business, a practice that was so pervasive in the tech-boom years as to be almost industry standard.

Now isn’t that swell. Enable people with an extreme financial incentive to spin the truth, or outright lie about the numbers, a 5-year get-out-of-jail cover, and what are you going to get? JOBS, silly! Duh!

Buckle your seat belt, Dorothy, ‘cause Kansas is going bye bye.

When developers first discover the wonders of test-driven development, it’s like gaining entrance to a new and better world with less stress and insecurity. It truly is a wonderful experience well worth celebrating. But internalizing the benefits of testing is only the first step to enlightenment. Knowing what not to test is the harder part of the lesson.

While as a beginner you shouldn’t worry much about what not to test on day one, you better start picking it up by day two. Humans are creatures of habit, and if you start forming bad habits of over-testing early on, it will be hard to shake later. And shake them you must.

“But what’s the harm in over-testing, Phil, don’t you want your code to be safe? If we catch just one bug from entering production, isn’t it worth it?”. Fuck no it ain’t, and don’t call me Phil. This line of argument is how we got the TSA, and how they squandered billions fondling balls and confiscating nail clippers.

Tests aren’t free (they cost a buck o’five)

Every line of code you write has a cost. It takes time to write it, it takes time to update it, and it takes time to read and understand it. Thus it follows that the benefit derived must be greater than the cost to make it. In the case of over-testing, that’s by definition not the case.

Think of it like this: What’s the cost to prevent a bug? If it takes you 1,000 lines of validation testing to catch the one time Bob accidentally removed the validates_presence_of :name declaration, was it worth it? Of course not (yes, yes, if you were working on an airport control system for launching rockets to Mars and the rockets would hit the White House if they weren’t scheduled with a name, you can test it—but you aren’t, so forget it).

The problem with calling out over-testing is that it’s hard to boil down to a catchy phrase. There’s nothing succinct like test-first, red-green, or other sexy terms that helped propel test-driven development to its rightful place on the center stage. Testing just what’s useful takes nuance, experience, and dozens of fine-grained heuristics.

Seven don’ts of testing

But while all that nuance might have a place in a two-hour dinner conversation with enlightened participants, not so much in a blog post. So let me firebomb the debate with the following list of nuance-less opinions about testing your typical Rails application:

1. Don’t aim for 100% coverage.
2. Code-to-test ratios above 1:2 is a smell, above 1:3 is a stink.
3. You’re probably doing it wrong if testing is taking more than 1/3 of your time. You’re definitely doing it wrong if it’s taking up more than half.
4. Don’t test standard Active Record associations, validations, or scopes.
5. Reserve integration testing for issues arising from the integration of separate elements (aka don’t integration test things that can be unit tested instead).
6. Don’t use Cucumber unless you live in the magic kingdom of non-programmers-writing-tests (and send me a bottle of fairy dust if you’re there!)
7. Don’t force yourself to test-first every controller, model, and view (my ratio is typically 20% test-first, 80% test-after).

Given all the hundreds of books we’ve seen on how to get started on test-driven development, I wish there’d be just one or two that’d focus on how to tame the beast. There’s a lot of subtlety in figuring out what’s worth testing that’s lost when everyone is focusing on the same bowling or bacon examples of how to test.

But first things first. We must collectively decide that the TSA-style of testing, the coverage theater of quality, is discredited before we can move forward. Very few applications operate at a level of criticality that warrant testing everything.

In the wise words of Kent Beck, the man who deserves the most credit for popularizing test-driven development:

I get paid for code that works, not for tests, so my philosophy is to test as little as possible to reach a given level of confidence (I suspect this level of confidence is high compared to industry standards, but that could just be hubris). If I don’t typically make a kind of mistake (like setting the wrong variables in a constructor), I don’t test for it.

In February I got a chance to chat with Paul Budnitz, founder of Budnitz Bicycles.

“I design and create beautiful things and then create businesses to sell them,” Paul says. He’s been a professional photographer, shot films, owned a company modifying vintage clothing, turned making handmade microphones out of a garage into a multi-million dollar business — the list goes on.

“I started out in eighth grade selling fireworks to all my friends in school and we actually programmed all our orders on the mainframe computers at the University of California, which at that time they were teletype computers. And we didn’t really know that those computers were controlled by the Department of Defense. So I eventually got arrested and suspended. That was my first business.”

In 2002 Paul founded Kidrobot, which makes limited-edition art toys. “I like to make immaculate products,” Paul says, “and if I run the business myself I get to do it my way.” Kidrobot uses Basecamp to communicate with suppliers and distributers over four continents with hundreds of active projects.

Budnitz Bicycles, Paul’s latest endeavor, uses the new Basecamp to work with manufacturers and suppliers around the globe. The bikes are gorgeous! Paul let me take one for a spin and it was honestly the most comfortable ride I’ve ever had.

You can find out more about Budnitz Bicycles at budnitzbicycles.com and be sure to follow @budnitzbicycles on Twitter.

Also, if you missed the profile we did on Happy Cog you can find it over here.