I’m curious what software test methodologies and tools you use. Are you religious about Test Driven Development (TDD)? Do you use Rails’ Test::Unit, or do you prefer alternate tools such as Behavior Driven Development with rspec? A list of the test-related plugins you use would be interesting to see.

We’re big believers in the benefits of testing, but I wouldn’t call it religious. I personally rarely get into the flow of writing my tests first, except for transformation-style methods that takes known A and turns it into known B. I find that test-first often doesn’t fit my brain all that great for molding beautiful APIs. I know, though, that it works wonders for others and I deeply respect that approach. On our team, it seems to work especially well for Jamis at times.

I’m perhaps also a little on the fringe in regards to how much I test. I don’t believe that getting 100% coverage is worth it in most cases. Especially not when it comes to testing view code. For our applications, it simply changes too often and the criticality of display bugs is low and very rarely realized anyway.

The areas of testing where I personally derive the most benefit is from object-interaction tests. Like testing the signup form that creates a handful of records at once and need to spread error checking across the lot. Or dealing with project limits in Basecamp that’ll interrogate multiple other classes. Or API interactions for the creation of Writeboards.

For people new to testing, though, I would recommend going all-out from the start. The only way to get a good grasp on when you can be more lax in your testing is to have done “too much” for a while and identified areas that could go with less without materially hurting your productivity now and in the future. So in that sense, easing your testing rigor is a late-stage optimization that needs to be done with great care. If you’re ever in doubt whether to spend time testing or not, test.

In terms of tooling, we’re using the vanilla Test::Unit flavor that ships with Rails. I’m personally not convinced that rspec holds much improvement over it and I tend to think the DSL-style has been taken a tad too far. But I do like the BDD-style focus of naming your tests in terms of “should” and I use that in my own style. In the large scheme of things, it really doesn’t matter one way or another. The important thing is that you test and if rspec and BDD makes testing more enjoyable for you, rock on.

Also, while I don’t think striving for 100% test coverage is worth it in most cases, it’s nice to know where you stand none the less. The rcov package provides a great way of seeing how you tests are holding up and where you might be lacking.

Chris asks:

What importance do you place on formal education in your team? In today’s information age, it seems that about any kind of knowledge is only a few keystrokes away, and anything one might want to learn about is freely available and can be mastered given the drive and, of course, trial and error to develop the skill. Naturally, this especially applies to web technologies (more so than, say, neural surgery). What kind of educational background does your team harbor, be it for business or technology, what practical advantage does it lend, and what do you think about the crowds of talented, self-taught “amateurs” which the web has made possible?

We don’t put much value in formal education when deciding who to hire. In fact, I believe only three of the eight people at 37signals have a formal higher education degree. Some spent a little time at college and decided it wasn’t for them. Some didn’t go at all. We couldn’t care less.

What we care about is intelligence, curiosity, passion, character, motivation, taste, intuition, writing skills, and the ability to make smart value judgements. A few of these qualities may benefit from exposure to higher education, but we feel most of them are better learned through practical experience. Further, we don’t believe taste can be taught—you either got it or you don’t. We believe taste is one of the most important qualities in anyone we hire.

Of course we don’t hold a formal education against anyone, we just don’t pay much attention to it. We’re more interested in someone’s experience, real work, and point of view than we are with their diploma, degree, or GPA. Formal education is probably last on our list of qualities we feel make someone qualified to work at 37signals.

Thanks for the questions!

So far we’ve received nearly 100 questions since posting the Ask 37signals announcement. We’d love to answer yours. Please send it along to svn [at] 37signals dot com. Title the email “Ask 37signals”. Thanks again!

Typography.com
Typography.com has a smart way of showing off fonts. They show them in context with multiple layouts instead of just the standard ABCDEF or “quick brown fox” treatment. Knockout is one example.

The “H&FJ Suggests…” feature at the bottom of the screen is a nice way to suggest font pairings too. Kinda like a sommelier but for fonts (typolier?).

Catalog Choice

Under assault from Pottery Barn catalogs? Then check out the clear copy and design at Catalog Choice, a free service that lets you opt-out of unwanted catalogs.

On the evening of Monday, November 12, we experienced a few of hours of downtime due to an explosion at Rackspace’s main data center in Dallas, TX. This event lead to the eventual failure of a backup cooling system. Without adequate cooling, our servers had to be shut down to prevent permanent damage. We have detailed the events that led to the downtime. We deeply apologize for any inconveniences this may have caused and will work hard to make sure we reduce the likelihood of this happening again.

“The Piracy Paradox: Innovation and Intellectual Property in Fashion Design” is sparking an interesting conversation about copycattiness in creative professions.

The paper argues that copying in the fashion industry does not deter innovation (and may actually promote it). James Surowiecki summarized the essay in The New Yorker and argued that fashion piracy results in “more innovation, more competition, and probably more sales than there otherwise would be.”

Designers’ frustration at seeing their ideas mimicked is understandable. But this is a classic case where the cure may be worse than the disease. There’s little evidence that knockoffs are damaging the business. Fashion sales have remained more than healthy—estimates value the global luxury-fashion sector at a hundred and thirty billion dollars— and the high-end firms that so often see their designs copied have become stronger. More striking, a recent paper by the law professors Kal Raustiala and Christopher Sprigman suggests that weak intellectual-property rules, far from hurting the fashion industry, have instead been integral to its success. The professors call this effect “the piracy paradox.”

The paradox stems from the basic dilemma that underpins the economics of fashion: for the industry to keep growing, customers must like this year’s designs, but they must also become dissatisfied with them, so that they’ll buy next year’s. Many other consumer businesses face a similar problem, but fashion—unlike, say, the technology industry—can’t rely on improvements in power and performance to make old products obsolete. Raustiala and Sprigman argue persuasively that, in fashion, it’s copying that serves this function, bringing about what they call “induced obsolescence.” Copying enables designs and styles to move quickly from early adopters to the masses. And since no one cool wants to keep wearing something after everybody else is wearing it, the copying of designs helps fuel the incessant demand for something new.

Law school professor Susan Scafidi calls this “a tired, old argument” and says it’s based on an outdated, pre-internet portrait of the industry.

The designers who suffer from copying are the little guys – those whose designs are copied, while their trademarks are not. Consider the accessories designer who received an order for a belt from a large department store – only to have the store place its larger reorder with a cheaper manufacturer. Or how about the jeweler whose work was admired by a buyer at a trade show and hoped for a sale, only to open the large company’s catalog months later and see an exact copy of her design? Maybe the dress designer who saw her dress praised in an online forum, only to have the next post recommend buying an exact knockoff elsewhere – followed by thanks for the “tip”? Perhaps you’d be convinced by the handbag designer who actually received a wholesale order, only to have it canceled a few days later because the buyer found an exact copy of her original design elsewhere at a lower price? The stories are common ones, but these are not hypothetical examples. These are real people, some of whom prefer not to be named. They have invested time, money, and talent – R&D to any other industry – in realizing their visions, only to have their work stolen, often by huge companies. You would recognize many of the names of the corporate copyists; I doubt that most readers would ever have heard of the startup designers.

Handbags at dawn offers some more pushback to the Piracy Paradox.

Gary asks a follow-up question to How has open source helped or hindered?:

Why do you use Mac OS X as your laptop/desktop/development machines OS instead of a Linux distro?

There is really nothing religious about our use of open source. We use it because it’s better on the scales of merit that we care about. For infrastructure software, such as web servers, databases, server operating systems, programming languages, and web frameworks, the scales of merit lend themselves incredibly well to open-source development. Thus, we use it and are passionate about it.

For desktop operating systems? Not so much. There are just too many disciplines involved that programmers are not naturally good at and don’t have sufficient levels of taste to prepare masterfully. And programmers constitute the vast majority of builders in the open source community.

So it’s not unreasonable to think that these programmers will do exceptionally well when they’re designing for them and their kind, but at the same time do less well when they’re trying to figure out what makes a great iPhoto or iTunes or what have you.

Therefore I tend to think that open source is at a natural disadvantage at creating end-user applications, in which I include OS X and Linux destined for the desktop. It’s not impossible, just very hard.

Firefox is always heralded as a great example of good end-user software, but I do think that in part is because they’re mostly just doing great infrastructure advances (spec compliance, developer tools, security) in a familiar shell (how much difference is there between Firefox and the early browsers on the UI?).

Which interestingly enough is also how my usage of Firefox goes. I use it for development purposes (primarily because of the developer plugins, like Live HTTP Headers and Firebug) and I use Safari for recreational purposes.

So what I’m trying to say is that for me, OS X is just better on the scales of merit that I care about when it comes to an operating system that needs to be so generally applicable as to deal with my email, IM, browsing, music, pictures, productivity apps, and more.

In other words, I’ll stick to OS X on my Macbook Pro (tight integration between hardware, software, and services is the hallmark of OS X’s superiority), but be equally thrilled to use Linux and FreeBSD on the server.

It’s incredibly tempting to create a road map when you’re driving a software product. You get to reap the glory of announcing desired features without even a downpayment of work. It takes no design, no consideration, or even discipline to respond to feature requests by making them a bullet point on a road map. It’s like buying goodwill on credit, but what you don’t pay for now, you’ll pay for later with interest.

Like functional specs earlier in the development phase, product road maps are fraught with trouble. The due diligence process is usually twice as shallow, which means that you’ll inevitably end up promoting illusions of agreement. When all we have to agree on is a slogan, like “Meetings Tab, 4th Quarter”, it becomes an empty imagination box that’ll fit wildly different expectations. Disappointment, however, is sure to ensue when only one set of expectations can actually be met.

Even worse than mismatched expectations, though, is the slippery slope of selling tomorrow over today. When you sell the software that you actually have, there’s a limited amount of wriggle room to cajole prospects. Customers has to make real decisions about whether the current state of your software is a good fit for them. Some times it’s not. That’s okay.

It’s better to turn customers away than to placate their instincts and lure them in with vague promises. It’s incredibly rare that a single feature will truly make or break your chance with a customer. If your software is a good enough fit, most people can make do without that one or two things that they’d like to see.

If your software is not really a good fit, it’s very easy for customers to convince themselves that it will be, if you greet them with an illusion of agreement over a few “deal breakers”. But when you finally do deliver, you’ll also burst that illusion and end up with disillusioned customers who thinks you suckered them into buying a cow and then gave them a goat.

And worse, while it might seem free at first to win customers by promising them gold at the end of the road map rainbow, it’s not. Accepting new entries to the product road map carries very real development debt. This debt will shrink your degrees of freedom. Your ability to pursue new great ideas as they arise will be seriously dampened when you’ve already committed to a tall stack of requests eagerly awaiting implementation.

Builders bound by the guesswork of yesterday are not going to be happy troopers. It’s demoralizing to be forced to work on something not because it’s the right thing to do at the time, but merely because the promise note is up.

But try avade and you’ll soon hope that it was merely your mob-connected bookie you were trying to dodge. Customers do not forget your promises — especially not the ones that were won over specifically because of the promises of a road map.

Ross asks:

37signals has contributed massively to the open source community, but in what ways has the open sourcing helped and hindered 37signals and would you advocate that more companies contributed or open sourced their software, if so what advice would you give them?

Open source provides an incredible amount of technical leverage for small companies. No matter who productive your rock-star programmers are and no matter how much judo you apply to your problems, solid infrastructure takes a long time and benefits immensely from broad involvement. It really does take a village to raise great infrastructure.

The Ruby on Rails framework of today is a lot more productive than the one I was using before it was open sourced. I use features every day created by others, enjoy polish done by others, evade bugs caught by others. All work I would otherwise have to do myself. So I simply get more done for less effort than it would otherwise have taken. The same holds true for the other open source projects that have been cultivated in 37signals, like Prototype and Capistrano.

Getting more done for less is the obvious benefit of open sourcing your work. But there are a lot of other positives as well. For one, it feels good to give back. 37signals as a company is possible in part only because of how open source lowered the barriers of entry for small businesses.

Everything from the operating system to the database to the web server to the proxy engine to everything in between on our servers is open source. If we’d have to go back to a time where all that carried big upfront licensing costs, we might never have gotten the product direction for the company off the ground. Helping further that ecosystem is very rewarding.

It also gives you a public arena to learn from other great programmers and to better yourself. Most of the people using Rails are working on proprietary applications, so we don’t get to share or discuss the specifics of the projects all that often. But we do get to share and discuss the specifics of the infrastructure. That’s the best continued training course you could ever run.

Through that public arena you get access to scout for the best minds and hire people of exceptional talent. We have found all our current programmers through the Rails scene and had access to evaluate much of their work as a result of open source. I also wrote about a few years back.

Sprinkle on top that if you happen to run a successful open-source project, you’ll probably attract a fair amount of press attention and customer goodwill. 37signals has definitely benefitted from both of those categories off the many open source projects that we run.

So you get all these positives, but what about the negatives? I really don’t see any. A big fear that a lot of people have is that they’ll somehow be giving away their secret sauce. Unless your actual product is what you’re open sourcing, it really doesn’t matter (and there are even plenty of examples of that working well). It’s unlikely that the piece of code that’s only seen internal development is such a silver bullet that you’re going to outshine your competition by its use alone.

All that to say that I think open sourcing infrastructure software is a great idea. I’d heartily recommend it to anyone sitting on a piece of code that more people could benefit from and that they’d like to see developed further.

Congratulations to our friends at VEER for being acquired by Corbis.

Scott asks:

Do you use formal personas when thinking about the users of a new app?

We don’t use personas. We use ourselves. I believe personas lead to a false sense of understanding at the deepest, most critical levels.

Every product we build is a product we build for ourselves to solve our own problems. We recognize our problems aren’t unique. In fact, our problems are probably a lot like your problems. So we bundle up the solutions to our problems in the form of web-based software and offer them for sale.

We recognize not everyone shares our problems, our point of view, or our opinions, but that verdict’s the same if you use personas. Making decisions based on real opinions trumps making decisions based on imaginary opinions.

I’ve never been a big believer in personas. They’re artificial, abstract, and fictitious. I don’t think you can build a great product for a person that doesn’t exist. And I definitely don’t think you can build a great product based on a composite sketch of 10 different people all rolled into one (or two or three).

Personas don’t

Personas don’t talk back. Personas can’t answer questions. Personas don’t have opinions. Personas can’t tell you when something just doesn’t feel right. Personas can’t tell you when a sentence doesn’t make sense. Personas don’t get frustrated. Personas aren’t pressed for time. Personas aren’t moody. Personas can’t click things. Personas can’t make mistakes. Personas can’t make value judgements. Personas don’t use products. Personas aren’t real.

People do

People talk back. People answer questions. People have opinions. People can tell you when something just doesn’t feel right. People can tell you when a sentence doesn’t make sense. People get frustrated. People are pressed for time. People are moody. People click things. People make mistakes. People make value judgements. People use products. People are real.

Get Real

So if you can’t design something for yourself, design something for someone you know. Get that person or people involved in your project early on. Basing your decisions on a matrix of personality traits isn’t what I’d recommend if you really want to build a great product.