Back in March of 2009 I joined 37signals as Signal #13 and the other half of our two person support team. At the time we relied mostly on bug reports from customers to identify rough spots in our software. This required the full time attention of one or more “on call programmers”– firefighters who tamed quirks as they arose. The approach worked for a while but we weren’t making software quality a priority.

I had a chat with Jason Fried in late 2011 about how my critical tendencies could help improve our products. Out of that, the QA department was born. Kind of. I didn’t know much about QA and it wasn’t part of the development process at 37signals. So my first move was to order a stack of books about QA to help figure out what the hell I was supposed to be doing.

It’s been almost two years since our first project “with QA” back in 2012. Ann Goliak (another support team alumnus) recently joined me at the stead. Our QA process isn’t traditional and goes a bit different for every feature. Here’s a look at how QA fits into our development process, using the recent phone verification project as an example.

Step 1. I sat down with Sam Stephenson back in early July for our first walkthrough of phone verification. Hearing Sam talk about “creating a verification profile” or “completing a verification challenge” familiarized me with the terminology and flows that would be helpful descriptors in my bug reports. Here’s what the notes look like from that first conversation with Sam.
Step 2. After the introduction I’ll dive right into clicking around in a staging or beta environment to get a feel for the feature and what other parts of the app it touches. This is often the first time that someone not designing/coding the feature has a chance to give it a spin, and the fresh perspective always produces some new insights.
Step 3. There are lots of variables to consider when testing. Here are some of the things we keep in mind when putting together a test plan:

• Does the API need to be updated to support this?
• Does this feature affect Project templates?
• Does this feature affect Basecamp Personal?
• Does our iPhone app support it?
• Do our mobile web views need to be updated?
• Does this impact email-in?
• Does this impact loop-in?
• Does this impact moving and copying content?
• Does this impact project imports from Basecamp Classic?
• Test at various BCX plan levels
• Test at various content limits (storage, projects)

Project states

• Active project, Archived project, Project template, Draft (unpublished) project, Trashed project.

Types of content

• To-do lists, To-do items (assigned + unassigned + dated), Messages, Files, Google docs, Text documents, Events (one time + recurring).

Views

• Progress screen, In-project latest activity block, History blocks (for each type of content), Calendar, Person pages, Trash, Digest emails.

When these variables are combined you end up with a script of tasks like this one to guide the testing. These lists are unique for each project.
Step 4. In Basecamp we make a few QA-specific to-do lists in each project: the first for unsorted discoveries, a second for tasks that have been allocated, and a third for rough spots support should know about (essentially “known issues”).

When I find a bug I’ll make a new to-do item that describes it including: 1) A thorough description of what I’m seeing, often with a suggested fix; 2) Specific steps to recreate the behavior; 3) The browser(s) and/or platform(s) where this was observed; and 4) Relevant URLs, screenshots, or a screen recording.

We use ScreenFlow to capture screen recordings on the Mac, and Reflector to do the same in iOS. We’re fans of LittleSnapper (now Ember) for annotating and organizing still screenshots.
Step 5. The designer and programmer on the project will periodically sift through the unsorted QA inbox. Some items get moved to the QA allocated list and fixed, then reassigned to QA for verification. Other “bugs” will trigger a conversation about why a decision was intentional, or outside the scope of the iteration.
Step 6. Before each new feature launch, QA hosts a video walkthrough for the support team. We’ll highlight any potential areas of confusion and other things to be on the lookout for. After the walkthrough, a member of support will spend some time putting together a help section page that covers the new feature.
Step 7. Within a couple weeks after a feature launch the team will usually have a retrospective phone call. We talk the highs and lows of the iteration and I use the chance to ask how QA can be better next time around.
At the end of a project there are usually some “nice to haves” and edge-cases that didn’t make the pre-launch cut. These bugs get moved into a different Basecamp project used for tracking long standing issues, then every few months we’ll eradicate some of them during a company-wide “bug mash”.
So that’s a general overview of how QA works at 37signals. We find anywhere from 30-80 bugs per project. Having QA has helped reduce the size of our on-call team to one. The best compliment: After trying it out, no one at the company was interested in shipping features without dedicated QA.

Pitchfork Media encompasses the award winning online music publication Pitchfork.com, the Pitchfork Music Festival, Nothing Major: a journal of visual culture, and their newest cinematic sibling: The Dissolve.

I recently sat down with Pitchfork Creative Director and friend Mike Renaud to chat about being part of the company as it’s grown organically, nourishing his analog roots, and his experience using Basecamp.

Tell me a bit about the team. Are you all centralized or spread out across multiple offices? How do you communicate?

The Pitchfork team is made up of about 45 employees, now about 54 including The Dissolve staff. Twenty of us are located in Chicago, with editorial and Pitchfork.tv split across two different offices in Brooklyn. The Chicago office is our headquarters where our design and development teams, management, and advertising staff work.

A few team members go out to New York a couple of times each month for meetings, shoots, etc. Because the offices are located in different cities we use a lot of Basecamp to communicate and help keep everything together. Probably one of the main reasons Basecamp works for us is because of the distance.

Within the company we have kind of an IM culture. Everyone is on IM every day. For communicating with the New York office it really helps, but in Chicago it kind of annoys me when someone sitting next to you IMs for something. I always prefer talking and facetime. I can see the argument about avoiding distraction from work but in its own way I really think it becomes distracting to have people IMing you constantly; it makes people feel like they can approach you more than they otherwise would. And I still feel like I need to stop everything and answer.

What types of projects do you use Basecamp for?

We fucking use Basecamp all the time. “The Dissolve” Basecamp project is insane. We have a project going with the development and design teams around building the website, and without Basecamp I don’t know if we could have done it.

We also have projects for all the different parts of the music festival – the festival website, the merch booth, the stage flags, etc. Outside of the festival it’s been great having a project for each cover story and we also make a project for every major advertising campaign.

Getting a whole team on board with embracing a new system like Basecamp can sometimes be a challenge at first. What was this like for you and your team?

My favorite moment with that was for the festival. Pitchfork proper has been on board with it for a while, and there was a little bit of resistance here and there, but pretty quickly everyone was like, “OK, cool, I understand why this is a good thing and I’m into it and it works.”

For the festival, we were introducing it to people who have never used Basecamp before and I introduced them to Basecamp for the festival website like, “OK guys, here’s what Basecamp is, we’re a lot about it, here’s why it’s good, here’s why you should use it.” The next day I’m still getting emails from them, and I’m just like, “Can we do this in Basecamp?” Just for weeks this went on—“I know this is annoying but can we please just do this on Basecamp? ... It’s just as easy, please just do it this way. You’ll see.” It was so rewarding a month later to get an email that was like, “OK, Mike, Basecamp is fucking awesome and I get it now! This makes sense. I’m loving it.” It just kinda clicks and you’re like, “I get it.”

I think for you guys it’s probably designers and developers who would be the ones promoting Basecamp and trying to get other people to use it, but it’s actually more useful for the clients or other partners than the designers themselves because it gives other people constant access to all these files and other things that they wouldn’t otherwise have. We love it.

Our good friends, Davey Sommers and Gwen Harley, are currently running a Kickstarter to support the opening of their locally-focused grocery and deli in Chicago.

We’re big fans of increasing accessibility to fresh and natural food products. A few months ago, Emily posted about the unique CSA benefit provided to the 37signals team. You can see this means a lot to us. Nourishing yourself with fresh, unprocessed foods translates into feeling and performing better every day. You are what you eat. /play rimshot

Part of PLENTY’s mission is to serve underrepresented diets. Sensitivity or complete intolerance to gluten is becoming increasingly common in our world of processed food. I myself gave up gluten one year ago and shops like PLENTY help take the challenge out of living gluten-free.

Just like any startup, jumping into the food business isn’t easy. Davey and Gwen can use all the help they can get to make this dream a reality. Every little bit helps in working towards their fundraising goal.

I hope they’ll inspire you to support the cause.