Dennis Eusebio asks:
Many design thinkers really advocate deep, rich design research. Its a lot of documentation, interviewing, meetings and even more documentation just to get to the planning/prototyping stage. After reading some of the literature 37signals has put out, it seems like you guys stray away from this workflow in general. What’s your general workflow like? Is it different from traditional design workflow?
We conduct research all the time. But it’s a different kind.
It’s important to know that research is guesswork. The further out you try to guess, the bigger your margin of error. Like you said, many people do research before they start building their product. They write lots of documents and diagrams and specifications. The problem is, it’s all hypothetical. Until you’ve actually thrown the ball at the wall, you don’t know how it’ll bounce back.
People often think that research is like preparing for a long journey. Like the more food you stow on the ship the safer you are. But you don’t want to go on a long journey. You want to take as short a journey as possible, and come back for a reality check. See how you’re doing, then go for another short trip. Long journeys are the death of software.
I said we do a different kind of research. Here’s how we do it. We build something, then we collect feedback on what we built. For unreleased products, that feedback is our own opinions and review process. Does it do what we wanted? Does it feel right? Is it easy to understand? Will it support more features and future changes down the road? In the case of released products, we always listen to emails and forum posts from our customers. In both cases, the feedback on what we’ve built arms us for the next phase.
It’s like a conversation. You don’t sit down at the cafe, listen to your friend for two hours straight, and then talk for two hours straight. You take turns, constantly going back and forth, and the discussion finds its way.
Of course, you might wonder how to start. We build products we need ourselves, so our initial research is made of our own wishes, itches, and frustrations. When it comes to client work, my best advise is to become friends. Spend time together and discuss what they do until you can see through their eyes a bit. Pay special attention to their language and the words they use. Make notes, but don’t put them in the documentation shrine. As soon as you know enough to build the first basic feature, do it. Break away for a week or two and build it. Then come back, talk about it, learn more, and decide what to build next.
Research can make you feel informed, but I guarantee that building real software in many small steps gives you a lot more knowledge and confidence. If you’re interested in how to better share headspace with your clients, I highly recommend absorbing the Ubiquitous Language pattern from Domain Driven Design.
Dennis
on 19 Oct 07Great response. I always wondered how well the iterative process applied to traditional design methods.
I mainly asked the question because every book I’m reading at the moment really stress the lengthy research period. Great to know there are other perspectives on the topic.
Lakshan
on 19 Oct 07Talking about Domain driven design did you guys feel any advantage of using spec tool such as RSpec to describe the cases ? Or would it be better to allow the code to do the speaking ?
anil (from last.fm)
on 19 Oct 07Increasingly I’m of the opinion that collecting quantitative statistics is as important (if not more so) as collecting verbal feedback on released features. The gulf between what people say and what they do is often huge, and given that we build web applications, we have the luxury of mining interaction flows, events/actions, impressions and of course the database for behavioural information.
You can build a real picture as to the use cases in your product if you ask the right questions of the data (you need to be capturing the data of course, but apache logs and a database are something every web developer has by default, and that’s a start). As long as the user’s privacy is kept intact, I think data mining is a designer’s biggest ally in feature design and possibly the most important component in design ‘research’ for web products.
This is how it fits in to the feedback loop: You release (or even beta release) a feature and capture verbal feedback. You then use this data to form hypotheses about aspects of your product. You then data mine to confirm or refute those hypotheses. At any point during new feature design you can also test a hypothesis about the current usage of your product that may inform your design.
Being able to interpret complex behavioural data is becoming a crucial skill in web product development.
Jeff Lash
on 19 Oct 0737signals has produced a number of great products using their process, so obviously there is something right going on. In general, I’m a big fan of your approach and output. However, the question and answer show a misunderstanding of how research can fit into a product development process.
First, “research” does not necessarily mean “spending a few months doing research, preparing a bunch of presentations and papers that no one will read, having a lot of meetings, and not generating anything out of it.”
Some research ends up being like that, but it all doesn’t need to be.
Research can easily fit in to the iterative process you’ve described, and many people are doing it effectively. Taking an hour to visit a customer or conduct some usability testing is not expensive nor time consuming, and doesn’t prevent you from still releasing software early and often. Why would anyone not want to get input from potential customers before releasing a product to the market?
Second, your process works great when you are a technologically-savvy small company creating software to be used by other technologically-savvy small companies. However, most people probably are not designing software for people like themselves.
What about designers of software for engineers to use to build skyscrapers? What about developers who are creating applications that accountants use for filing corporate taxes? What about the 23-year-old male programmer in San Jose who is working on a web site which is targeted at the 77-year-old grandmother in Topeka?
In those cases, some initial research is almost definitely needed to get the domain knowledge and understand the needs, goals, and behaviors of the target customer. It doesn’t need to be “a lot of documentation, interviewing, meetings and even more documentation just to get to the planning/prototyping stage”—a few days might be enough in many cases.
Take what you learned and produce some initial prototypes and designs; then, rather than relying on your own opinions and ideas, get feedback on them. No matter what domain you’re working in, you’ll always learn something new every time you watch someone else use your software, regardless of how well you think you know the customer or you think you are the customer.
I don’t think the process 37signals discusses on this post and elsewhere (blog, book, etc.) and a process that incorporates some hands-on research with customers are mutually exclusive. I do worry that by saying that “research is guesswork” and that your research is only based on your “own opinions and review process” and reading “emails and forum posts from our customers,” some will interpret that there is no need for actually getting out of the office and interacting with customers and end users.
RS
on 19 Oct 07Thanks Anil. Yes data is a good ally. Sometimes we run database queries to answer questions about how frequently certain features are used, for example.
Regarding the difference between what customers say and do, that’s true individually. At the same time, it’s worth looking at a pain point when a large number of customers write you about it.
Jeff, I couldn’t agree more. Whether designing for yourself or for clients, it’s necessary to have some domain knowledge before the first iteration and to continuously build on it.
Research is guesswork. That doesn’t mean you should never do it. It means you should know its limits, and don’t get fooled into thinking a pile of documents has more knowledge in it than your codebase or living breathing customers.
Matt Raw
on 19 Oct 07I think you’re fundamentally misusing the word research. Research is not a predictive act. Research helps us understand the way things are. It’s often tempting to derive implications for design from research; as a result research findings are often used as support for a software development proposal or prediction.
But ultimately I think your beef is with the act of prediction itself, which is indeed guesswork. Research in and of itself is not guesswork.
Alexander
on 19 Oct 07What this glosses over is that you yourselves are representative users of your target audience. If you were designing software for people you aren’t so familiar with, it’d be dumb not to do some preliminary research.
Robby Russell
on 20 Oct 07Echoing some of the other comments made, I believe this is a case of where there are different perceptions of what research is. We perform a lot of necessary research in our Design and Development process so that we can have supporting rationale for complex decisions that we make.
Every design and architecture decision made, should be able to be questioned… and the person asking it should expect that there be some rationale behind it.
When you’re communicating new ideas with an audience that isn’t you… yourself, you can’t afford to guess on your clients dime. Oddly enough, we often find ourselves using research as a tool to protect our clients from their own ideas. With supporting evidence, we can have an open Dialogue with our clients and echo our concerns about a design or feature idea in an effort to reduce their margin of error.
In the case of 37signals, they appear to be building applications that they’d, themselves, use… which is great, so the need for research is reduced as they know what they’re trying to acheive and for whom (themselves).
In my the case of my firm, we’re building a handful of large projects a year for very different target audiences and without a sane process for conducting research, we’d not be able to design and develop solutions that meet the needs of our clients and their customers.
The key is to avoid spending too much time trying to research too much information, too soon. This is why we work on very small pieces of the puzzle at a time and iterate throughout the project cycle. Research, to some degree, is almost always happening, but it’s not happening on the entire project before an Interaction Designer works on wireframes for a new interface, or a developer implements some new code. We’re all working closely together and we keep the requirements gathering and research as close to the “final” implementation as much as possible.
In a nutshell, research isn’t the evil monster that people tend to label it as. Research is a tool and it’s good for helping solve certain problems, but it’s not going to solve them all.
Anonymous Coward
on 21 Oct 07I think the point is not that research itself is bad, just “deep, rich design research [with] a lot of documentation, interviewing, meetings and even more documentation just to get to the planning/prototyping stage.” That’s what agile development is against – not studyin’ stuff.
Research can only tell you, at best, what your application should do, in very general terms. The “release-and-listen” approach adds to this the ability to tell whether your application is doing what it should do. Present vs. future. Truth vs. speculation.
///ark
This discussion is closed.