Some notes on the building of CodeZoo Marc Hedlund 15 Apr 2005

19 comments Latest by Rick

Last week, O'Reilly (where I work) launched a new site called CodeZoo, which is a directory of free, reusable components for software projects. The launch was a big success, with an instant Slashdotting and a lot of fantastic feedback from early users, and Jason suggested I write up some of the things we learned while doing the project.

Macintitis

I'm starting with Macintitis because I first learned this general lesson in 1994, but apparently not well enough. Apple has succeeded in getting most of the CodeZoo team to switch to Mac, and as a result we spent much of our time reviewing the site during development through Mac browsers and Mac monitors. Of course, that's stupid -- most visitors to CodeZoo have not yet made the switch. As we got towards launch, we found that the site is terribly broken in older versions of Mozilla/Firefox on Linux, which is a more important platform for O'Reilly than for many other sites. Dumb, dumb, dumb.

You can't assign developers a primary work platform just to ensure diversity, and expect them to be happy. As Jason pointed out in my interview with him on ORN, developers will be happier using whatever programming language with which they're most comfortable, and of course the same applies for text editors, web browsers, and operating systems. You do have to make sure, though, that someone is looking at the site through all of the platforms you care to support, very frequently. Don't leave browser and platform testing to the end. Each new design, template, or page should get a once-over in any browser a user is likely to use -- and probably a bunch they're unlikely to use, too.

My friend Nelson told me he planned to look at the site in Lynx as a way of testing it, and what do you know, it broke in Lynx. Does that matter for anyone other than Nelson? Well, Lynx is a good lowest common denominator, and if it works in Lynx, it probably works on cell phones, airport web terminals, WebTV sets, old versions of Netscape, and whatever other freaky browser a user might choose to use. You know what? All of the browsers I just mentioned already show up in CodeZoo's access logs. So yes, Lynx is a great test. Use it.

Examples

I really like the 37signals "blank slate" model for just-opened interfaces, and I wanted to do something similar. CodeZoo allows users to submit Amazon-style ratings of components, and also to contribute tips for how best to use components. We used the 37s "EXAMPLE" model to show what reviews and tips will look like on a page if none have yet been submitted.

While Jason and I were chatting before launch, he asked if the model was really useful in this context; there already is plenty of component descriptive content on our pages before the user gets to it, so why do we need examples? We wound up agreeing that the good thing about this use is that the user gets good review style modeled for them -- they can see appropriate length, tone, and detail in the example, and use that in their own contributions.

Unfortunately (Macintitis again), we didn't test the visibility of the red "EXAMPLE" text on different monitors, and some users couldn't see the red text on their setup. They thought the Star Wars-themed example reviews were bugs in the system. "Why is R2-D2 reviewing Hibernate?" Again, assume that every wristwatch-with-a-browser will hit your site at some point, and plan accordingly.

Playing dead

We used Joel Spolsky's "playing dead" model for inappropriate user submissions -- a user who submits a tip or review will always see it on the site immediately, but other users won't see it unless and until it is approved by the moderators. If the submission is abusive or inappropriate, the submitter will think they've succeeded in spamming or subverting the site, but no one else has to worry about that. Suffice it to say this has already been a huge help. You have to assume people will try to use any community software to spam or otherwise misuse the system; playing dead is a great way to minimize the harm of that misuse.

Reduce, reduce, reduce

The greatest interface work we did on CodeZoo came from reducing the number of choices a user had to make to get a component. The worst parts of CodeZoo, in my view, are where we didn't reduce enough. Reduce, reduce, reduce, and then reduce some more!

Many other open source code repositories will revel in options for a user: what packaging format do you want? which version? what platform? which variation? We started with a simple goal: for any component, there should be one button that allows any user to get that component. We called this the "buy button." Where there were multiple versions available, we chose the most recent, stable version and made that the default. Where there were multiple packages, we chose the Zip format (which is widely available on any platform) and got rid or tar.gz, tar.bz, .exe, and everything else we could. If there were many parts of the component, we defaulted to the one file that gave the user the most value in a single download. Other files, versions, and packages fell to the bottom of the page -- available for the users that absolutely need it, but far from distracting for the users that just want to get their work done. Reducing choices makes the site more useful, not less.

We did the same thing with RSS feeds, though maybe not as successfully. I couldn't figure out why any site needs to publish RSS 0.91, 0.92, 1.0, 2.0, and Atom -- doesn't this make confusing terminology dramatically more confusing? -- so I just chose Atom. It turns out that many people seem to recognize the term "RSS" but not "Atom" -- we got a bunch of requests for per-component RSS feeds, which are already present, labeled as "Atom Feeds." And boy, wasn't I surprised when I went to test the Atom feeds in NetNewsWire, and found that the current stable release of NNW -- 1.0.8 -- shows a "Feed not found" error when you try to subscribe to an Atom feed. Huh!? The 2.0 beta does support Atom, but it's pretty dumb that 1.0.8 gives a "not found" error -- what it really means is, this format isn't supported. I'm amazed that, a full year after Blogger switched to Atom-only feed support, any popular feed reader would give such a confusing error message for Atom, not to mention failing to support Atom in all current releases.

We could have reduced more, and we should have. I was joking with Jason that while many software releases proudly announce, "Now with seventeen new features!" I'd be happy to put out a release that proclaimed, "Now with seven fewer features!" If you think you've removed everything you can, try harder. You'll be glad you did -- the site gets more usable as it gets simpler.

Web tools

We made very heavy use of Mantis and Basecamp while developing CodeZoo, and I just can't imagine doing another project without them both.

Mantis is my bug tracker of choice; it is a free, web-based PHP/MySQL application. If you're not using a bug tracker on a development project, you should really give it a try; and if you're using Bugzilla, please, get help. Bugzilla is browser-based masochism. Don't try to keep track of issues in email; put them in a database. Mark each issue with a priority, and agree on a priority level above which an issue must be fixed to launch. Bugnotes are a huge savior when the going gets tough. Being able to assign an owner to an issue, and hand off issues from one owner to the next with history attached, is a huge time saver. Bud Tribble, one of the first Macintosh engineers, once told me the Mac team got rid of the distinction between features and bugs, and just tracked "tasks," which is what we did, too. Why waste time arguing about the genus and species of work that needs doing?

We also used Basecamp very successfully. We had a widely distributed team (not international, like 37s, but bicoastal), and for milestones, to-dos, and ongoing discussions, Basecamp was a huge improvement over email. We didn't wind up using the file attachment feature -- we checked all files into source control, and just pointed each other to the file path -- but otherwise, we used every feature Basecamp offers. As a long-time MS Project hater, it's so nice to feel love for project software. What a relief.

Daily calls

One of the practices I inherited from XP was the idea of a daily check-in meeting. For our distributed group, we used a dial-in 800 number for a daily, 10-minute call. This was a great help in keeping people together and in touch. For a long time, two of the three people on the team had never met in person, so at least hearing the voice of the other person helped them feel connected and personally involved. Some days, the call was pretty useless; most other times, we caught things that had fallen through the cracks or came up just through chatting. I'm a big fan of some supertextual connection for at least a few minutes a day; smileys and frownies are no substitute for the enthusiasm or anxiety you can hear in someone's voice.

Small teams rule

At my last company, the engineering team started with eight people breaking ground on the first day, and grew substantially from there. For CodeZoo, we had three people for most of the project, and a fourth at the end. I took care of writing up the idea, making the business parts happen, doing quality assurance, writing the text for the site, and being the "catkiller" in the room. (At Organic, where I once worked, the term "catkiller" came out of a client project in which the interface showed a black cat from the earliest version of the design. Each time we did a design review, internally or with the client present, someone would ask to have the cat moved from one place on the screen to somewhere else on the screen. The cat became the focus of many discussions and many more revisions. For the final review, the highest authority at the client finally saw the interface for the first time, and the sum of his comments was, "Love it. Kill the cat." This became a new rule at Organic: have the catkiller in the room early, and you'll save a lot of heartache.) One engineer did everything with the frontend and about half the database work; the other engineer did the rest of the database work and wrote the "crawler" that went around the net finding new components for us to include. Towards the end, a graphic designer joined (which should have happened much earlier; it didn't for practicality, not preference). That's it.

The rule at 37signals, and Apple, and Amazon, and at many other teams whose work I respect, is the same: keep the team as small as possible. Metcalfe's Law, that "the value of a communication system grows as approximately the square of the number of users of the system," has a corollary when it comes to project teams: the efficiency of the team is approximately the inverse of the square of the number of members in the team. I'm beginning to think three people is optimal for a 1.0 product release. Going back to "reduce," start out by reducing the number of people you plan to add to the team, and then reduce some more. (As Jason points out in his talks, this means each person on the team needs to have many skills.)

Domain name transfer

We wound up purchasing the codezoo.com domain name from someone who owned it already, and in the process, I had a brutal introduction to the world of domain name registrar transfers. In the name of openness and deregulation, ICANN allows many companies to register a domain name, and then manage the registration of that domain, for you or your company. The past owner of codezoo.com had registered it with a registrar named Omnis, and O'Reilly manages its domains with another firm. We had to transfer the name from Omnis to our registrar in order to use it. Of course, that doesn't benefit Omnis at all, so they did everything possible to prevent the transfer from occurring: holding requests for as long as they could, directly lying to us on the phone about it, day after day, and eventually refusing to take our calls at all unless we wanted to sign up for services from them. Towards the end of this process, when we had launched the site as codezoo.net since Omnis had prevented us from launching with the name we'd bought, I realized: "Wait a minute! I'm dealing with the phone company, that's what this is!" Ma Bell is gone, and in her place, the new locus of bureaucratic metastasization is domain registries. I guess the Internet really has grown up! Don't do business with Omnis, and if you plan to transfer a domain name, allow more than a month for the whole thing to happen.


CodeZoo was a very enjoyable project, and I think it shows in the results. We learned a lot from 37signals, both in their approach to building software and from using the software they make. I'm hoping we can continue to improve the site, and the way we make it, and pass on whatever we learn in the process.

19 comments so far (Jump to latest)

Matt 15 Apr 05

Marc, thanks for the excellent write up. Lots of valuable advice, particularly the “playing dead” model.

Could you elaborate a bit more on your preference for Mantis as a bug tracking tool? I agree Bugzilla is a UI nightmare, but from a quick look at the demo website Mantis doesn’t seem to fare much better.

Marc Hedlund 15 Apr 05

To be honest, I haven’t deeply analyzed my appreciation for Mantis; I just know it. I know also that the demo site is several versions out of date. But I’ll try to be more verbose….

The view issues list has visual markers for the state of bugs that work well for me. The state of the bug is shown in a color system that maps well to my feeling for urgency; hot pink needs action, light grey is closed, and so on. Lots of bug systems have this, but for whatever reason the scheme in Mantis makes sense to me. Priority of bugs is shown with red ‘^’ marks for high, and inverted, blue ‘^’ marks for low. The sorting and filtering are fast and intuitive to me. I haven’t ever needed to look at the manual in order to figure out how to make a feature work.

I also like Mantis’ use of email. I get email when I would want it and not when I would not. If a new bug is filed or a bug I filed has an update, I get an email; if I manually subscribe to another bug, I get an email. If I don’t have reason to worry about a bug, I don’t get email about it.

There are a few too many fields and options, but they don’t slow me down. (I agree with Joel Spolsky, whose competitive FogBugz product sharply limits the number of fields, that fewer fields make for better bugs. Last time I looked at FogBugz, though, its UI was less useful to me than Mantis’.) The preferences are too numerous, but the defaults work well for me.

I hope that helps somewhat. The feedback from the other CodeZoo developers (who’d not used Mantis before) was similarly positive. It’s free and easy to set up, so if you have time to give it a try, I’d suggest it.

Tom Reinhart 15 Apr 05

What a useful write-up! As a programmer who has never really looked into usability, the examples here are immediately useful, and, looking back, seem like they should have been obvious. Thanks for the insight.

Andrej 16 Apr 05

So true, the bugzilla UI really is a nightmare. We’re using iTracker for our project, the user interface is pretty good, Just as you say, they’ve removed all which isn’t really required. And if you’re missing anything, you can add it as a custom field. Only problem is that the current release is pretty slow, but this has been fixed in cvs.

We’ve added a custom field in itracker which holds the estimated number of hours to implement a task/bug. And we’re basically using 2 reports to see how the project is going: 1) open tasks/bugs grouped by owner, 2) closed tasks/bugs grouped by week. That’s all a project manager really needs to know to manage a project.

Jeroen Mulder 16 Apr 05

Very interesting write up. I always enjoy reading how people go through a project — especially the “Blank Slate” and “Playing Dead” tips are something I’ll carry around now!

I am left with one thing wondering though. How long did you guys spend on this project?

Mario Parise 16 Apr 05

I made the mistake going through Omnis about a year ago, due to what seemed to be a ridiculously good price ($7.95/year domain registration). It was the worst thing I have ever done.

Periodically, my website will actually be forwarded to their advertising page, claiming I’m “parked for free”. This would last an hour or so and the go back to being okay. In my e-mails to them they deny it ever happened.

So though this comment is partly off-topic, I just want to support to idea of not using Omnis.

Some Guy 16 Apr 05

A bit of conflicting information here - Be usable by every browser, but support only one RSS feed? RSS is just another browser! Maybe the better thing would be to have one default RSS feed with a link for “other feeds”?

seth 16 Apr 05

I’ve never heard about “playing dead” before. Awesome tactic that I’ll have to put into my next project.

Lea 17 Apr 05

Thanks for the writeup, Mark - I don’t think too many webbies working on larger sites are putting into words the challenges they are finding in the development process. Its valuable to put it Out There, and in the long run it will make the web a better place :)

(and, yes, NNW not supporting Atom is a right pain :( )

jsp 17 Apr 05

Marc: thanks for this article. Very interesting glimpse into the process. Love the “catkiller” anecdote. As to tools, I remember looking at Mantis and thinking it was a bit clunky. Have you looked at Trac at all?

Some Guy: RSS is not a “browser” at all, it’s a format, just like HTML. Except it’s a really bad, fragemented format, which is why it’s completely appropriate to choose the well-documented, validating variety: Atom.

Alex King 17 Apr 05

Great post, very interesting.

we checked all files into source control, and just pointed each other to the file path

What did you use for source control? Did you have a particular server or client that you liked?

Marc Hedlund 17 Apr 05

Some Guy: Atom isn’t an RSS browser, Bloglines is. My hope had been that all of the RSS readers would support Atom by now, but as I say, the current release of NNW, 1.0.8, does not. Had I discovered this earlier I might have switched to some other format since, as you say, the goal was to support everyone.

jsp: I haven’t looked at Trac, and will. I realized after responding to Matt that FogBugz has gone through two major releases since the last time I tried it; I’m planning to take another look, and if anyone has used it recently, I’d love to hear comments on it. The major point, though, is to find some system that works for you, and in my opinion that shouldn’t be email. Mantis works for me, but maybe years of Bugzilla suffering make me more tolerant of Mantis than I would be otherwise.

Alex: We used CVS for source control since that is the O’Reilly standard. My preference is Perforce. I haven’t done a major project with Subversion, though, and need to give it another review (I last used it well before 1.0).

JF 17 Apr 05

We use Subversion and Trac here at 37signals.

Jonathan Nolen 18 Apr 05

If you’re evaluating bug trackers, you owe it to yourself to take a look at JIRA (http://www.atlassian.com/jira). Hands down the best bug tracker I’ve ever seen. And it has an interface that even 37Signals could love. We’ve been using it for more than three years and it’s been everything we could hope for from a bug tracker. Highly recommended.

Chris Lu 11 May 05

Hi, Marc,

I found CodeZoo’s search works, but not great.

If possible, I would like you to take a look at DBSight (http://www.dbsight.net). It’s a search engine speciallized for relational database.

Chris

Rick 19 Apr 06

Marc, can you comment on how you integrated the use of Basecamp and Mantis? It seems that an issue tracker can serve at least some of the functions of Basecamp, and we often put “to do” items in our issue tracker. Did you find there to be a lot of overlap between the two and that redundant work occurred (entering the same thing manually into mantis and basecamp)?

You may email me at rmann ATSIGN latencyzero YOUKNOWWHATGOESHERE com

Thanks!