I recently read and watched “Moneyball”, and enjoyed both greatly. It’s a great story in and of itself, but I also found it to be an interesting parallel to the state of the “web software” industry today.

Moneyball starts in the week before the 2002 baseball draft, with a set of meetings that pit Oakland A’s general manager Billy Beane against his team of scouts. The scouts’ primary mechanism of evaluating players was visual – did the guy look, walk, and talk like a major league baseball player? On the other hand, Billy, with his assistant Paul DePodesta, had a largely objective system for evaluating baseball players based on things like how often they got on base.

Billy won the fight over talent selection and picked players that met his system, even if his scouts disagreed. This pattern continued throughout the season, and the A’s went on to set a league record for consecutive wins.

When I started writing I thought if I proved X was a stupid thing to do people would stop doing X. I was wrong.
—Bill James in his 1984 Baseball Abstract

In many ways, the “web software” industry is still where these scouts are. For most people, the primary way of evaluating their software is with their own eyes and emotions. Over the years, people have tried to bring some objectivity or framework to do thing this with things like “personas”, but the process is still a largely subjective one, just like a scout looking at how a player swings and never really looking at whether he gets on base.

The reality, of course, is that this is no longer necessary. Just like baseball in the years since Bill James coined “sabermetrics”, we have the tools now as an industry to do better. We can identify the outcomes we want to see, and we can objectively evaluate a design in the context of those outcomes.

It’s never been easier to test your designs and find out what works where the rubber meets the road. You can use a tool like Optimizely for any site or something like A/Bingo in a Rails app and have a test running in a matter of minutes. Measuring and understanding behavior in other ways has also never been easier—there are new tools and startups helping to do this every week.

For Billy Beane and the Oakland A’s, using data was about leveling the playing field between their meager salary budget and the huge budget of teams in places like New York and Boston. For the web industry, the playing field is already fairly level – it doesn’t take much more than a web browser and a text editor to build something. What data does for web software is reduce the role that blind luck plays. You’re more likely to – on average – find success if you evaluate your work using real data about the outcomes that matter.

You can choose to keep working like those scouts did and go on gut instinct alone. It might work for a while, but I think most people would say that baseball’s moving forward now, and the people who haven’t made the switch are being left behind. Our industry will move forward too—do you want to be left behind?