One of the easiest ways to shoot down good ideas, interesting policies, or worthwhile experiments is by injecting the assumption that whatever you’re doing needs to last forever and ever. Which means that the concept has to scale from 5 people to 5,000 or from 100,000 users to 100 million. That’s a terrible way to get from those 5 people to 5,000 or reach those 100 million users.
To reach the top, you have to be willing to use all the tricks that makes sense at the earlier stages. That’s your advantage over the guys who are already sitting up there. So you’re not Google and don’t do a billion dollars in profit every quarter. But I bet you that you’re way more capable of quick, sweeping changes. When you have 100 million users on your email platform, you can’t do the same quick iterations that constantly push upgrades out. When you just have your first few hundred or thousand, you can.
So stop worrying too much about whether giving everyone in your company a credit card at 10 people is going to work when you’re a hundred times bigger. If it doesn’t, you change, come up with something that does work for that size.
The same with your infrastructure. We started on a single server for everything when Basecamp was first launched. There was no point in growing a huge farm of machines if the thing was going to flop anyway. Today we have many more machines and redundancies and surveillance and more because we’re at a different level.
The best way to get to the point of needing more is by optimizing for today. Use the strengths of your current situation instead of being so eager to adopt the hassles of tomorrow.
Oisín Ó Murchú
on 06 Mar 08I think I’ll have to agree strongly with this – I’m more than guilty of it myself a lot of the time.
It comes down, I think, to fear and being afraid of making a ‘mistake’ now that just might have to be corrected at some point in the future. And this blinds me to the fact there may not be a future (for the system\application\whatever) unless I get rid of the fear and just do it for today.
Andrew Brown
on 06 Mar 08Sounds like your an advocate for Practicing the Power of Now.
Jaan
on 06 Mar 08Words well spoken. The “build for the future, ignore today” approach is far to common. I see two main reasons.
1) Sometimes it is strongly encouraged by VCs against the express will/knowledge of the entrepreneur. Which is another good reason to keep outside funding out of the picture for as long as possible.
2) Inexperienced management who feel the need to “become a proper company” or simply aspire to play in a bigger (ego boosting) league before they even make it in the minors.
Both are equally destructive.
Matt Radel
on 06 Mar 08I completely agree. You just need stakeholders that are willing to continue development as the user base grows. Or even as the product ages. Sometimes that isn’t the case (as stupid as it sounds).
Patricia Garcia
on 06 Mar 08Wow. Now, I don’t know if this is because I am naturally high on my adrenaline right now, but this really spoke, no sang to me in terms of applying it to life in general.
Anson
on 06 Mar 08This is certainly a better approach than architecting the crap out of a project, layering in abstraction after abstraction, and deploying to an expensive, difficult-to-maintain cluster of super servers. All with a user base of zero.
But I think there needs to be a better approach. It remains to be seen whether Twitter will ultimately become a success story over the longer term but it seems to me they’ve suffered from this lack of foresight – from being to focussed on the now. Their technology hasn’t scaled well, and is constantly falling over. Very probably a lot of this is a result of poor (or no) decisions early on. Now admittedly they lead their field and have a large, growing user base but the question is how vulnerable are they to competition?
It seems to me the time is ripe for someone to come up with a lightweight blueprint for web apps that’s more than just “launch today and iterate often”.
This blueprint would involve some allowances for scalability, some guiding architecture principles (and these are my own unproven ideas) like designing your API before your website and becoming the first customer of it; like putting in proper safeguards on day one for bots and spammers. Too many projects seem to naively launch as if they’re living in some feel-good utopia instead of the opportunistic dog-fight the web really is.
And how many products do you see that don’t really work when the data and user base does arrive? Corkd is a great example of a site that became a slow, unwieldy mess once the masses started doing what the creators wanted and cataloging and tagging their wine. Still today it’s a barely functional shell of a product. Almost abandonware.
So I guess the moral of that particular story is another principle – test your application with ‘real’ data. It’s stunning how often this is neglected. Data validates or invalidates your design decisions and shows bottlenecks and problems with your UI. Data is the blood in the veins of your application. You are not Dr Frankenstein. Do not just flick the switch on launch day and hope your creation lives.
Anyway, I would love to hear your feedback on these ideas and whether I’m alone in thinking we could put together some kind of architectural best practice for web apps that strikes the right balance between the two extremes. Comment here or email me at ansonparker [at] gmail [dot] com.
Charlton
on 06 Mar 08I’ve seen the “become a proper company” attitude in management - and this was a team of experienced people who had worked their way up in traditional corporations and now found themselves interested in a startup. It killed the startup I worked at - the people putting forth the money had managed to attract a diverse, excited group of people who were willing to put in long hours to get the job done but didn’t always show up in business casual clothing or arrive promptly at 8:30.
So management instituted a dress code and started giving people warnings about punctuality. The brightest found other places to work; the ones who stayed, as it had been made clear that butts in chairs by 9 am mattered more than anything else, stopped working late. And as you probably know, when people have joy beaten out of them, they become “average” and no longer have the drive to be rockstars.
On the flip side, while it’s worth developing for now rather than for later, a little bit of forethought now saves a lot of hassle later. You don’t need to build a system capable of handling millions of transactions an hour if you’re currently handling a couple hundred a day, but if you made poor choices when building that first system, you’re going to have a much harder time scaling in place when you need to handle a million transactions an hour.
DHH
on 06 Mar 08There’s tons of architectural best practices out there. Many of them can be easily followed without being more work than the “naive” approach. Rails institutionalized a lot of best practices for software development, for example.
But different systems will need different best practices. The best practices for scaling Basecamp are not going to be equally applicable to something like Twitter (where a backend message queue system is of high priority). So it’s equally easy to make sweeping generalizations that won’t hold might weight when the specifics of a given application comes to play.
I think these examples you bring up are flawed in the sense that you assume that these people could just as well have gone the full nine yards on everything and still a) be in the position they are, or even b) care about it.
For Corkd, the guys sold it and haven’t been around to follow the normal progression of the site. Which, for them, seems like a good deal. They put the amount of effort in that was needed to reach their objective (get out). It’s the buyers who’ve dropped the ball since then, if the site is perceived as slow.
Also, I don’t think designing your API first is a good idea at all. In fact, I think it’s a terrible idea. The consumers of your API are far less picky than the consumers of your web interface. An application with a crappy web interface is unlikely to be a smash hit off a beautiful API.
Using your own application while you build is sound advice, though.
Joe
on 06 Mar 08Definitely good thoughts.
I work on an aircraft program (a “platform”, we call them) which has existed for a few decades. While your wisdom was applicable years ago when we were developing and growing the platform, those shortcuts nowadays keep our costs up and prevent us from maintaining the business as we’d like to, making us vulnerable to competition.
Not that you aren’t right – you are. But there’s another edge to that sword – years from now, you will have real challenges based on the early architecture decisions.
Sean Murphy
on 06 Mar 08I would make one suggestion: when you make a decision, record the likely benefits and under what conditions you should re-evaluate. For example in your credit card example you might say the benefit is that we shouldn’t waste employees time on minor expenses. We should re-evaluate when headcount hits 20 or the second time an expense review shows someone using poor judgment. Russell Ackoff calls this a “decision record” model. This can be a few sentences in the minutes from a meeting in a workspace or wiki page.
Adam
on 06 Mar 08I do this myself far too much, and then don’t deliver to shorter term milestones, or at least enough of “something to show” in the short term. Good comments to keep in the back of the mind.
Richard
on 06 Mar 08Can’t agree more!
But in embracing constraints you must embrace certain constraints in both directions and realize that some things are just necessary and shouldn’t be in the ‘we’ll do it some day’ list. These include things like:
- Security - Fail over (load balancers, servers) - Back up (recovery procedures AND knowing they work aka testing recovery)
The approach we all here seem to agree on is at times overly seductive and causes people to leave the proper and pragmatic things undone which unnecessarily exposes their clients and their company/investors.
Having been self employed since I was 19 (some 16 years) I definitely understand the constraints of finances as well. This being said financial constraints should keep you from finding the correct solution (sometimes creatively) for the necessities.
Great words of wisdom though.
Matt Brown
on 06 Mar 08Great post David!
Robert Long
on 06 Mar 08This is so right on its ridiculous!
Nick Husher
on 06 Mar 08Donald Knuth once said that “premature optimization is the root of all evil.” The complementary advice to this is that “premature scalability is the blight of an agile project.”
Striking a balance of preparing for the future while enjoying the fruits of your labor today is the great challenge of life, in all its facets. Advocating ‘sieze the day’ without ‘prepare for tomorrow’ is just as dangerous as the reverse.
MattH
on 06 Mar 08Just like with coding – Premature optimization is the root of all evil.
brad
on 06 Mar 08Heartily agree. And besides, thinking too far ahead can lead your brain into an anxious and unproductive state (eg. happiness plummets).
Anson
on 06 Mar 08Use Cork’d to find me a recommendation for a good Californian Pinot Noir under $20. You can’t. As the kids say: FAIL.
Maybe you have some inside information on Cork’d. If they just set out to build something they could quickly flip then sure, they succeeded. But as for building a community around wine recommendation it never really worked. And I think that’s because they never tested with a decent set of sample data. Which leads me to my original point:
With some simple guidelines in place beyond just launch and iterate Cork’d, and thousands of other applications out there, could have been much better products.
It goes beyond frameworks like Rails. I think there are some generalizable best practices that will work for all web apps beyond just this is how to talk to your database and organize your GIFs.
Leave it with me…
sandofsky
on 06 Mar 08When people form a band, the first thing they do is waste a rehearsal arguing over a name.
Jon Buda
on 06 Mar 08Couldn’t agree more. Well said. Early optimization and over-planning is a killer for web-apps. Desktop apps maybe not so much. But seeing how easy it is to launch a web app these days you might as well experiment often. Just go out and build something, or it might just get tangled up in hours long arguments over complete bullshit.
BillP
on 06 Mar 08I totally agree… with one specific caveat.
I build data warehouses and spend an inordinate amount of time tuning databases that were built by programmers. This means dealing with the usual culprits – UDFs, elegant stored procedure nesting, missing indices, and cursors a-plenty. Works fine with 10,000 rows, but literally chokes to death with 1,000,000 rows. Exponential scaling = really, really bad.
It doesn’t take any longer to write SQL code that scales nicely (set-based logic) vs. code that scales terribly (cursors). The real issue is spending the time learning about performance tuning & to know the difference between the two types of coding.
it does keep my family fed though…
Just my 2 cents.
Charles
on 06 Mar 08Isn’t this a variation of “YAGNI” (You Ain’t Gonna Need It”)? Either way, a great principle to keep in mind during design and coding.
Geoff
on 06 Mar 08I think the most important part of this post is the idea that when it no longer works, you change.
Change can be hard. Change can be scary. And part of addressing today’s challenge will inevitably require trashing the work that addressed yesterday’s challenge. One must be prepared (both emotionally, and in the way that BillP described above) to change.
On a micro-scale, I think y’all appropriately named this “Getting Real.” On a macro-scale, this is called creative destruction.
matt
on 06 Mar 08@Nick Husher: Unfortunately that is one of the quotes that is almost always taken out of context:
“We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil” A small percentage of the time then you should optimize up front. A typical example? Using proper string appending techniques to speed things up / lower overhead. Often when you have lots of small things that could be optimized it becomes much more difficult to make a simple change or two to have large performance increases (not like…using bind variables in oracle which in 95% of the cases will give large scale performance increases)
Other quote I hate that people cut off is “Great minds think alike, and fools seldom differ.”
pwb
on 06 Mar 08Great post! I particularly like “Use the strengths of your current situation instead of being so eager to adopt the hassles of tomorrow.”
I like when folks like BillP come along and say “yes, but…”. NO, NO, NO!!!
Or Anson who suggests “if they just took more time and energy to…”. NO, NO, NO!!!
SeanG
on 06 Mar 08This is a GREAT POINT and something that is worth keeping in mind when writing code!
This also reminds me of this awesome quote that guides me every day: “It doesn’t have to be perfect, it just has to be DONE.”
Thanks!
Mike
on 06 Mar 08great advice. I love the stuff that comes out of this company
matt
on 06 Mar 08One thing this reminds me of is the “throw it over the wall” mentality of some developers. While I’m not saying DHH and 37signals are these kinds of developers, please remember people need to support and operate your shit. That’s one of the reasons I dipped my toe into operations was to see the pain that they go through and how to improve the lifecycle of a system.
Erik Mallinson
on 07 Mar 08I found the same thing true in another context: We were blowing a lot of money on 37signals products. I took a look at what we were using, downgraded to suit what we needed today. We were able to free up some money to use elsewhere. I then realized I was able to use some of that monthly money to donate to a charity.
RF
on 08 Mar 08Designing your API as you design the Web interface can be one of those architectural best practices that pays for itself even in the short run. An app I worked on recently was a widget (embeddable in anyone’s site) that communicated with our servers through JSONP requests. It just happened that it was convenient to make our widget the first consumer of the API.
I think one of David’s implicit problems with Anson’s comment was that it has a “waterfall” feel—1) design API, 2) do everything else. For me there was a lot of back-and-forth: tweaks to the API as I recognized the different things the UI would need, etc. David also talks about the need to give the UI more weight than the API, and I agree: plenty of effort went into making the UI nice, and I had no problem with “corrupting” the API spec to be sure the UI would be snappy, etc.
(For instance, to make the product feel snappy and get 1.0 out quickly, I combined a few API calls into one. It would be best for the API to either force a separate HTTP request for each call or to add a generalized batching mechanism, but was best for UI perf and schedule to lump somewhat-unrelated things into one API call. I did what was good for the UI.)
And I think a common thread for the commenters is that it seems like David’s comment means “never worry about scalability.” You do have to worry at some point; it may involve adding caching in a panic or hiding resource-intensive features or adding servers really fast, or brief overloaded periods. Skilled programmers can handle all that without site-killing downtime, unless you’re a credit card processor or something. Reasonable levels of planning for load by running load tests and profiles are great, too; you just shouldn’t think about a million users when you haven’t yet managed to attract 100K.
We’re arguing over generalizations when DHH’s point is really, “do what you need now” - emphasis on “need” and “now” - which is hard to dispute. The right argument to have is over what your particular project needs now, which is one that should be happening every day within healthy dev teams.
M. Hom
on 10 Mar 08Good day! Team efficiency starts with a project team building, connecting and leading with a strategic overview that enables them to see how the initial objective connects with other objectives, that leading to the overall grand goal.
With a properly-built strategic overview, the team gets a 360 view of the entire process. They know the general priorities, the approaches and the circumstances. The overview enables the team to perform the following: see the critical path, focus on their strengths while avoiding their weaknesses, adjust strategically, etc.
#
@ the end, process precedes technology. Process is about preparation from top down. To get this standard of preparation running, it is important to having the proper professional experience and a good grand process that encompassing everything from initial preparation to building the strategic overview and then building marketing development plans, product development plans around it, marketing implementation plans, etc.
Does any companies possess any strategic project management process like this?
Ron Patiro
on 12 Mar 08Great post. Optimization is an absolute requirement in order to survive online. Amazon is a great example of a company that has held a strong focus on optimization. They continuously optimize elements of their site. When they roll out a larger redesign as they did several months ago, they already have a lot of the knowledge on how to do the redesign from the knowledge gathered in their previous tests.
This discussion is closed.