We blame bureaucracy for being wasteful and taking too long when things like the Denver International Airport or Boston’s Big Dig arrive years overdue and billions over budget. But it’s not just huge organizations and the government that mess up planning. Everyone does. It’s the “planning fallacy.” We think we can plan, but we can’t.
Studies show it doesn’t matter whether you ask people for their realistic best guess or a hoped-for best case scenario. Either way, they give you the best case scenario. It’s true on a big scale and it’s true on a small scale too.
We just aren’t good at being realistic. We envision everything going exactly as planned. We never factor in unexpected illnesses, hard drive failures, or other Murphy’s Law-type stuff.
If you believe 100% in some big upfront advance plan, you’re just lying to yourself. There’s a good side to this too though: You’re liberated. That messy planning stage that delays things and prevents you from getting real is, in large part, a waste of time. So skip it. If you really want to know how much time/resources a project will take, start doing it.
Derek
on 12 Jun 09I appreciate your perspective but this is one of the times I’m going to disagree with you.
1 – If you truly know your business you can accurately estimate time and effort. I always walk through feature sets and come up with an estimate based on the different “buckets” involved. At the end of it I’ll mark it up 20% to account for all the unknowns. I’ve never painted the best case scenario to a client, probably in most cases I paint more the worst case because it ends up being right more times than not.If I tell them it’s going to take 200 hours and it ends up taking 125 they’re going to be happy to have saved money, time etc. If I tell them 200 and it takes 250 they’re not going to be thrilled.
It’s all about setting appropriate expectations.
2 – In a sales type of scenario it’s going to be really hard to get clients to sign up for an open ended contract. If you have a long standing history with them they may be willing to do this, but in a startup scenario you’re going to miss a lot of opportunities.Tathagata Chakraborty
on 12 Jun 09Now try to convince the managers in the run-of-the-mill software establishments about that :) If there is nothing to plan then what do the managers while their time doing? There are managers who try prove that they are doing something worthwhile by estimating and re-estimating till the project goes down the drain.
John
on 12 Jun 09Over and over I see people doing scientific simulation studies where they simulate what they hope will happen, or maybe what they expect will happen. But they really need to simulate what they fear might happen too.
Daniel Tenner
on 12 Jun 09Like most such pieces of wisdom, it is useful in certain contexts. What’s true for an innovative web product isn’t true for, say, a bridge.
If you step out of the IT bubble you might find that some people do manage to plan things properly in fields other than software engineering.
Mike
on 12 Jun 09Whenever someone asks me for something I think about how long it will take, then I triple that.
I know they will be back to ask for it early anyway.
Dean
on 12 Jun 09I think you are correct … but only partly so. Yah planning never survives the skirmish with the action. Planning does however create a web of thoughts in our minds that allows us more effectively to capture those things that are important to our projects as we go through the messiness. So I say go ahead and do some planning, and stay flexible in recognizing the important things along the journey that will help you get to the end alive and intact.
GeeIWonder
on 12 Jun 09The linked article focus on a couple of interesting, but certainly biased examples.
Academic projects are done, and only done, once an external observer (e.g. supervisor, committee) decides they are. These observers may have preconceived timelines of their own, for a variety of reasons ranging from funding to honest-to-goodness ‘cure time’. And by the way,guess which side of the fence the guys doing the study fall on?
The rest can be mostly chalked up to anectodal or confirmation biased evidence.
Elsa
on 12 Jun 09Government projects like the Big Dig have no choice but to run longer and pricier than the initial projections. That is how the contractors got the job, by saying they would do it quicker and cheaper than the other guys. Everyone knows that in reality it will be way over budget and take forever.
The only thing that angers people is that (at least for the big dig) was that they couldn’t do it right. Embezzle all you want, but get it done right.
Eric
on 12 Jun 09General Dwight D. Eisenhower once said, “Plans are nothing; planning is everything.” He went on to say that once the battle begins, nothing goes according to plan. But the act of planning makes it that much easier to improvise in the heat of the battle.
Joe Casabona
on 12 Jun 09I agree 100% that bureaucracy is the main reason things don’t get done. People are so wrapped up in the process, the planning, the check points, and making sure everyone is happy that nothing really gets accomplished.
I had to give a presentation to a potential client where the “head of the website committee” was 30 mins late and I didn’t get the job because the other guy was cheaper. They knew that before hand because they got proposals from both of us. Now I wasted time preparing/giving a presentation (as did the other developer), they wasted time listening to them, and their website is weeks behind because they need to go through this whole unneeded process.
Kyle
on 12 Jun 09Trying to plan everything 100% up front is definitely unrealistic, but a bit of upfront planning is worthwhile. It helps you identify your goals and an overall plan of attack. Jumping blindly headfirst into projects almost never ends well.
So while it’s definitely impractical to overplan, it’s equally impractical to underplan.
Dave
on 12 Jun 09Put aside the actual entity that governs Boston and pretend there’s a single Czar who can make any decision. That’s the only way your proposal is remotely possible….
Czar: “I don’t like that elevated highway. Tear it down and build a tunnel through the city. While we’re at it, build another tunnel to Logan and a bridge to the north-west.”
Person tasked with making it happen: “OK, who shall I assemble to start planning this project?”
Czar: “Planning? ‘Skip it… just start doing it’. Don’t assemble any engineers, project managers, architects, or planners. Just get the contractors and dump trucks down here. It’ll take whatever it takes.”
Do you seriously think that’s the best solution? Instead of the actual years and billions over-budget and a single civilian death, lack of planning the Big Dig would have resulted in something much worse.
‘Just do it’ sells sneakers. Months of off-season game planning and preparation wins championships.
Don MacDonald
on 12 Jun 09If only all projects could be as poorly planned as the Big Dig. Yes, it cost more and took longer than expected. Yet it, like the Sydney Opera House in the linked article, turned out to be a stunning success. It has given Boston the beautiful Zakim/Bunker Hill Bridge—already a city landmark—reconnected the North End to the rest of the city, given back acres of land that was previously a wasteland under the elevated central artery, and provided direct access to the airport from the Mass Pike. I’d say that, pace Elsa, although they screwed things up along the way, at the end of the day, the project accomplished all of its (incredibly ambitious) goals. I’d say that’s “getting it done right.”
The first time I stood in the North End and looked downtown and saw not a hulking green overpass with a dark wasteland beneath it, but a park with government center and Quincy Market beyond it, I thought “Damn. It was all worth it, wasn’t it?”
Don MacDonald
on 12 Jun 09Oh yeah. The planning part. I got so caught up in my mash note to Boston that I forgot about it. Fortunately Dave wrote pretty much what I would have written better than I would have written it, so I’ll just second his remarks.
Dave
on 12 Jun 09@Don, I’ll second your remarks as well.
If you only define a project’s success as it’s ability to stick to the timeline and budget, then the Big Dig was not successful. But, like you said - standing on the waterfront and seeing the city - was the vision of the plan. By that standard, the Big Dig is a huge success.
pwb
on 12 Jun 09“What’s true for an innovative web product isn’t true for, say, a bridge”
That’s the whole point! Just because you have to plan a bridge doesn’t mean you have to plan software!!
pwb
on 12 Jun 09And if you are planing software, here’s a trick: cut the scope in half and double the estimate.
Dave!
on 12 Jun 09Define planing? I’d argue you’d better do some planning when you’re writing software… you’d better define what it’s supposed to do, for example.
Planning is not over-rated. It’s just (apparently) not done properly very often. There’s a huge difference between proper planning and no planning. I’m pretty confident that a project that has some level of planning-even just skeletal outlining-will end up better in the end than something that has absolutely none at all.
Dave
on 12 Jun 09@pwb Just because you have to plan a bridge doesn’t mean you have to plan software!!
But this article isn’t about software. There’s no mention of software or web products at all.
The only projects cited are the Big Dig and Denver Airport.
Jay Godse
on 12 Jun 09Software development is mostly design and almost no production (except perhaps loadbuild and deployment). With software, production is essentially free. Design is not predictable. Neither is innovation. In these cases, “just do it” is usually the better strategy.
Construction projects have much less design relative to production. Also, construction production, such as the digging of roads, building of bridges, etc, is very expensive. When production costs dominate, and production is expensive, planning is very important.
Cody
on 12 Jun 09I like that quote from Eisenhower. I think it can be useful to envision possible scenarios, but it’s easy to get carried away.
NewWorldOrder
on 12 Jun 09Like most things, planning is useful up to a point. After this point, your reward for your effort in planning isn’t handsomely rewarded.
For instance, if you were planning to go to Vegas, you would do some sort of planning. For example, you may check when you could get a flight into Vegas. You’ll then organize yourself so that you can make your flight given traffic and your required check-in time for your flight. If your goal is to have fun in Vegas, then doing things ad-hoc once you get to Vegas may work. Planning things to do may not provide that much more fun for you compared to doing things ad-hoc—b/c fun is fun after all.
However, if you want to optimize your fun (whatever that means for you), you may want to plan for things you’ll definitely want to do in Vegas.
ML
on 12 Jun 09It’s not that all planning is always bad. It’s just we give it disproportionate value compared to what it’s actually worth. And often, we use it as an excuse because it’s easier to talk about stuff and write stuff down than it is to actually build something.
Doug
on 12 Jun 09Wait, stop, hold on. The problem is not with planning. How the hell would you do a project like DIA without a plan. Just let people drive bulldozers around a big field east of Denver and hope an airport forms organically. You have to plan, in fact you do plan.
Your problem is with estimating how long it takes to complete the plan. It is an estimating fallacy, not a planning fallacy.
If you do safety critical software, and need to have accurate time estimates, use the PSP from Watts Humphrey. It is difficult and time consuming.
Another method that works is Wideband Delphi. (It has nothing to do with the Delphi programming language.) http://www.embedded.com/columns/breakpoint/49901892?_requestid=328213 Add and flash warning.
Schedule estimating can be done accurately. Very few people are willing to make the effort. That is not a reason to have no plan.
mknofp
on 12 Jun 09I work for NASA and have to provide what we call SWAG’s (Scientific Wild Ass Guess, yes this is actually what it stands for) for all of our software projects (I’m a developer). This means I am required to give a time and resource estimate on projects that I know little to nothing about.
My current estimation philosophy is this: expect everything to go wrong, determine how much time (in days) it will take you to fix all that broken shit even with the forces of evil trying to destroy you at all times, then double that, then add another 25%. Thats the final estimate (SWAG) and it’s been pretty accurate so far.
Alejandro Moreno
on 12 Jun 09@mknofp: That’s hilarious! I’m totally using both the acronym and the formula next time!
nate
on 12 Jun 09managing expectations is important when dealing with clients of any type. if your client expects you to wing it, and everybody involved is on board, go for it. but most of my clients expect to have some idea of estimated time and cost. in some cases, the estimated time is even more important than cost, as downtime ends up being more costly than anything else.
i’m only speaking from my experience with IT consulting, not software development or anything else, but managing expectations applies to any interpersonal relationships and deals we make.
both overplanning and underplanning definitely lead to problems, just as overly complicated bureaucracy. setting some realistic guidelines and being flexible enough to reassess periodically is a good rule of thumb.
Get A Grip
on 12 Jun 09NASA’s probably not the best example to follow, not being able to convert between metric and imperial and all…
Brian
on 12 Jun 09I would love it if our clients would let us just start the project without an estimate and plan. However, they all want fix-priced time-boxed projects. A better blog entry might be how to convince stubborn clients that getting quality code is as important as their made-up deadlines.
Brenton
on 12 Jun 09In an ideal world, clients would tell me their budgets and their goals. I’d work through their budget until a) the goal is accomplished or b) the budget is exhausted.
Occasionally, I get a project like that (usually from a big ad agency). More often than not, people like estimates. Estimates suck, since on especially small projects, little wrinkles end up taking a disproportionate amount of time. Such wrinkles are really hard to plan for.
Something that should take 90 minutes can take 6 hours some days. So it goes.
Scott Semple
on 12 Jun 09The tech world’s current trend of anti-planning blog posts is approaching cliche status. It makes me throw up.
ML commented with: “It’s not that all planning is always bad. It’s just we give it disproportionate value compared to what it’s actually worth. And often, we use it as an excuse because it’s easier to talk about stuff and write stuff down than it is to actually build something.
This is absolutely true. Many people value planning more than it’s worth. But it doesn’t follow to conclude that therefore plans are worthless.
Fixating on a plan as a guaranteed solution is a mistake. But assuming that there are human beings over the age of 12 who think a plan is a minute-by-minute blueprint of the future is ludicrous.
Like it or not, 37signals plans just like everybody else. Once upon a time, someone at 37signals decided, “We’re going to build a simple project management tool.” That’s not very detailed, but like it or not, it’s a plan. It redirected the status quo and changed the future. That’s what plans do.
How many world-class athletes do you know that have never used a periodized training schedule and diet plan? How many world-class musicians do you know who didn’t cut their teeth on twinkle-twinkle and then follow a typical progression from there to mastery?
Whether you call them plans, intentions, direction or progression is irrelevant. Imagining where you are today, where you want to be, and deciding on the first step in between is a plan.
xo
on 12 Jun 09Suppose you need to actually execute a project such as the Big Dig. Your suggested approach is to just execute the project, and we can find out how long it will take and how much it will cost when it’s done. This seems like a reasonable approach for a software development job that you know will conclude within a few days or weeks, but for anything more it’s irresponsible to you and your client to not even bother making educated and ongoing estimates of time and materials.
Does your software business need to do any forecasting for expenses, and income? If so, then it seems wise to include your largest liability (software development costs) in your business management calculations. If not, then by all means don’t bother planning, but don’t whine when you lose your shirt because you didn’t bother to assess what the future held.
Software development planning and estimating may be difficult, but not bothering to assess available information that has an impact on your project and business is irresponsible.
Bob Barnes
on 12 Jun 09“Plans are nothing; planning is everything”, Eisenhower
MT Heart
on 13 Jun 09For the big government jobs, don’t the defense contractors get paid according to agreed to milestones? They won’t get a milestone payment without a milestone delivery. Which means planning.
There is also the sunk cost fallacy to consider when things don’t go according to plan.
Gavin
on 13 Jun 09Completely disagree. It’s a silly idea.
schwa
on 13 Jun 09@derek the 20% rule is great. i usually use 15% but that extra 5% really accounts for long-term financial progress.
imho, the best page of getting real is the implementation scenario - how million-dollar ideas are scaled to a tenth of their value (or worse) unless they’re done. and getting them done (ie getting real) takes planning - and then doing.
funny that basecamp is all about helping achieve milestones along the way. isn’t that planning?
Scott Perez
on 13 Jun 09Thanks for the post. Can I just say that this blog is pure spun gold. The information and ideas you guys provide are terrific.
I can’t wait until your new book is released.
Duke Nukem
on 15 Jun 09We tried this approach. Didn’t work out so well.
Matt Henderson
on 15 Jun 09Just curious: When you guys were working with potential publishers for Getting Real 2.0, did you require them to commit to any sort of plan, timeline and/or costs (and did they require you to commit to a plan), or did you just skip all that and go for it?
Jeff Small
on 15 Jun 09Matt Henderson leads to a good point Business requires a certain amount of planning. There’s no way around it.
There needs to be a distinction between service work and product development work. Product development work can (sometimes) be done in a more loose manner. you have product managers that want to see something working quickly and make decisions based off of prototypes.
On the other hand, if a client wants you to build software, they want a contract, want to know how long it’s going to take and what the cost is. If you tell them, let’s just get going and we can work out the requirements and schedule as we go along, they will find another vendor.
Now, waterfall processes don’t work entirely, but you can’t completely throw out planning. Yes, overplanning is an issue, but underplanning can be equally as detrimental.
Kevin Chan
on 16 Jun 09I’m amazed at the feedback for this post. Although I try to live by the simplicity with operability rule, I must agree with most of everyone that posted their comments here.
Planning is a necessity. There is no way around this. You can simplify it, and not go overboard with it, but could you risk underplanning and see it blow up in your face?
A mindset change would have to be enforced to think of ways to approach an idea and simplify it up to maximize productivity, rather than to eliminate the whole planning process.
I must admit that this is a good post, to challenge everyone’s way of thinking and ways to improve it.
cecil
on 16 Jun 09“Plans are useless but planning is indispensable” (D. Eisenhower).
I don’t really agree with this post. Over planning is a pain in the neck and corporate over-management is evil, no question about that.
However, planning really is necessary. I ‘ve been in the IT industry for a while now and I’ve noticed that properly planned projects are more comfortable end easy to run. And, strangely enough, they are more likely to complete on time and budget.
John Gallagher
on 16 Jun 09I love what you guys do and really respect your work, but man, are you wrong on this one.
As others have said, overplanning is what you seem to be talking about. But no planning? At all?
Good enough planning (planning just enough to get you to your next iteration or sprint) is surely essential to the development process.
My guess is that you all do planning. You just don’t formalise it on paper. When you sit down to add a new feature in Highrise or Basecamp do you really just dive into the code and make some changes?
By thinking about what you do affecting the future, even as you’re doing your work, that’s planning. And I’d say that’s essential.
Iain Dooley
on 16 Jun 09Check this out:
http://www.joelonsoftware.com/items/2007/10/26.html
It’s my favourite planning article. Evidence based scheduling is one of the most realistic approaches and the great thing is that it puts the timeline in the hands of the developers. You can see very very clearly that the schedule will slip and you need to either start paying overtime or start cutting features :)
m.velu
on 16 Jun 09Dear friend,
I am really bothered about how people misunderstood the concept of planning. Planning is not a magic but it is a skill – a famous quote by my manager.
Yes, We should plan for everything before we start a project or strategy or journey whatever it is.
It is 90 % sure that Planning will orient us to path where we want to reach & It is only 10% that we will reach destination without planning. One of my project managers always belive in planning a project which is nothing but the splitting the project in to different sub activities so that we can monitor the each and every step now and then,
As you quote, Planning is not a time waste , but wasting time to produce a realisitc programme will give you atleast 85% success and remaining will take care itself. So, dont blame planning, Planning is not failure but we are failing to monitor and control the project or job by proper progress updation. Unless we monitor the project we cannot control the project now and then and i am sure that you will end up the good one.
m.velu
csonnet
on 17 Jun 09good point
Matt Kocaj
on 18 Jun 09fantastic advice
cizgi film izle
on 18 Jun 09good point…
This discussion is closed.