Anonymous Coward 14 Oct 05

well,
a) people imagine things at their best & most smooth; and far more importantly,
b) most purchasers want to believe int the best & most smooth solution proposed

Adrian L. 14 Oct 05

I think in general, people don’t pay attention to how much time they actually spend doing something. This inattention is then compounded by the very human tendency to overestimate ability and productivity.

When it comes to software development or web development, scope creep and unforseen problems, then compounded by the human tendency to overestimate ability and productivity, are the culprits.

Freelancers who track their time carefully are probably more likely to have an estimate that bears some resemblance to reality simply because normal revenue performance is so very close to the bankruptcy line.

Daniel Von Fange 14 Oct 05

As Adrain said, continual bad estimation comes from not tracking previous time spent on something.

chris 14 Oct 05

This blog is like white noise now.

You fire shatter shots and anything and 40 people kiss your ass and agree with you.

sad really, you just used to post interesting discussions.

Darrel 14 Oct 05

- Projects ALWAYS change en route (scope creep)
- committees
- it’s often an easier sell to lowball and add change orders later than to overestimate up front
- unforseen issues (namely previous owners when talking about home repair

Daniel Morrison 14 Oct 05

I was reading in some golfing book that the author very often sees amateurs use clubs that, even if they had their best shot ever, would barely go far enough.

For me (programmer), I tend to estimate thinking I’m a lot more efficient than I really am, or I think that the project will somehow be perfect and bug free. I’m really overestimating my own abilities, not underestimating the project.

David 14 Oct 05

Estimates often are more optimistic than the estimator thinks it will take. If they were more accurate, someone less honest would likely underbid them.

Alex Schroeder 14 Oct 05

In our company, developers guestimate without taking potential problems into account, and project managers multiply by two. Mechanically. That drastically improved the projects I’ve been involved in, at least.

Dan Boland 14 Oct 05

Because “how long will this take?” is the most loaded question on earth.

F2 14 Oct 05

I�m really overestimating my own abilities, not underestimating the project.

As a programmer I run into the same problem. For me it’s not just underestimating one project, it’s also accounting for one or two other concurrent projects as well as daily tasks. Now adays whatever length of time I think it’s going to take initially, I automatically double (or even triple) that just to be on the safe side. It may not please the client but I’d rather give myself room for bugs and get it right the first time.

Noah 14 Oct 05

If a company asks for time and money estimates from 10 contractors the cheapest fastest one is most often picked.

So if you just pull an estimate out of your ass that sounds good, you’ll get more work.

It’s wrong but people do it.

Benjy 14 Oct 05

I wonder how many people get estimates “wrong” as a sales technique? Say they can do it in 5 days to get the job, knowing it’ll take 8. Because in the end, what will really matter most is whether you’re happy with the end result. A year from now, what’s important isn’t the 3 days extra time it’s that you are still satisfied with the work.

If somebody really did a nice job, you’d probably still recommend them even if they took a couple extra days. On the other hand, if they stuck to the time estimate and did only a mediocre job you probably wouldn’t recommend them to others.

Tally 14 Oct 05

Because people use smoke and mirrors just to get ahead.
They’ll blow smoke in the form of an estimate and when it’s not done they deflect the heat by pointing a mirror towards someone else.

Estimates are a joke unless:
1. The person doing the work is given incentive to meet/exceed the deadline in favor of the consumer.

2. The person doing the work is given financial penalty for every day they don’t meet their “estimate”.

Chris S 14 Oct 05

Because, like Noah alluded to, it’s easier to get forgiveness than permission.

In other words, would you rather be realistic and NOT get hired, or be idealistic, be late, have everyone pissed at you, but still get paid (eventually…because maybe in their minds this allows the client to delay payment, too… :-)

Tomas Jogin 14 Oct 05

What is it about people that makes them ask for estimations, when they, again and again, learn that estimations are never accurate?

Andrew 14 Oct 05

“You fire shatter shots and anything and 40 people kiss your ass and agree with you.”

I don’t know, 37S’ main product is a project management application that ostensibly helps you plan out your time.

If one of the salient issues in project planning is that people almost always estimate wrong, isn’t it worth wondering why? Might that be something that Basecamp could take into account? This is one of those “users say one thing, but do another” problems that are so hard to figure out.

Jeff Grimmett 14 Oct 05

The reason that estimates are never accurate is that people don’t want accurate estimates.

I’ve never had an realistic estimate accepted as-is. It doesn’t matter that my experience says that x and y are factors, the boss or the customer doesn’t want to hear about that.

Instead, people prefer to live in happyland and then point fingers when things go wrong and drag things out to where the pessimist (realist) put things in the first place.

In sort, people are delusional. People are insane. Certifiable. That’s why.

Alistair Holt 14 Oct 05

I’m usually a bit over optimistic on estimates, not deliberately though! Life always throws up the unexpected, that is why it’s hard to sometimes estimate things.

Rick Turoczy 14 Oct 05

Sniff sniff. What’s that smell? Is that some essence of a Basecamp reporting feature in the oven? Allowing me to tag projects and report on average times from concept through completion by project type so that when I walk in to a client meeting I know how long that type of project tends to take? Can I end every sentence in this comment with a question mark?

Aaron B. Russell 14 Oct 05

I see this happen the most when contractors are trying to persuade clients to sign a contract with them. They want to impress the client so give often ridiculous completion estimates.

David Demaree 14 Oct 05

As people have said, a big part of the problem with estimates is that people always assume that things will go smoothly, that they’ll be working at their best and most efficient. But a big part of why my estimates turn out wrong is communication. There’s the obvious communication breakdowns with clients — feature creep, dithering about approvals, simply not responding to e-mails or phone calls in a timely way — and with one’s own team.

We recently had a huge problem at my company where one of our programmers would be given the specs for his part of the project, go off and work on it for several days and only later would it be discovered that he didn’t understand the specs and resented being called on his own screw-up. So we lost time not only to redoing that code, but to long, pointless arguments about whose fault it was that we were redoing the code.

Another big cause of poor estimates would be the estimates themselves: I think that having estimated how long it will take to do something people find themselves in a psychological comfort zone where the timeline has been “dealt with,” and mentally they move onto other battlegrounds such as design or functional details. This is an especially big problem for creatives working alone (and I include freelance programmers here) who are well acquainted with managing their own creative or production processes, but can’t always predict how the client relationship will affect the timeline.

Peter Cooper 14 Oct 05

You might not get the job if you say an eight month job is going to take eight months, whereas the guy who says it’ll take four months will get the job, and then the client will be so wrapped up with them that they’ll stick it out. Sad, but true.

Charles Richardson 14 Oct 05

“In other words, would you rather be realistic and NOT get hired…”

Yep. At this point, I really would, and have. The consequences of underestimating have become too painful, too often. About a year ago, I lost a lucrative contract because I told the client it would take a year to get their project up and running. They insisted they could have it done in six months. I wouldn’t agree to that.

At six months I called them and asked them how things were going. They said they were “80% done.” I didn’t have the heart to tell them that the last 20% generally takes as long as the first 80%. it’s been a year now, and they’re still not up. And I thank the gods someone else is being blamed for that…

I can also say that I have never lost net contracting hours by turning down bad projects with unrealistic estimates.

Steve 14 Oct 05

I agree with the first post that people envision the ideal solution. When they don’t get their “ideal” vision, scope creep becomes a problem. This is why I think “getting real” isn’t quite an ideal process when you have a fixed budget, because it fosters an iterative process where scope creep is almost encouraged.

If you don’t have requirements documents, and have the customer sign them in blood, scope creep will always happen.

As someone who does the estimates on my own work more often than not, I always like to multiply by 1.4 for the unexpected costs (I would go x2, but I just can’t seem to get away with it). 40% extra usually gives me enough time to deal with the project management issues that I really shouldn’t have to deal with, and the minor code modifications that aren’t really scope creep, but aren’t really apparent until you have dove into the project.

X = Cost to “write the code”
+ %50 for Expected “Extras” (QA, Performance, Commenting, Training, Launching, etc.)
——————————————-
X * 1.50 = Expected Costs

+ 40% for Unexpected Issues (Necessary Changes, PM)
———————————————
Expected Costs * 1.40 = Amount Bid

Then I get them to sign it in blood, and away we go. Everybody has their own estimating style, and the bottom line is that if it works for you, and your accurate with it, it’s a good style to use.

As far as scope creep, I’ve taken a page from JF and gone with the “thats a great idea for version 2”. Works pretty much every time, and almost every project I have a version 2 ready to be bid out shortly thereafter.

-s

Michael 14 Oct 05

It’s interesting, people are also very bad at estimating probabilities. For instance hardly anybody correctly estimates the probability of two people, in a room of 25, having the same birthday. Most people say guess 2 or 3 percent. However it is 56.9%, and it jumps to over 70% for a room of 30 people.

Tomas 14 Oct 05

All I can say is Break it down. Break it down. Keep breaking it down, until the entity can’t be any smaller. Then estimate the individual tasks, add them all up, Double it, and add 2 weeks.

Works everytime.

Andy 14 Oct 05

This blog is like white noise now.

You fire shatter shots and anything and 40 people kiss your ass and agree with you.

Seriously, can there be one blog entry on here without someone saying “this blog is crap”?

Well genius - how about this? Don’t read it! Noone has a gun to your head.

Drew McLellan 14 Oct 05

I’ve found it helpful to think about tasks in term of how much time should be allowed for the task, rather than how long I think it will take. Typically the former should be significantly more than the latter.

For example, if my estimate for a feature is that it will take 20 hours to develop, and I understand that my estimate is likely wrong, plus the client might just change things along the way, I might allow 40 hours for the feature to be developed.

If the job comes in at 30 hours, the client gets a smaller bill and everyone’s happy. If there are problems, there’s a buffer that has already been allowed for.

Of course, this only works if you’re honest enough to bill less than the estimate at the end if not all the time was used.

Gregory Haase 14 Oct 05

Actually Tomas hit the nail on the head. You have to build what project management training refers to as a “work breakdown structure”. You take ever objective and break it down into deliverables. Then you take ever deliverable and break it down into multiple tasks. You should really break it down into the smallest element possible. At that point, you estimate how long it will take to complete each task and then roll them up to the top.

It sounds like a lot of up front work, and it is, but it is really the only way to get a true estimate. If a client asked for e-commerce site and you said to yourself “I’m going to build pages for each product, a search, and a shopping cart - that should cost about $x” (I realize I’m oversimplifying here - but that’s really what happens). However if you break it down to the level where you have several tasks: 1.) create a table to hold product information 2.) create a query to pull product information to the page. Not only are these smaller items much easier to estimate, but you’re probably already thinking - “Oh yeah, I need an admin page where new products can be added”

Josh Williams 14 Oct 05

Dan Boland: Because �how long will this take?� is the most loaded question on earth.

Exactly.

ZX 14 Oct 05

call steve jobs - i think he found the solution…http://live.watchmactv.com/?p=69

Brad 14 Oct 05

I prefer to offer my clients fixed price contracts rather than extimated timelines alone. They tell me the work they want (in detail), I give them a fixed price and a time range estimate. It seems to work. If they change what they want, I (can) change the price. I have incentive to finish a quickly as I can given that my rate is fixed, they have incentive to know exactly what they want.

Kyle 14 Oct 05

Gregory Haase hit a big part of it in his comments. I used to work with a guy who could hit an estimate pretty much down to the day we would deliver the project. It’s called professionalism, and it was a big inspiration to the way I work now.

The other big problem, as pointed out by someone else, is that you can give the client a realistic timeline, but if they’re not a good potential client, they’ll get an offer from someone else to do it faster(read: cheaper) and usually go with it.

Good estimates are made from a combination of a smart team and a smart client. It’s about being honest about expectations. Unfortunately, if you’re small, unproven team, it can be hard to get the opportunity to do it right, because you’re always in need of a client.

Chris 14 Oct 05

People are optimists.

bill 14 Oct 05

I can only speak to the design and development side of things but I think in general people are knowingly under-estimating a job in fear of not getting the job all together, or if it’s a case where they already have a job (internal position), they don’t want to defend how long they think it will truly take.

A couple of reasons for this are that people are scared to be honest and tell someone that things will take longer than it should because of beurocratic bullshit. Also though I think that there is a lot of instances where people feel inadequate and think that maybe they are just slower than others and don’t want to bring that to light.

I’d much rather walk away from a job because the client thinks my estimate is too high than lose my ass on a project because I tailored reality to fit someone’s skewed expectations. All you do is breed failure in that scenario.

On a side note when did the word “estimate” lose any and all meaning? I’ve never once given an estimate and had that estimate not turn into a hard number that cannot change once the project has started without an uprising. Why can’t people just pay for your time? I’d gladly lower my rate and take all jobs as time and materials and only get paid for the actual work I do. As it is I have to inflate my rate and factor in nonsense into every project that is estimated and at the end of the day only one party is getting a fair shake. While both parties may be happy I think the client ends up paying for time that you didn’t use or you end up giving away time in each and every project.

Or maybe I’m just doing it all wrong…

Michael Spina 14 Oct 05

My favorite method for getting cost estimates is to look at a similar project I did in the past and figure out how much I would charge to allow if I had the opportunity to do it all over again. My estimates still aren’t 100%, but getting closer each time. I’ve had more luck with that method than looking at the project at hand and breaking it down, which I still do if it’s a first project of that type.

Christopher Hawkins 14 Oct 05

“On a side note when did the word �estimate� lose any and all meaning? I�ve never once given an estimate and had that estimate not turn into a hard number that cannot change once the project has started without an uprising. “

Welcome to the business! ;)

Clients who treat estimates as bids are very difficult to deal with.

I have recently taken to revising estimates weekly, even if only by a half hour up or down each time, to help clients break out of the thinking that an estimate is a bid.

It’s a very new practice for me, and I’ve not been doing it long enough to gather any meaningful data yet. I’ll write it up on my blog once I know more. ;)

Dave Simon 14 Oct 05

To those who keep complaining about SvN - either stop reading or contribute more to stimulate the discussion!

I think part of the reason that people are bad at estimating is because clients get in the way of everything happening in time.

But a big reason is that some people fear they won’t get the job if they bid 40 hours, so they bid 20 and deal with the consequences later. Which is just bad customer service.

pwb 14 Oct 05

Because everyone wants to be optimistic but people don’t really care. If you really care, just take the estimate, chop the scope in half and double the estimate.

Does anyone ever come in *under* estimate?

Bhaskar Vijay Singh 14 Oct 05

Nope. I think people are excellent at estimating (or guessing) , its only that every person down below is a perfectionist.
A person can easily touch down his mental deadline but he strives to go a bit further into perfection thus falling prey to the perfection trap.
Truly , being a perfectionist is the best excuse.

Paul 14 Oct 05

There are plenty of people who can create and deliver accurate estimates, but they never get the job.

cellis 14 Oct 05

people are such bad estimators because predicting the future is just an impossible thing to do. this fact doesnt make estimating useless. just as a jet is never on the correct course until it lines up the runway, software estimates should be continually revised until the job is complete.

Darren Oakey 14 Oct 05

I speak only for what I know, software development - but I think most developers are not optimists - most developers realise that something will take longer than it takes “to code” - and understand things go wrong…

In my experience the problem is first with the situation - when someone comes in asking “how long is this going to take” - they either a) have already basically got the money approved, and have a very definite preconceived answer or b) haven’t yet got the project approved in which case no one has done a serious amount of work scoping it - ie no one really has a clue what is being asked for.

Most developers go through the following exercise in their minds: “hmm lets see - gotta do A which will take 2 days, plus B 3, plus C 1 etc etc… that all adds up to 60… multiply by 3 for all the testing, doc and stuff, we get 180… ok.. sounds good”…. then they look at their manager, who has continued talking and said, “we are going to try to get it delivered my the end of the month”… and the programmer goes “yeah right” and instantly shelves their 180 because they know the manager will just have a heart attack right then, and just picks a random number - “no I think it will take at least 2 months” - and then the manager pales, and says… too long - can I add more resources? And the developer doesn’t think, they just say I suppose and so the manager adds another resource and then says 1 month… fine…

Where in reality, the original 180 days was probably pretty correct, and another resource would bring that down to… say 140 days maybe, when you account for all the extra overheads.

And eventually the developers just basically use the following paradigm for estimation:

a) pick the largest number that won’t get them fired instantly (but is still way less than the actual time)
b) get beaten down to a smaller number
c) completely ignore the estimates and schedules, and just develop until it’s finished and deliver early, late, or on time as appropriate

At least that’s my experience of how development and estimation works.

Ron 14 Oct 05

Two equally competent contractors. One adds extra time to his estimate because of the above mentioned things in an attempt to arrive at a REAL estimate of time. The other contractor does not. You go with the second contractor because his estimates claims he can do it faster.

Tha is why time estimates are almost always low ball.

Krista 14 Oct 05

There are a lot of accurate estimates that happen, but what we remember later are the ones that went over. Over budget, over deadline, etc.

Paul Epps 14 Oct 05

People are not bad estimators.

If people were really trying to provide good honest estimates — and were just bad estimators — you’d expect to see about the same number of overestimates as underestimates.

The fact that almost every estimate is too low shows that estimates are prepared mainly for political effect and have little or no relationship to reality.

Ali Pasha 14 Oct 05

Bill Gates has written in his first book and over and over repeats that “People usually overestimate what can be achieved in one year but underestimate what can be achieved in ten years”
In his Videocast Interview with Scoble on Channel 9 even he had to confess that he is really bad with estimations.

Yet priod based estimations do not matter in terms of accuracy! What matters is the ability to project in the future the factors that will grow and the factors that will remain relatively constant or even depreciate. If you project long enough you will see what makes a difference and needs our attention now and what we can skip.
That simplifies your decision making a lot.

Adrian 14 Oct 05

Because everything sounds quick and easy until you try it.

I thought adding a simple events box to the home page of my web app would be effectively a single task taking not much more than half an hour.

So I tracked it. It needed 27 discrete tasks and took 4.5 hours.

Now multiply that kind of thing across a whole project and you can see how weeks turn into months turn into years.

Mike Sigers 14 Oct 05

People aren’t that bad, they’re just liars.

Most contractors think they won’t get the job if they tell the truth.

They try to guess what their competition said and beat it.

D 14 Oct 05

“White noise,” B-S! Certainly a relevant issue to ponder, since most of us have been either the estimator and/or the estimatee!

Garrick Van Buren 14 Oct 05

Maybe we’re bad at estimating because deep down we know they’re wasted effort.

Jess 15 Oct 05

Recently I had my apartment painted, and when I asked the painter to estimate the time he said “2 or 3 days” and when I asked about the price he said “$600-$800”. “Good”, I said, “Then I’ll take the one to 2 days and $600”!

Anon 15 Oct 05

There’s plenty of real life to draw on. Bush’s people said invading Iraq would be finished in six months and cost $50 billion. General Shashkavili said it would take years and cost $200 billion minimum. They fired him.

Guess who was right? Shashkavili appears to have lowballed the time and amount, actually…

Why do you think this happened? Was it because Bush’s people were just bad estimators? Give me a break.

Estimates are political things. Estimates for how long X takes are usually done by people *who want X to occur*. Thus they’re usually low. When they’re done by people who *don’t want X to occur*, they’re usually high. When they’re done by people who *have no stake in whether X occurs or not*, they’re usually pretty accurate.

Jens Meiert 15 Oct 05

“Anon”, good points. - Sometimes it’s just the question if you have the balls to communicate honest estimations.

Carey 15 Oct 05

Estimates are required because the person inquiring about the work doesn’t know how long it should take or how much effort is needed to finish a project. I think the estimator usually knows what is reasonable and accurate, but severely underestimates because they don’t want to deliver the perceived “bad news”. Instead, the estimator tries to provide an estimate that they think the other person wants to hear, essentially trying to peg what a less informed, reasonable person’s estimate would be; in hindsight, often the “best-case scenario”.

Swati 16 Oct 05

Because life is unpredictable.

Greene 16 Oct 05

I can’t remember where I picked this up from but it has proved useful in negotiations with clients.

I explain that a project will be governed by timeframe, cost and scope and the client can choose two of these as having fixed values.

If they want it ready exactly on time, exactly as asked for, they need to be prepared for an upwardly flexible cost. If they want a fixed timeframe and cost, the scope will need to be flexible.

Most clients can see the wisdom in this and will general go for a fixed cost, which means that I have a good reason to defer extra features for a future release.

Gilles Satg� 16 Oct 05

Because of Hofstadter’s Law: “It always takes longer than you expect, even when you take into account Hofstadter’s Law”.

Thibault 17 Oct 05

I think it’s important to have a right estimate.
A real professionnal has to know how long it will take to do a job.
It’s a basic skill and it’s part of the job.

For political or commercial reasons you may play with your figure but that’s another issue.

As an example we’re working with a company and they are always right on their estimates.
Our decision makers are still thinking they are too slow and too expensive. But they always deliver on time the critical work we ask for.
What is the “the critical work”?
Sometimes (if not all the time) WE don’t do our part of the job and we do not provide to them the right informations on time (the right business rules by example) -or- we ask them too complex things.
Hence we’re loosing time here and here.
Rightly this company come back to us and says: “well if you -really- want this feature we can do it, but we will have to delay the shipping date becasue at the end it’s too complex to do it within the budget and the time frame you gave us”.
And then our decision makers always prefer to keep the shipping date within the budget. Then the most important thing at the begining become not that important at the end! Everything else was critical and properly done.

After all we’re doing a creative work. We invent. We never really do twice the same thing. So pitfalls happens, isn’t it? And decision makers do not understand they order a creative work.

And as one said : ” A real artist ships!”

eb* 17 Oct 05

It’s human anture to make quick estimates that ‘make do’. See Steven Pinkers book for more.

http://www.amazon.co.uk/exec/obidos/ASIN/0140244913/qid=1129565416/sr=8-2/ref=sr_8_xs_ap_i2_xgl/202-3755940-9992604

Lorenzo 17 Oct 05

There’s something called the 3x2x rule, and people who have done some home improvement themselves will recognize the pattern:
It takes 3 times as long as you had planned (3x)
It will cost twice as much as you had planned (2x)

Just my (estimated) 2 cents

Don Schenck 18 Oct 05

What Ron said.

Also; take your estimate, double the number and pick the next higher time frame.

One week becomes two months … three months become six quarters …

(not 100 percent true, but fun an amazingly accurate)

Ruben 18 Oct 05

I think that people should refrain from estimating and instead list the detterants that will slow them down from completing something then work from there.

Jordan 19 Oct 05

I think its because most people have a hard time saying NO. Saying NO has been something I have been working hard to do more and more lately.

I started padding my estimates… and then just started going past that estimate itself. Mainly because I felt that gave me the option to say YES to clients, employees, co-workers or whomever.

Its a much better option to say no and reach your goals than to say yes and fall short of your goals.

Paul 20 Oct 05

I think the problem is all of the things mentioned above. It’s envisioning everything going perfectly, low-balling to win a contract, not understanding the requirements well enough, scope creep, etc.

Another part of it is not conducting a post-project review, and trying to understand what the differences were and how they might have happened. Without this mechanism, you can’t possibly recognize the same situation before it occurs on other projects.

Another pet peeve of mine that has some effect on estimation and estimation technique: Why does the IT world in general refuse to look outside their domain for common problems and solutions? Why are we so convinced that the concepts of what we do (building something) are that different from other fields like building a home or car, etc?

We aren’t the only profession struggling with these same basic issues. If a client hires a home builder to build a home, doesn’t that home builder have to deal with these exact same issues? He has to gather the requirements from the users, decide what tasks need to be accomplished, what resources are needed, and provide an estimate and cost.

How do professions like these handle these same issues? One thing that always struck me that has the potential to be useful in SW is the concept of a blueprint or plan as a starting point.

“Here’s some floor plans for a 3-bedroom, 2 bath, 2 car garage home. Look them over, pick one that is close to what you want, and then let’s dicuss what you want different.”

This is one approach I think if we moved towards in the SW world, would help our estimation greatly. Really, are our customers asking for something so vastly different or are they asking for a basic model customized in certain areas to meet their specific needs?

We have started to move that way, but it seems largely by accident and without really understanding why.

We should know better than anyone that re-inventing the wheel doesn’t solve much, so why are we so quick to try and do just that when we encounter issues that are very similar to what other professions have encountered and solved (or at least improved)?

oliver 25 Oct 05

i’m a project manager in software development. my estimates tend to be too high. i don’t take a pessimistic approach but i realize that some things do take time. the trick is not to become known as someone who always overestimates.

it’s not that hard to come up with good estimates. you just have to be honest with yourself and others.