“When is it going to be done?” is a reasonable question and we as software developers should try to come up with the best answer we can based on our experience and analysis. What we should not do, however, is treat our answer as solemn oath.
When you treat estimates as promises instead of guesses, you bind your worth as a worker to it. If you do not meet your own deadline, you are a failure. And since nobody likes to be a failure, they’ll indulge in risky behavior to avoid it, like burning the midnight oil and checking in bad code with scanty or no tests.
Rushing to meet your estimate promise once or twice might be bearable, but it’s ultimately unsustainable. Software development is inherently unpredictable. There are just too many moving parts and ice tips that turn out to be icebergs.
If you treat the estimate as a “best guess based on the limited information available to me before I start the work”, though, you’ll change the frame and break the cycle of deadline anguish. Now the task becomes collaborative and you can share new discoveries with the stakeholders.
Found out that doing the feature as originally designed requires changing some fundamental infrastructure that’ll add another two days to your one day estimate? Maybe it’s not worth it any more. Ask the stakeholder if he’s still interested in the feature when it costs three days instead of one. Or if there’s a way to simplify it such that the infrastructure change is not necessary.
That’s the true value of estimates. That it sets up conversational constraints that can be used as boundaries for trading concessions. Not that they’re nails for your own self-erected cross.
Obie Fernandez
on 02 Feb 10Great reminder, David. And one of the best tools for collaborating with clients on reasonable and accurate milestone dates is Pivotal Tracker.
Brandon Ferguson
on 02 Feb 10Couldn’t agree more with Obie on that. It along with scrum changed the conversation at my company and got us to release and not chasing our tails.
Aaron
on 02 Feb 10This is why I have trouble, and don’t like saying yes to when I get asked I van have a task done by.
David Alison
on 02 Feb 10I’ve found that stakeholders and project sponsors generally want highly specific deadlines because there’s a lot of other components to a software release. Between marketing, sales, support and general infrastructure there are lots of moving parts that are impacted people outside the development process.
Sometimes a stakeholder or senior executive wants a deadline because they really don’t understand the nature of software development and this is their way of exercising a level of control of what is otherwise FM to them. This doesn’t mean it’s the best way to develop software, just a reflection of what many developers are faced with.
Having worked for a large commercial software builder back in the early 90s we often had those incredible build ups to a launch where the development team was working around the clock to prepare a release for public consumption. When you were preparing to press hundreds of thousands of floppies and get them pushed into your distribution network in time for a large staged event, that was just how it worked.
Old habits die hard. I’m hoping the giant push before a release finally does go away.
Max Horkheimer
on 02 Feb 10‘shanty’ should read ‘scant’
D. Lambert
on 02 Feb 10Great article. It takes a strong business sponsor and a fair bit of trust to really make this work, though. If you can achieve enough education and transparency so that everyone understands what’s going on as estimates mature, you’re doing pretty well.
Andy
on 02 Feb 10Hey dude – its a guess! Chill !!
HAHAHAHAHAHA – right. Get real.
Marketing, sales, and customers need to coordinate their plans with development. If you want to work in blessed isolation, go into academia.
Vaibhav
on 02 Feb 10Makes a lot of sense in the current context. The world is changing and so is the art (if I may say) of software development. Instead of wasting time in correcting silly mistakes created by rushing in, it makes a lot of sense to spend some extra hours on ensuring quality and doing something new.
If a person is working on something with passion, introducing a deadline might lower his/her morale at times – just as you said – no-one likes failures. Specially in passion driven software development, a person might drift to read something interesting, try something new – this may not be completely wrong as it generates new ideas.
@Obie – really, everyone should try pivotaltracker once.
Paul Marden
on 02 Feb 10Couldn’t agree more. There are many Project Managers out there that whip their developers for failing to hit estimates. Thats unfair and it’s not what the plan was for. Life hits unplanned points and you just have to replan for it, and accept that the gantt chart (or whatever you use) will just have to be updated again.
However, there are many developers out there that only mention a delay after the deadline has passed. Nothing infuriates me more as a PM than when people ask me stuff after the deadline when I’ve got no choice. It completely disempowers me.
The contract should be that the PM won’t shoot the Dev for things going wrong or not to plan – if the Dev speaks up early and often to seek guidance and collective decision making.
Tommy
on 02 Feb 10Sounds good!
Any suggestions for this one?
Brielfy, I’m building web interfaces for 2 x databases of research paper abstracts for a large giv org.
I “estimated” db design and some simple CRUD opertaions on the backend.
I was given a flat xsl spreadsheet and had to spend about two weeks writing code to parse all of the data into relational tables which turnded out to be a harry ordeal requiring manay iterations.
I’m behind by about two weeks on deliverables but feel I failed to communicate the spreadhseet dilimma and have to to basically suck it up and “burn the midnight oil.”
Should I try and get some wiggle room at this point or gut it out?
DHH
on 02 Feb 10Andy, you can absolutely coordinate deadlines with multiple departments. Just as long as you keep the scope flexible. As in “we’ll ship SOMETHING by X”. If you try to fix scope, time, and quality all at the same time, one of them is going to suffer dearly. Also known as “if it doesn’t have to work, I can be done tomorrow!”. And you build an environment that people will dislike which will lead to attrition and ulcers.
Tommy, you should have raised the concern as soon as you saw that the tip was an iceberg. Now it’s a lot harder to do it. It’ll come off easily as just “I want more money”. That’s why time & materials is typically a better model for developers that can’t fix scope up front.
Tommy
on 02 Feb 10But here’s the rub….
I don’t want more money, I just want a reasonably satisfied client that I can build into a reliable income stream in this economy.
Mattias
on 02 Feb 10Software project time estimation is a complex art, and entire books have been written on the topic (such as “Software Estimation: Demystifying the Black Art”).
I think the main reason why software projects are hard to estimate is that you never solve the same problem twice. Unlike building a house or designing a poster, you can and should package the solution in a library and reuse it when necessary in future projects. As a result, all of development time spent is working on new problems, exploring, experimenting and learning – which are much harder to estimate than a honed skill at a task.
Tommy
on 02 Feb 10Well put Mattias.
I agree, but I have hit road blocks trying to repurpose source code.
I’ve re-used “techniques” or “architecture” but sometimes trying to reuse code, especially if I’ve learned better ways of doing things can be a hinderence.
That’s what I run into, sifting through old source and realizing I would have saved time “baking” it fresh.
Nastassja
on 02 Feb 10That’s great when you are out of somebody’s hands and can decide everything yourself. But there are a lot of situations, when other people are affirming the deadline, and you just have to obey. Or the project is so huge and it’s impossible to estimate it all. Anyway, developers should learn to know their own limitations.
DHH
on 02 Feb 10Nastassja, if your project is so huge that it’s impossible to estimate, only fools would attempt to develop it. There’s 30 years of history on what the odds are of delivering mega projects: Terrible. Even top tier software organizations get it wrong all the time.
Second, if you’re developing software in an adversarial environment where it’s a fight between stakeholders and developers instead of a collaboration, you’re in for pain and misery anyway. I wish you the best of luck!
Tony
on 02 Feb 10This is why a lot of devs take the Scotty from star trek approach, and grossly overestimate.
Quality of estimate is directly proportional to the self-awareness of the developer and the quality of the given scope.
Estimate adjustments are inevitable – the best you can do is try to manage expectations, and alert involved parties the moment a new piece of information changes your estimate.
Steve
on 02 Feb 10My favourite quote: “It’s an estimate – not an actual”
Giles Bowkett
on 02 Feb 10It’s not “shanty”, it’s “scant.” “Scant” means “hardly any”, “shanty” means either “tiny/crappy house” (as in “shanty town”) or “song” (as in “sea shanty”).
Avonelle Lovhaug
on 02 Feb 10Here’s the quote I use (somewhat similar to Steve’s):
This is an estimate. Estimates are best guesses If I knew exactly how much time to implement this, I’d call it something else.
Nastassja
on 02 Feb 10David, everyone can be be sadly mistaken. For example when there are a lot of defects on design phase and developer can suffer because of it. How would he get to know all nuances at once?
Thank you for your kind wishes, same to you!
Nic
on 03 Feb 10My favourite quote on forcasting software completion:
From FogCreek EBS 2.0
Also see the intro to this concept on Joel’s blog
Personally I think their method of evidence based scheduling (even if you don’t use their software) is a very sensible and practical way of forcasting a realistic completion date, whether it is for marketing or planning or just to keep track of progress.
Disclaimer 1: I have not had the opportunity to use this method personally. Disclaimer 2: I am not affiliated or associated with FogCreek in any way.
pht
on 03 Feb 10Good advice assuming you are in a structure where you have any go on the up-front planning. As in “the customer wants to ship on xxxember the yyth” or “we must demonstrate our stuff at the Grand Internationnal Submit of Whatever”
Sure, what you’ll ship is most likely to be crap, and the grand opening will be delayed since reality always end up taking this toll.
But sadly Andy has a point, most of the time it’s not like there really is a choice but commiting oneself …
So true
on 03 Feb 10There are quite a lot of projects in France where there are contracts where the estimates are obligations.
You just have to produce everything in 200 days for example. If it’s not done the company doing the software for the customer loses money. But if the estimates are too high, they don’t get the customer.
And such contracts are awful, a lot of pressure, deadline never respected… and all that leads to this attitude “better make all the features even if it’s all bugged than not meeting the contract’s X features requirements”.
zakiwarfel
on 03 Feb 10Nastassja, the trick with large complex systems is to break them down into smaller manageable parts. Taking this approach will allow you to estimate, ship often and early and ultimately show value faster and reduce risk.
Rob Smith
on 03 Feb 10Commercially, deadlines are required. There is a simple way to ensure you hit them in my belief:
- Make sure the project being undertaken isn’t too large. If it is, split it into phases of delivery - Overestimate the amount of time it will take - Spend more time and resource in the planning and estimating stage – it will save you 10x that in terms of deadline hitting - Nail down the scope. If anything changes, add more time - Be strong on your estimates. If the client tries to squeeze it down by a few weeks, hold your ground.
Then, in our experience, you can hit deadlines without rushing.
fix_the_typo
on 03 Feb 10Is DHH going to fix the typo that several people have pointed out? It’s glaring.
Nastassja
on 03 Feb 10zakiwarfel, I totally agree with you. Usually there are a lot of people working with big projects and they can divide them into the parts, they are going to work with. In this case the other side of the coin – is human factor, its hard to take the responsibility for everyone and to be sure that they will cope with a task on the dot.
Don Schenck
on 03 Feb 10So it’s late. So what?
Microsoft has been late so many times, it’s become standard practice. Yet, they still rule the world.
No one cares if it’s late if it’s great. Seriously … they get over it.
My philosophy has always been to add “stuff” in layers, and “it’s done when the client says it’s done”.
Wayne
on 03 Feb 10Neither promise, nor guess – an estimate is an estimate, which implies confidence boundaries. “10% likely I’m done tomorrow, 50% likely I’m done next Tuesday, 90% likely I’m done next Friday.” Calling it a “guess” denigrates your own thought process and encourages you to be childish and ignore it … leading to feature creep, being too late with something for opportunities, etc.
Johan Strandell
on 03 Feb 10@Tony: the danger of always overestimating is that sooner or later other people will pick up on it and adjust their estimates/deadlines accordingly, so if a developer says something will take 2 months, the other part will try to aim for 2 weeks instead.
Which works as long as you’re always overestimating the same amount, but might lead to problems when a task actually takes the 2 months you said.
My favourite method is actually doubling the estimate, and going up a unit – 1 minute is 2 hours, 4 hours is 8 days, etc. It’s often depressingly accurate.
indie_preneur
on 03 Feb 10Good points David. I think that it’s our duty, as designers and developers, to educate our clients about the fiddly nature of web design and development (and software development as well). The more you can communicate at the beginning of a project the better that the client will understand any potential issues that come up. Also, if you’re open and willing to share your progress actively with the client, I’ve found it gets them on your side more often and more actively involved in the project, usually leading to the being more understanding about potential unforeseen issues.
Verneri
on 04 Feb 10The problem with deadlines is that they are detached from the reality. You need to acknowledge that they are goals instead of rules. If you don’t you’re merely creating an illusion of control where it doesn’t really exist.
So instead of trying to control things you can’t, you should concentrate on adapting to the reality in order to lead your product to meet the goal. If it becomes evident that the goal cannot be reached it has to be changed or the failure communicated immediately.
Illusion of control is one of the major sins of traditional software development methods.
Alora
on 04 Feb 10Absolutely, it’s a guess. And everyone needs to be reasonable about the fact that things happen, and the first thing a good project manager does is to attempt to de-risk the schedule as much as possible: build in buffer time, identify potential obstacles that can be removed before they become an issue, etc.
But the reality is that business has deadlines. Some of them are certainly self-imposed, but some of them aren’t. I’ve spent a lot of time working in regulated industries, where there are simply deadlines that must be met. And the problem that I’ve experienced isn’t that there isn’t enough time to do the work, it’s the developers are rarely pragmatic about how they approach estimating.
One of the things I’ve always loved most about working with developers is that they tend to be very optimistic by nature. But the more optimistic the developer, the less reliable the estimates, and the more disastrous the project outcome.
Of course software development is complex and fraught with opportunity for obstacles and delays. But business is business. And, in the end, if the world around your business only gives you limited opportunity to get something shipped in order to sink or swim, then that’s all the time you get. There may be plenty of good reasons for why the estimate the business was counting on couldn’t be met, but that doesn’t change the fact that missing the target can still have business-crippling potential.
If it can’t be done, say so. If you see potential obstacles on the horizon, speak up immediately. Bad news doesn’t get better with age, and the longer you wait, the harder it is to deal with the downstream implications of a late delivery.
The biggest problem that the business side of the house always has with development is feeling blind-sided by being informed of delays at the last minute. And yet, in most cases, the developer saw it coming before that point. In more than a decade of doing this, both internally and in professional services environments, where I’ve seen the REAL problem is in not the late delivery itself—all reasonable people understand that things happen.
The real source of frustration is in poorly managed expectations. No one wants to be the bad guy by pointing out that the date can’t be made, so they wait. And then everyone ends up looking bad, which just further deteriorates the entire situation.
pmhut
on 04 Feb 10This post seems to assume that the developer is accountable for the schedule (checking in bad code, etc..). I don’t think this is true.
It’s the Project Manager that makes the estimates, which he or she is accountable for. It is up to him/her to make a reasonable buffer to make sure that the tasks are finished on time.
Ben Rowe
on 05 Feb 10I think when your project relies on investor funding, it’s commercially reasonable to be able to “Hit the date”, give or take one or two weeks.
The key is to manage scope. Aim for the ‘Stage 1’ launch that satifies investor needs, with an application that meets the client needs.
It means you have to cut back on certain features to achieve the deadlines, and focus on “what’s essential, and nothing else”.
Which is kind of what you 37Signals kids are all about, isn’t it?
pht
on 05 Feb 10Another thing : I perfectly understand how that can work when you’re a company designing your own product form public to use (you’re the one who decices what goes in, what does not, and when you open it to outsiders.)
Now as pointed by “So true”, lots of work involves external, hardly changeable deadlines (contracts with both a deadline and a fixed scope, where I guess you’re supposed to compete on how many monkeys you can use to work, and your overall awesomeness).
Also, from what I understand, even you guys at 37signals used to do ‘contracting’, so you must have met some customers with deadlines that simply couldn’t be played with. How did the “let’s reduce scope” thing (also known as “no, you’re not going to be able to get the list of xxxx in yyyy state in this screen, because we’re not going to have enough time to design those screen, and sure your users just need that”) worked for you ?
Anonymous Coward
on 05 Feb 10@Alora “One of the things I’ve always loved most about working with developers is that they tend to be very optimistic by nature. But the more optimistic the developer, the less reliable the estimates, and the more disastrous the project outcome.”
On the contrary, I’ve found a lot of PM to be too optimistic. Rather listening to the optimistic developer than the pessimistic one.
And when there’s not optimistic developer, they just say than the pessimistic is wrong and that it will take less.
This discussion is closed.