We had a good debate on Thursday at XP Manchester about #noestimates. I volunteered to defend the need for estimates, so below is a transcript of my opening rant in defence of estimates…
Have we earned the right to say #noestimates?
Imagine I wanted a garage built and I found a builder. I show them my driveway, the side of my house and the size of my car. I ask them how much it would cost me and they say “I don’t do estimates but fund me and my crew for 4 weeks and I’ll deliver you something useful”.
Would I go for that?
Do I look stupid?
But we all know that is a silly comparison. As software developers we know that design goes down to the code – it’s not just like laying bricks one on top of another. We also know that in many cases some aspect of the context under which we are developing software is unique – team member skills, number and roles of people, company culture, familiarity of team with domain, familiarity of team with the technology needed to solve the problem etc.
Whereas a garage is a garage is a garage. But even with something as simple as a garage, you know there will be some plans put together and building regulations will be enforced. We have things like that to stop buildings falling down and killing people. Would you trust a builder if they didn’t conform to building regulations and started without some sort of blueprint or design?
Coming back to software development, where are our building regulations? Where are our blueprints?
Oh, we don’t do those – we’re agile!
And there aren’t any building regulations really. There’s only us, marking our own homework.
“Trust us”, we say. “We will build you something useful.”
Have we earned the right to say #noestimates?
Why don’t we want to give estimates? Are we really saying that we don’t want to estimate or do we just need more courage?
Or is it that our estimates in the past have been crap?
If so, why have we given people crap estimates?
Was it…
- mortgage-driven development: “if we don’t give the business the estimates they want then they will offshore the work”. If our business are that stupid or Machiavellian then let’s do ourselves a favour and get a job elsewhere as they will screw us some other way
- fear of looking incompetent – do we have the courage to “display your ignorance” by playing Pawel Brodzinski’s No Fucking Clue or Too Fucking Big cards in planning poker?
- not being open to alternative approaches to estimating: like Dan North’s Blink Estimation for example
- that we just went through the motions rather than helping to seriously manage project risk, in order to, as JBrains says “Explore higher-risk aspects of the system sooner to protect against nasty surprises at inopportune times”
Risk. Now risk is an interesting one.
As developers, how much do we really understand risk? Have we explored things like the Cynefin framework for classifying pieces of work for example?
Would we be ready to work this way – risk first, probing, delivering value? What if the early decision is “no go”. What happens to us then? Do we need to find another job? Are we prepared for the consequences of honesty and truth?
Have we earned the right to say #noestimates?
“Trust us”
Have we brought this on ourselves with a cycle of: We’ll try (which usually means) we’ll fail (which then leads to) no trust
Are we really doing the best we can do? When I look round many agile development shops, the words that spring to mind are “cargo cult”. Lots of boards, lots of cards, maybe even some big monitor screens. But do we really live the values? Do we really walk the talk?
Someone, somewhere has cut us some slack to fund the Sharpies and the pizzas. But outside the technical bubble it can be a cold, hard world.
Ultimately, why should someone fund our team if we can’t give them a reasonable idea of what they might expect at the end of it. Other parts of the business are going to be asked to forecast sales, plan marketing campaigns, forecast profits, and so on. Why should we be exempt?
But… “I’m a poor liddle developer and these nasty business people want me to provide estimates”
Hello?!? These nasty business people are currently paying our mortgages. If the company doesn’t make a profit we will no longer have a job – unless, of course, we jump ship to another company as soon as the going gets tough.
Can we change? Will we speak truth to power? Will we take on responsibility outside of our technical bubble? Will we stop spending our careers in a flurry of buzzword-laden job hopping?
For the last time I ask: have we earned the right to say #noestimates?
XP/agile time boxing is essentially saying “trust us to use our professional judgement and skills to deliver something useful in that time box”
“Trust us”
How can we prove ourselves trustworthy? How about we…
- stop including untried and untested technologies in a project/product just because they will “look good on my CV” or “they’re soooo cool”
- stop designing cathedrals of code when what is really needed in this case is a rough-hewn stone chapel
- stop focusing on just the coding without considering the testing, release management, impact on the users, and so on.
- stop muttering to the other techies that the objectives of the project are unreasonable and say it to the project manager or even better to the business or the sponsor
- deliver something useful in as short a time as possible to start getting feedback on it (you know, like we keep saying we want to do because that’s what it says in the agile and lean books – but there’s always another yak to shave or another cool framework to play with)
- work as a real self-organising team, doing whatever needs to be done to deliver, acting in the best interests of the team and the customer rather than in our own interests (‘cos it says that in the books as well)
- in other words, how about we start acting like grown-ups for a change
The way we get trust is by delivering useful, usable, maintainable software in a reasonable timeframe. If we don’t do this then why should people trust us? We need to earn trust at the start of a project/product by being trustworthy (see above).
Maybe then we can earn the right to say #noestimates.
Because this is what I believe it boils down to:
#notrust == #no_option_but_to_estimate
blueskyline 
While this is all good ideas, the notion of estimating the cost, schedule, and likelihood that the needed capabilities will show up on time is a business process not just a development process.
As a business person, “trust” comes with “verify.” As a governance based business person, “trust and verify” are part of the governance basis of business in the same way double entry accounting is.
Verifying that within that time box “something” of value will be produced, requires knowing something about “what” will be produced, with what effort, and with what quality. This “verification” is not a replacement for “trust” it is the foundation of trust.
In the same way accounting “verifies” that the received invoice matches the Purchase Order and then matches the receiving report – this is called Three Way match. “We placed the order, got the material, and it matches what we ordered.” This is the basis of “stewardship” of our shareholders money.
Estimating the future cost, schedule, and delivered capabilities in probabilistic terms is part of the stewardship of the shareholders money. The probabilistic nature of Software Development requires we manage in the presence of uncertainty using estimates. Uncertainty comes in two forms.
1. Reducible – I can “buy down” this uncertainty by spending money
2. Irreducible – I can’t “buy down” and con only deal with it with Margin
How much money and time is needed to buy down the reducible uncertainty, and how much margin is needed to protect our firm from unanticipated cost, schedule or performance variance is also an estimating process.
In the end the “Value at Risk” drives – or should drive – the conversation. With Low Vale @ Risk, we’re willing to trust and not verify. With High V@R, our governance processes – stewardship – tells us to verify the basis of trust.
Those proffering we can make decisions in the absence of estimating the cost of that decision, the impact of the decision, and most importantly the cost of “not deciding” – the opportunity cost – have not come in contact with the Microeconomics of software development in the presence of governance frameworks.
No governance framework, you’re pretty much free to spend as you please.
These seem like very valid observations from my personal experience. Cost is the rarely-mentioned non-functional requirement in most systems but it seems not to be considered by many software developers but it is an essential consideration for any project or product. Without some form of estimating, cost cannot be assessed so a key pillar of successful business planning is missing.
Risk is also a personal bugbear of mine. I like Lister and DeMarco’s reflection of risk management being project management for grown-ups. Software teams cannot themselves accept the risk of project failure so must allow others to put in place mitigation for some form, which could involve more planning than the team themselves would want.
Thanks Andy, you clarify in my mind that NoEstimates is not for immature teams. Good post
Good point. I have come to a personal conclusion that most agile and XP practices are not for immature teams.