Update: One thing I forgot to mention. If you didn't want to read all this or if you disagree altogether. Ask yourself: you don't want due dates. So are you willing to give up your salary due date as well? Why can't you estimate what to deliver and your client has to pay you regardless? Let's make this even: you do #noEstimates if, and only if, you are willing to #noSalary. Your employer shaw withhold your payment until you deliver, and in this scenario your payment must be depreciated the longer it takes. Unfortunately labor law doesn't allow that. But it would be an interesting scenario.
There's been a lot of people currently talking about #noEstimates. I've read most of the arguments in its favor and you can Google it easily enough so I won't make any extensive references to any of them. The gist of it is that estimates will never be good enough, the more specification and planning you do seems to never increase the quality of the estimates, specially because in a dynamic market specification change too often and the more estimation efforts you do, the more the waste. And because estimates seems to be such a waste why not get rid of them altogether?
Feels like a noble idea, specially for software developers. Software is maleable, it's abstract, it just feels like it doesn't fit traditional notions of project management. And while we are at it, why not get rid of the entire notion of projects as well? And hence, another trend just emerged: the #noProjects.
My intention here is not to answer each of their arguments, that's not the point. What I will do is explain why the entire idea is absurd in the first place. So let's get to the basics first.
One thing that I advocate since at least 2008 is the idea of project management and markets in general through the models of Complex Adaptive Systems, Chaos Theory and Evolutionary Biology. I've been largely influenced by the ideas of Nassim Nicholas Taleb and his magnum opus "The Black Swan". It's an incredible idea that the markets are not bound by linear paths, but by chaotic agents that influence a complex system. All companies are managed to deal with averages, with limited sigmas as margins of operational error. But once something big, a "Black Swan", such as the 2008 economic crisis arrives, most are not prepared to deal with it, no models are able to predict it, and the whole system shuts down and collapse.
If you're unfamiliar with the idea, Google it for a moment and you will understand that companies, markets and people relationships in general, are dynamic systems that conform to evolutionary biology rules. Decentralized systems seems to be the way to go. And those concepts influence many of the Lean movements we see today. So, yes, I am very well aware of those effects.
But as a summary, the whole gist of it is: the ones with the most chance of survival in such a complex system are the ones that are the most adaptable, not the ones that strictly conform to plans. Conforming to long term plans and sticking to it with rigidity is the easy way to fall down when Black Swans happen.
That said, it made me think about another concept: Einstein's General Theory of Relativity. In modern cosmology it superceded the Theory of Newton. When I first learned about it my main thought was: if Newton is "wrong", why aren't we using Relativity to calculate everything in our day-to-day lives? The answer is that Newton's theory is only "wrong" if you define it as the being able to calculate everything, but it isn't. It can't be applied to the very very large, gravitational calculation, galaxy level calculations. But, if you limit it to Earth like sizes, where we are calculating the path of an airplane, the trajectory of a bullet, etc, it is still applicable, as the margins of error are negligible. So in the day-to-day operations, we can reduce the problems to Newton and not use General Relativity. This is an oversimplification, of course, but bear with me.
The same applies to companies. We are all bound to Power Law distributions, Evolutionary Biology, the relentless forces of Chaos that make everything behave like Complex Adaptive Systems. But in a constrained environment I will argue that we can reduce the calculation back to Bell Curves. This is the most difficult to "prove" so I will not attempt it right now but the following explanation may get you there.
Let's define what a company is: it's a set of operations. Operations are repetitive activities. So you have activities such as "pay a supplier", "send a purchase order", "process the payroll", "transport products", etc. The set of all those activities define what a company is.
The whole idea of a company is to perform those operations in the most efficient way possible. You do that by continuously refining the process in small steps or by means of a breakthrough, changing the entire way you do a particular operation. For example, back in the day, there were entire groups of people dedicated to fill in paper forms and organize them to make information flow within a company. With the emergence of digital systems, ERPs, all that paperwork is not necessary today. We were able to get rid of an entire profession of typewriters and we added efficiency and more precision to the system. Breakthroughs are usually the digital automation of manual labor or getting rid of a process altogether.
To accomplish such breakthroughs there are Projects. Projects are temporary endeavors where a group of people concentrate to achieve some pre-established goal. Projects usually have a fixed starting and due date, a fixed budget, and a fixed amount of people involved.
And here we arrive to the Estimation part: all projects want to achieve some goal. In the particular case of a software project, we implement software code that is meant to achieve that goal. To achieve a goal we come up with software features and we break them down into Use Cases, User Stories, Requirements or any artifact meant to describe what is to be built. And then we estimate how much resources (time, money, people) are necessary to implement each of those pieces and integrate them together in a "solution" that solves the problem and achieves the goal.
What software developers complain is that it is not possible to estimate those pieces with precision and projects will always be delivered late and overbudget. So the best idea is to not estimate and simply start coding and delivering value as soon as possible and consider it done only when its done.
Some software developers get so fed up of this whole notion that they want to leave their companies to start their own tech startups where they will be able to do whatever they want without any controls. Then they look up for investors because they need a lot of money and a lot of time. Of course they do. What they don't realize is that EVERYBODY needs a lot of money and a lot of time. And investors know this: the idea is irrelevant, execution is key. The people that deserve more money and more time are exactly those that can pressure themselves into delivering underbudget and ahead of everybody. Just doing something and not being worried about constraints is exactly the affairs of the mediocre. And mediocre don't deserve anything.
Now back to basics: we have something called Economy exactly because resources are not infinite. Everything that has value has a price, be it a physical product or working hours.
Understand this: in the services business (where we software developers are all in, regardless if you're an employee, a co-founder, etc) value has only two variables: quality and efficiency. We tend to regard quality as the only thing that matters. Worse: we tend to regard what we think as quality as the only thing that matters.
This brings us back to CONTEXT. Most programmers are bad at estimates. And the root reason is because they are usually utterly incompetent with understanding context. As someone with a Math background I read all articles about processes, methodologies and stuff like "#noEstimates" as "formulas".
Formulas alone don't mean anything. Any mathematician knows that you have to define a Domain and an Image, the source and the destination of the all inputs and all outputs. For example, if I show a formula such as "f(x) = x / 0" you might argue that this is an invalid and wrong formula as I can't divide zero by zero. But if I say that the Domain is every Natural number but zero, now this is a totally valid formula for the Image of the Naturals.
So, when someone says "#noEstimates" it begs the question: on which Domain and for what Image? This is the origin of the discussion for most of the statements made in the Internet: people will argue for or against an idea because each is in a different Domain. That's the same for Agile in general, Lean Startups, Lean Manufacturing, etc. They usually only define practices, procedures, formulas, but they rarely define Domain and Image. This makes a confusion and loses the point.
What I will define, though, is that Projects are necessary everytime there is a defined goal to be achieved. This is the domain. And I will also state that when people state "#noEstimates" they are not in the Domain of Projects but of "Ongoing Operations". Actually, this is where Lean Manufacturing in general lies as well. I have explained Operations above, and this is where small improvement steps (Kaizen) emerges. Sometimes the feedback loop from an operation gives enough input to justify a Project to make a larger step and a breakthrough.
Projects, on the other hand, continues to be a temporary endeavor. The whole idea is to establish the boundaries, such as time and cost constraints. And we come back to the original question: but it is impossible to predict the efforts necessary for something as maleable as software development.
First of all, yes, it is impossible to predict with exact precision. Again, let's define it: it's impossible to predict a number with zero margin of error. Estimation is prediction with margins of error. And why is that some projects cost more than twice as much and take twice as much time as the estimation?
More often than not, because the team is incompetent, that's 90% of the cases. The problem is not the estimation, it's the execution. Estimation is the establishment of an expectation. Expectations must be managed. An estimation is only good if the context is taken into consideration. Most people don't like to estimate because "what if" scenarios. What if the client change his mind? What if we find a difficult obstable? What if a meteor strike the Earth and all living creatures perish? We can't manage "what if", what we can manage is what we know and create constraints. Project constraints start with its goal. To accomplish those goals we also establish the rules of engagement: the premises. Without goals and without premises there is no game.
None of those guarantees a Prediction. An estimation is only as good as the execution. Now we have to manage it. Everybody has to manage it. It's no good if you define clear rules and all of a sudden you see a programmer doing nothing and when you ask him why he is doing nothing his answer is "oh, because I emailed the client about some requirements and he never replied back so I was waiting". And when you ask "and did you try calling him?", usually the answer is "No, I didn't". There is no amount of processes, methodologies that can "fix" an incompetent employee. Lack of technical skills can be fixed. Bad faith cannot.
The ones advocating #noEstimates can argue that they are not like that and I can believe it. But 90% of the projects that failed had employees like that. Programmers tend to put the blame on the client, the bosses, the market, but never themselves. And as a programmer I will argue that the reason most projects fail has nothing to do with changing requirements, limited time, but with lazy employees. Want to make projects go right? Start with Human Resources first, then go find methodologies and practices.
The problem with all methodologies not stating the Domain under which their formulas work is that most people don't realize that the Domain start with "having competent, committed and skilled employees". Most adopt methodologies with the hope that they will turn incompetent employees into competent, and that just doesn't happen.
With that out of the way, why do we need estimates? Or more generally, why do we need constraints? Because that's the core of value of any system. Nature puts pressure on any living species: changing climate, limited food supplies, predators. The species that are most adaptable evolves, the ones that can't adapt perishes.
When someone says "it's impossible to do X", that's probably the most valuable goal to pursue. Because the whole statement is: "it's impossible to do X with what we know today". It was impossible to go around the globe in 24 hours, it's not anymore. It was impossible to communicate with the other side of the globe in real time, it's not anymore.
Constraints are the foundation of innovation. If I had to define innovation I'd say that "it's the process by which you accomplish something that was deemed impossible before".
If you have infinite resources, or if you just don't need to worry about constraints (that's what happens in bubbles), you don't innovate. This is so important that I will repeat it:
Innovation is the byproduct of Constraints.
We estimate based on past knowledge. If we can't out perform our past selves, how incompetent are we? I mean, we can argue that software is not predictable, and I agree. But making mistakes of orders of magnitude in recreating similar software only smells incompetence to me. Most of the software we produce are not brand new ideas, breakthrough new algorithms. They are really mostly the same: content websites, ecommerce, elearning, social networks, social commerce, forums, polls. Unless you work in research programs, what other kinds of different software did you do lately?
Estimation is rarely the problem. All estimation comes from a set of premises. Not managing those premises is the problem. If requirements change, this is not a problem. We can always manage that. We can't manage accumulated problems in the last day of the project. And usually the case is that programmers defer dealing with problems. Again this is not a problem with estimation, it's a problem with Human Resources.
Meritocracy only exists in a system of scarcity, where one stands out against another compared to a constraint. In a system where there is only abundance, there is no need to innovate, there is no need for merit. Companies that just received a humongous amount of cash will invariably show symptoms of laziness, not innovation. The confusion arrives because some of the "practices" being stated derive from this temporary unreal situation of a bubble, and can't withstand the pressure of time. Give it enough time and you will realize "why did we, with so much cash and so much time, accomplished so little, and the other small tech startup, with so limited resources was able to outperform us?". And this is the answer of why a Yahoo! buys a Tumblr, why a Facebook buys an Instagram, why a Google buys a Waze.
If you advocate #noEstimation, why not go a step further and advocate #noWork? It's just one step further.
![]()
