On Estimations
One of the first rules that one learns as a software developer is that everything takes three times longer than you know it should. This may seem an overstatement, but there are reasons behind this claim. I have tried to analyze them in the articles Estimating through randomness and Estimating through biases, and the objective of this third article is to go over some of the learnings that I have gathered both from this research on estimations and from missing an awful lot of estimates as a software developer and manager.
But before proceeding to these learnings, it is important to define the boundaries and expectations around what we are providing when we provide an estimate. More importantly, it is necessary to define what we are not providing, and avoid mixing concepts that even though are related, are not quite the same.
Commitments are not estimates. An estimate can and probably should become a commitment, but it does not work the other way around. When a commitment is established before a proper estimation is made, it blurs the entire estimation process and distorts its very purpose, turning an estimate into a negotiation tool.
If a particular estimation was that a project will be concluded in eighteen months, and the business requires it to be concluded in twelve months, an estimate is not a tool to split the difference and set a fifteen months goal. And even though this may seem obvious, it is something that I have witnessed time and time again. Very often projects that have been committed on a timeline before a proper estimation was made reach their final stages and the realization that the date will not be met leads to a crisis, but this crisis is merely the end of an illusion. The objective of having good estimates is to avoid creating these illusions, and to establish commitments that can be relied upon.
Let’s take a closer look at what makes for a good estimate so that we can then analyze how to build one. A good estimate is honest. This is a fundamental piece of it, because honesty is what makes a team delivering a project to be truly committed to its milestones and deadlines. In addition to this, a good estimate is also accurate, and it is precise within an acceptable range. And that is it. An honest, accurate, and precise estimate is a good estimate.
I believe we can take for granted what honesty means, but let’s analyze accuracy and precision in the context of estimates. A few weeks ago someone who knew that I was writing an article on estimations asked me about its completion date. If I would have responded that I was certain I was going to complete it sometime before December 2040, it would have been a really accurate estimate, but not a very useful one. It was lacking precision. Instead, I replied that it was likely that I would finish it by August 2021, trading off accuracy for precision and providing a more useful estimate.
Once we have defined expectations around accuracy and precision for an estimate, we can begin working on it. A good recipe is to start with a broad estimate, and then iterate to refine it. We can use data of completed comparable projects, taking into consideration similar scenarios for which we have information, and adjusting that information by adding to our analysis the particularities of our project. It is important to remark that a comparable scenario takes into account not only the nature of the project but also the composition of the teams that are building it.
If we do not have this information available then we will need to build an estimate from the ground up. A good approach is to start by analyzing every task that we think will be necessary to complete a project, and then to multiply the resulting estimate by a certain factor. What is the multiplier best suited for each project will vary greatly, and the considerations to be taken are in direct relation to the size of the project and the expected range of accuracy and precision. I do believe that for each team and project there is a proper multiplier that can be applied to the first estimate draft, resulting in a significantly better estimation. And I have to mention that this multiplier is never, ever, one.
If a team that has been working together for years is estimating the next 2-weeks sprint, then possibly a 5% deviation — a 1.05 multiplier — will be sufficient. But if we are estimating a project in which multiple teams will be working together for the first time, and the project will require the development and integration of tasks that at first glance account for at least 6 months of work, then that multiplier may be closer to three. With this multiplier we are accounting for our intrinsic biases as estimators, unexpected random events that will arise, tasks that we did not foresee, time consumed integrating tasks, and many other factors. The risks of a project are not the risks that we want to calculate, they are the risks of the real world.
Another important consideration is that when we are estimating large projects, our estimates should never be an exact date but a date range, and this range should take into account both optimistic and pessimistic scenarios. When someone asks us for an estimate and we reply with a date, we are collapsing every possible scenario into a single outcome that encompasses all possible scenarios. This is explained in the book Thinking, Fast and Slow, which reviews the many ways in which we are routinely misled by our intuitive way of thinking. So when we are analyzing the future, we should do it in terms of probabilities. There are methods such as Three-point estimation that can be used to build a probabilistic estimate, but more important than the particular method is the concept of estimating date ranges with probabilities associated with it, and never a date.
Another key factor for improving our estimations is to acknowledge that when we find out that an estimate is wrong, it means that some assumption that we have based our estimate on is wrong. Under this situation, we should analyze which assumption was untrue to learn from it, and for us to be able to do this we should define the milestones of a project before it begins. Without doing this we will just go along with the project as it unfolds, and justify its development as unavoidable. We will be throwing the arrow first and drawing the target around it later. So we need to document both our milestones and our assumptions, and compare them with the actual dates as the project develops.
As a summary, we should aim for estimates that are honest, accurate, and precise, and for doing so we first need to recognize that there are both internal factors, such as cognitive biases, and external factors, such as randomness or real-world risks that will inevitably affect our estimates. We should also avoid creating unrealistic expectations by committing to dates when a date commitment is not possible, and instead provide date range estimates with a probability range associated with them. At last, the only way for improving is to measure, and for doing so we need to set the expected path of the project before it begins.
Once again, estimating is one of the easiest parts of software development, said no one, ever. But if we recognize where it is that we are more likely to fail and implement the proper methods and tools to compensate for those potential failures, we can do a very good job at it.
