Estimating through randomness

Estimating is one of the easiest parts of software development, said no one, ever. Hard as it is, it is just as important, and being able to deliver projects as planned makes teams, projects, and ultimately companies reliable.

As with everything else, an important step towards getting better at estimating is finding out the reasons behind its complexity. I believe randomness and cognitive biases are among these factors, so I will focus this article on how estimations are impacted by randomness, a second one on how they are affected by biases, and the last one on processes and practices for improving estimations.

But first things first. Let’s broadly define an estimation as the most knowledgeable statement we can make at a particular point in time. In the domain of software development, estimates typically come in the form of ‘How long will it take for this task to complete?’ questions, adding up for answering ‘How long will this project take to finish?, oftentimes integrating into the final and most important of all: ‘When will this initiative be delivered?’.

And as tasks pile up, so does the probability of occurrence of random events. Estimating is a probabilistic exercise, and as such, randomness is an inevitable part of it; we should take this into account so that we don’t take for certain what is merely probable. Applying the aphorism for modeling complex systems: All estimations are wrong, some are useful.

Randomness comes in different shapes and sizes. A widely spread categorization for random events in the context of project management groups them in Variation, Foreseen Uncertainty, Unforeseen Uncertainty, and Chaos, and even though I do think these are all fascinating from a theoretical point of view, I will analyze here the impact of Variation and Foreseen Uncertainty. I am doing this because I think the first two can be managed operationally, while Unforeseen Uncertainty and Chaos can only be mitigated from a company-strategic perspective.

Let’s assume that we have estimated a project composed of one hundred tasks, concluding that each of them would take approximately one hour. To keep things fair, let’s also assume that we are very good at estimating.

The project unfolds. 60 tasks are completed right on time and 20 tasks come in at half the original estimations. Up until that point, 4 out of every 5 tasks have been completed either on time or better.

For the remaining 20 tasks, nothing dramatic happens. Just a few slips. 15 tasks take twice what we estimated. One morning, a developer has problems with the internet connection, so there is not a lot of progress that day; we can account for that as 4 tasks taking a whole day instead of half a day. Finally, there is one task out of a hundred that proves to be extremely difficult, and it takes not only reading through every article even vaguely referring to the topic in StackOverflow and daydreaming of opening a bar at the beach in Rio, but also two whole days to complete.

When everything is done, the vast majority of tasks were completed either on time or early… and the project is delivered 24% late. Converting this estimation to days implies almost a month of delay in a 3-months project, having started with an estimation that was almost unrealistically accurate.

This seems not only unfair but also a bit counterintuitive. We completed eight out of ten tasks on time or better, underestimated 15 of them, and had just one morning off and one unexpectedly hard task. Where is all the time we gained in the 20 tasks we delivered in half its estimation?

The answer lies in asymmetry. As easy as it is to miss our estimation by a factor of two, the impact of completing a task in half the time is very different from the one of doubling it. For every task lasting twice what we expected, we need not one, but two that take half the time to compensate.

A small number of surprises and a reduced amount of (likely expected) randomness can have a substantial impact on estimations. The very definition of randomness is unpredictability, but even though we can not predict randomness, we can account for its presence in varying amounts, and consider that the higher the number of tasks and the longer the duration of a project, the more likely it is that unexpected things will happen.