This morning I had the opportunity to chat with software engineers and data scientists at the AI Dev World Conference on a topic I just happen to be v...
As Yogi Berra said, "It’s difficult to make predictions, especially about the future." Hard to argue with—particularly if you work in software engineering.
We’re terrible at estimating. It’s why we’ve had decades of over-budget deliveries and projects that drag on and on. We’ve all seen it. I’ve been a part of it. And unfortunately, it’s at the core of what feeds some of the cynicism about software orgs. Small wonder that estimating remains one of the most difficult and least enjoyable aspects of a developer’s job.
Yet we still cling to estimating and spend countless cycles doing it…with dubious results. Part of the problem is our optimistic nature. We are (over?)confident in our development prowess and assume we’ll escape the surprises and outside influences that inevitably arise.
Our own teams here at Pinpoint aren’t immune. The graph below comes from Pinpoint’s Sprint Health view. It shows, across six months of our early sprints, how our initial plans compare to what was actually delivered.
Read this graph like a funnel or pipeline, flowing from top to bottom. The first bar represents all the issues in initial sprint plans. The second bar shows how many issues made it from initial plan to final plan (the dark blue), as well as the number of issues added as a part of final planning (lighter blue). The last bar shows how many of both initial-plan and final-plan issues were actually delivered.
What jumps out?
We consistently delivered about half of what we thought we would at the beginning of the sprint.
Even more intriguing: across our customer base, we see this exact pattern over and over.
I’m glad you asked. I do think Agile has helped a little. We are aware sooner that our estimates are off. But are we more accurate?
To their credit, the various flavors of Agile tried to make the best of a bad situation. We’ve seen lots of creative solutions – from story points to planning poker. But even Ron Jeffries, co-founder of XP and one of the original signatories of the Agile Manifesto, has some regrets:
"There are a number of ideas about how to estimate using something other than time. Points, Gummi Bears, Fibonacci numbers, T-shirt sizes. These were originally invented to obscure the time aspect, so that management wouldn’t be tempted to misuse the estimates. (I know: I was there when they were invented. I may actually have invented Points. If I did, I’m sorry now.)"
For software teams and their business stakeholders, these abstractions have only added confusion and opened the door to more questions: Do we size based on complexity or amount of work? How many days equal a story point? Do we factor in uncertainty and risk? Do we adjust story points mid-sprint if estimates are off? Should we assign story points to bugs? Why are story points different across teams? You’re estimating my project using what?
Consider two key facts:
In other words, let’s let data science do the heavy lifting, while letting developers get on with the actual work of building great product.
By scanning the entire work history of the organization, and examining the metadata associated with all the work items (including parent items, children, grandchildren etc.), Pinpoint derives historic Cycle Time and Throughput at the organization, team and individual levels. We refine this calculation further by analyzing type and priority of the work. Armed with this analysis, we can apply it to any new work and automatically forecast the time required.
Illuminating historical results in this fashion can make life easier in a number of ways:
As teams pack sprints (or other groupings of work) with work items, aligning to their historical delivery benchmark is a fast way to establish an initial plan. It doesn’t remove the need for experienced team members to review, confirm and tweak the plan, but it does provide an automatic, actuals-based starting point in place of the usual ceremony around estimating.
To illustrate, below is an excerpt from Pinpoint’s Sprint Pipeline view. The Plan Capacity on the right shows the calculated forecast of the work designated for a sprint (yellow bar) compared to the average historical capacity (gray bar). In the case below, we see the sprint is only 58% full.
I can’t count the times I’ve been assured (or assured someone else) that we were on track to deliver, only to find out right before the ship date that it will take another week...or three. The same forecasting intelligence can surface to leaders and teams when something is potentially at risk, so we can take better corrective action, sooner.
As an example, consider the screen snippet below taken from our Work Forecast view. Work Forecast automatically creates a Risk Profile for every major work initiative. This particular example shows that one of the teams (“Front End”) has more work on this particular initiative than can realistically be completed in the time remaining.
As described above, Forecast Work Months is automatically derived by looking at the entire organizational history of work, and using that mass of data to determine, by work type and priority, how long it’s likely to take each team to finish its piece of work on each initiative. Think of Work Forecast as an early warning system for issues at risk, and a guide for where to start asking questions.
There’s another big thing afforded by having your historical results and forecast signals at your fingertips. The ability to shift discussion and debate from subjective (and sometimes heated) opinions, to neutral, hard data. Let’s not spend time debating if more work can be squeezed in or if we are already over capacity. Let’s not gather opinions on whether or not we can pull the date forward. Let’s instead look at what our history says, and use that data to have a rational conversation about the right course of action.
The more time and energy we can shift away from manual estimating, the more momentum developers will have for doing what they do best. Let’s stop estimating, stop debating, and start delivering.
Meet Andrew. He’s our Director of Backend Engineering and is a member of Team Bolt ⚡️ who is currently working on buildi...
Meet Mike. He’s a Platform Operations Engineer here a Pinpoint and member of Team Bolt ⚡️ . who is currently working on ...