When the business ask for a change, do you know how long it will take until they are able to use that new feature in production? What is the basis for your estimate? How confident are you in achieving it?
Recently a video showed up in my YouTube feed by Cat Swetel1 titled, The development metrics you should use (but don’t)2. She started off using just the start and completion dates of tasks to create a metric, Time in Process. The units of time taken to complete a unit of work.
Within the our team, all of our tasks are tracked in JIRA. I put together a simple bash/python script to extract this information, and using an Excel sheet from Focused Objective3 generated some of the Graphs that Swetel described based on our data. To keep things simple, I’ve assumed that each JIRA story (unit of work) is of equal weight.
Time in Process scatter plot
The first of these graphs (Time in process scatter plot) plots the feature delivery date on the horizontal axis, and the number of days to deliver on the vertical axis.
You may notice that there are vertical lines grouping on certain dates. These are our monthly release dates. Not everything fits on these monthly release dates, as we have some QA third party testing tasks, or UAT defects which do not require a formal release to be completed.
We can also see that it takes on average 60 calendar days (the Excel sheet does not account for weekends or holidays) from a story being first entered into JIRA until it’s completed and usable. This does not account for time in which the BA thinks about a Business requirement, i.e. compiling Confluence documentation, before creating a JIRA.
How responsive are we?
If someone asked how long it would take to complete a story, we can respond that on average it takes us 60 days. How confident are we that we can complete a change within that time? At a simple non-mathematically sound (no standard deviation etc.) response would be 50% confident, as half the time we finish stories quicker, and half the time it takes longer.
Taking this basic logic if we want to be 90% confident we can complete a story within the estimated time, we would draw a line where 90% of the dots are below it. This is the 130 day mark. That’s a big difference between confidence levels.
This is a large variance in Time in Process to increase the confidence level of delivering within the estimated time. That can also be seen in the graph above where the Time in Process appears to have no consistency. There are the high outliers where the stories have low priority, and been put off many times.
So what does this mean?
Due to the variance with the time in process, it makes it hard to reliably predict how long a feature takes to deliver. Sixty days, or two release cycles, sounds a reasonable response time, but I feel that we should be able to do better. We have a really good team, and if there is a date on which a change is required (i.e. regulatory requirement) the team will find a way. But this hero culture is not ideal.
What we want to do is find a way to become more consistent with delivery. A monthly release cycle make it inevitable that completed stories will sitting around until the release date. However, we also have other buffers in our development flow, between the BA analysis and handing over to the dev team for some design. Then until the product owner prioritises the story to be developed in a Sprint, and then again at the end when the QA team want to all the related stories in a release. All through the flow there is waste.
It would be interesting to see how much time the stories spend in each of these buffers. That is a story for another time.