Problem in a nutshell: During development of a feature or project work can often (although not always) be delivered faster with more people or teams. At some point in a project, most often near the very end, a few defects or missing features remain unfinished to a releasable quality. These items block the release, but CANNOT be accelerated by adding more resources or cutting scope (they are part of the minimum viable set of features). To forecast delivery date of this phase of a project or feature, you must consider the longest individual item forecast, not the backlog driven probabilistic forecast.

Near the end of a delivery cycle work is blocked by a few remaining items, either defects or agreed necessary completion of features. Much of the delivery system will appear idle at this time but these items don’t reduce time with added people. They are staffed to the necessary level and all viable ideas for accelerating them have been implemented. This phase of a delivery I call “Close-out.” Its where how much work remaining matters less than how long the longest single remaining item will take.

I find probabilistic forecasts (eight out of ten of mine) reliably predict a completion date EXCLUDING this phase. When i’m wrong, i’m wrong because i’ve not adequately managed or modeled this part of a project or feature delivery. My only mitigation is to minimize or avoid this phase by ensuring done means done, and by putting effort into excluding risky outcomes earlier in the project. Its a significant issue especially for large installed applications and user bases where regular releases still aren’t as common as they should be.

How to tell you are entering the Close-out Phase

First step in understanding the close-out phase is recognizing when you are entering it. I look for a point where work-in-progress for the entire team or organization decreases below its 75th percentile. A key point of close-out is that the number of people who can work on the remaining items are at saturation levels, so the remaining people help, but sit idle. In reality, its not that simple to see. Idle people aren’t tolerated (although they should, but that’s another blog post), they move onto the next release or do technical debt removal or needed refactoring. This is healthy, but not pertinent to the current deliverable. I exclude these and look for the actual work in progress at an item or person level for the deliverable held hostage. When this is down 25% from its highest, I assume if 1 in 4 people are doing other stuff, then the system is saying we are time-based not scale-based in throughput. I use this 1 in 4 people to manage and mitigate future close-out risk.

How to manage the Close-out phase

I capture a list of all of the remaining items blocking release. Using the ¼ of the people NOT directly involved in the close-out items but still allocated to the project, i give them the mission of brainstorming items that still may crop up (to avoid worsening the problem), and to be available on request for immediate code reviews and testing the close-out items. I also collect ideas as to how these items may have been detected earlier to minimize this impact earlier. Ideally, this brainstorming should be constantly performed be all team members.

Close-out item causes and mitigation

Forecasting in advance is difficult, the best process minimizes or avoids these close-out items be looking and dealing with them earlier. When there is no data for the proposed project I look at other projects this team or organization has done.

I particularly look for external players who can block a release. My most common reasons are –

  1. External parties have a final go or no-go decision, and they give late feedback
    • Know who has the final go and no go decision – if its external incentivize them to help earlier
    • Get sign off as early as possible; give these people previews and hand-hold them
    • Learn how they will test and decide the system is “good,” for example, get them to supply you with THEIR test data and cases during development
  2. Performance testing is performed at the very end and the feature fails required performance.
    • Agree what impact is acceptable for the new feature or project from the prior
    • Build enough of the system to do performance testing early and retrofit changes earlier if it fails
  3. Edge Cases. Browser versions (Internet explorer in the old days), operating system updates (android and iOS particularly)
    • Learn what browsers and operating systems are essential. Ask sales and marketing what the top 10 customers use and make sure you do these!
  4. Localization and Internationalization
    • Late changes to text means late response to language translations. Get FINAL text signed off early.
    • Don’t wait to the end of the project for someone to check German number and date formats!
  5. Defects
    • Keep defect count low. Keep things at known point of stability, and encourage that point be releasable (thanks Michael Tardiff for this terminology)
    • Rigorously get defects discovered. Schedule bug bash days to shake out the defects early. Celebrate finding defects, and then get them fixed. Biggest risk is that you just think you are in a known stable and releasable point.

Sitting down with a team early and looking for items such as these and avoiding them is the only way to minimize the final impact and allow forecasting to be reliable. Do brainstorming session specifically on these items often by asking the simple questions “What could stand between us and release of this product or feature tomorrow?” – its my standard stand-up question, or the first question I have for anybody I meet in the hallway.