Here is a list of the top 10 tips i find myself giving out. Its not in any particular order of importance, just the order they come to my head. Its a long weekend, so writing things down helps me relax. Would love to hear yours, so please add them to the comments.
1. If two measures correlate, stop measuing the one that takes more effort. E.g. If story counts correlates to story point forecasts, stop estimating story points and just count.
2. Always balance measures. At least one measure in the following four domains: Quality (how well), Productivity (how much, pace), Responsiveness (how fast from comitting), Predictability (how repeatable) (thats Larry Maccherone)
3. Measure the work, not the worker. Flow of value over how busy people appear. Its also less advantageous to game, giving a more reliable result in the longrun. Measuring (and embarassing) people causes poor data.
4. Look for exceptions, don’t just explain the normal. Find ways to detect exceptions in measures earlier. Trends are more insightful than individual measures for seeing exceptions.
5. Capture at a minimum, 1- the date work was started, 2 – the date it was delivered and 3 – the type of work (so we can see if its normal within the same type of work).
6. Scope Risk play a big role in forecasts. Scope Risks are things that might have to be done, but we aren’t sure yet. Track items that might fail and need reworking, for example server performance criteria or memory usage. Look for ways to detect these earlier and remove. Removing isn’t the goal – knowing if they will definately occur adds more certainty to the forecast.
7. Don’t exclude “outliers” without good reason. Have a rule, for example 10 times the most common value. Often these are multiple other things that haven’t been broken down yet so can’t be ignored.
8. Work often gets split into smaller pieces before delivery. Don’t use the completion rate as the forecast rate for the “un-split” backlog items. Adjust the backlog by this split rate. 1 to 3 times is the most common split rate for software backlogs (but measure your own and fix).
9. If work sits idle for long periods waiting, then don’t expect effort estimates for an items to match calendar delivery time. In these cases, forecast system throughput rather than item sizes (story points).
10. Probabilistic forecasting is easier than most people expect. If average are used to forecast (like traditional burndown charts) then the chance of hitting the date that gives is 50% – a coin toss. Capture historical data, or estimate in ranges, and use that.