Add to Google
Agile Release Train Metrics
Posted on June 22, 2011 by Dean Leffingwell in Agile Release TrainI’ve had an On-again, Off-again, love-hate relationship with software metrics throughout my career. The On part is:
- of course we have to measure ourselves, less we a) not advance the science of software engineering in any credible way, and b) not provide any professional, quantitative input to the fiduciaries and key stakeholders who ultimately determine the economic benefit of our work (and indeed decide to pay us or not)
the Off part is:
- a) why don’t any of these metrics we’ve traditionally applied actually seem to work?, and b) can’t all of these be used by management to drive behaviors that may eventually be counterproductive, and/or c) be manipulated by teams to some similar, ill result?
But the On part of this debate within myself continually informs me that “measure we must” , particularly if we want to be credible with our agile model in the enterprise setting. So, I took my best shot at project, process, and an enterprise “balanced scorecard approach” to metrics in Chapter 22 of Scaling Software Agility. I reviewed it today and it still seems credible (at least to me).
Recently, however, I’ve been repeatedly faced with suggesting metrics that can work well in the context of each PSI/release in an Agile Release Train. That’s a simpler discussion, because there, context is known, and the measurement problem is focused on a specific program, with specific teams, in a specific technical and business context. To that end, I’ve recently been presenting the following set of metrics to the release train programs and Agile Working Groups I’m involved in.
A set of reasonable metrics for an agile release train
At each PSI, each team self assess and report on the above, and the aggregate can be rolled up into a program level summary, which should establish some sense of health and progress for that train.
Most of these are fairly obvious to agile teams. I described least obvious one, “percent of business value” achieved in this earlier post.
Hope this helps at least somebody.
We have tried the above metrics and had mixed results. If you ask the team to present these metrics at a demo, it will turn into a metrics demo and a working software demo. I recommend team publish their metrics on an internal wiki or better yet straight out of an Agile tool. We are a Rally shop and this weekend I had the chance to dig into their “standard” reports that contain metrics. I group them into 3 types:
1) Work (tracking epics, features, storied and tasks)
a. PSI Burnup (if setup correctly you can track it at a team, product and portfolio level)
b. Story Burnup (if parented correctly, you can track story, feature and epic progress)
c. PSI Cumulative Flow (track how much work is in progress
d. Sprint Burndown and Burnup (if setup correctly you can track the entire trains sprint)
e. Story Cumulative Flow (used for epic, feature and story tracking)
2) Quality
a. Sprint Defect by Priority (see how many are Critical, High, Low, etc within the sprint)
b. Sprint Defect by State (see how many are open or closed
c. PSI Defect Trent (see how many are open and trending during a PSI)
3) Capacity
a. Velocity (team, product portfolio)
b. Cycle / Lead Time (see how fast stories get through you system
c. Throughput (The count of work items accepted in a given interval)
My hope is to used the standard reports and see what other questions people have. Like someone once said, “Metrics are like a light post for a drunk, they provide illumination and support.”