Agile Health Metrics for Predictability

This content is syndicated from LeadingAgile by Andrew Fuqua. To view the original post in full, click here.

LeadingAgile uses Agile Health Metrics to demonstrate the results of our process improvement efforts and to identify areas that need further improvement. We have many internal documents describing our approach that we share with our clients, but to my surprise, it seems that we have never blogged about it. Here is a high-level view of the metrics we often start with.

When deciding what to measure, the place to start is with a goal. First, ask yourself what outcomes are you after, your goals. Then consider what is needed to meet those goals. And finally, what metrics indicate whether you have what you need. You may recognize this as the Goal-Question-Metric approach.

Our clients tend to care about predictability, early ROI, improved quality, or lower cost. Predictability seems to be paramount. They want teams to get good at making and keeping promises, consistently delivering working, tested, remediated code at the end of each sprint. A team that is not predictable isn’t “bad” – but they aren’t predictable.  Without stable predictable teams we can’t have stable predictable programs, particularly when there are multiple dependencies between teams.

This post focuses on metrics for predictability. The goal, then, is:

Teams can plan, coordinate, and deliver predictably enough to make a release level commitment.

Here’s how we break that down:

  • Does the team deliver the functionality it intended each sprint?
  • Has the team established a stable velocity?
  • Does the team frequently deliver working, tested, remediated code?
  • Does the team have everything expected each sprint to perform the work?
  • Does the team have confidence they will deliver the functionality expected for the release?

We answer these questions with the following metrics:

Story and Point Completion Ratio

  • Number of Committed Stories Delivered / Number of Committed Stories
  • Number of Committed Points Delivered / Number of Committed Points

This metric helps teams become predicable in their estimating and sprint planning. It encourages smaller stories and more effort getting work ready prior to the sprint. We like to see delivered points and stories to be within 10% of the commitment.

Velocity and Throughput Variation

  • Recent Velocity / Average Velocity
  • Recent Throughput / Average Throughput

This metric helps teams become stable in their performance. This will encourage managing risks and dependencies ahead of the sprints, and not over committing within the sprint. We like to see recent velocity be within 20% of average. We also want to see a reduction in the standard deviation of the velocity over time.

Lead Time

  • WIP to Throughput Ratio

Building a large inventory of untested code typically increases the costs and time associated with fixing defects. This in turn increases the costs and challenges associated with version control, dependency management, and the delivery of working, tested, remediated code. Our objective is to improve lead-time and to deliver frequently. There should not be more than 4 weeks worth of throughput active in a team from Ready to Delivered. Less is better. We like to see 2 weeks or less.

Team-member Availability Ratio

  • Headcount available / Headcount expected

We need an indication when planned team-members aren’t available. Stability is critical for teams to be able to make and keep release commitments. When people are pulled across multiple teams – or are not available as planned – it is unlikely that the team will be able to deliver predictably. We like to see this be within 10% of plan.

Release Confidence

Use the team’s insight and record of performance to evaluate the team’s confidence that the release objectives can be achieved. This metric is useful for planning and commitment purposes. Release Confidence is a consensus vote where 1 is no confidence and 5 is very confident. If a team has heavy dependencies, they should include a vote from the Agile Project Manager of the team handling the dependencies. If the team is missing a skill or if a role is unfilled, the team should take into account the likely impact to release success. Support this metric with a release burn-up.

Other Metrics

That’s just a taste of the metrics we use for predictability. We also use quality indicators like build frequency, broken builds, code coverage, defect rates or technical debt. Likewise, for Product Owners we are interested in things like major initiatives, features remaining, features released, size of release cycle, and more. And for value, we are interested in things like time to value.

Using metrics responsibly provides insight across the organization to understand the organization’s ability to meet expectations. These metrics help establish a shared understanding of the respective capabilities of the teams, and guidance for improvement efforts.

The post Agile Health Metrics for Predictability appeared first on LeadingAgile.

Leave a Reply

What is 10 + 1 ?
Please leave these two fields as-is:
Please do this simple sum so I know you are human:)

There are 101 ways to approach anything.
To find the best way, sometimes you need expert help

What People Say

“Kelly revolutionised the way our digital department operated. A true advocate of agile principles, he quickly improved internal communication within our teams and our internal clients by aligning our business and creating a much enhanced sense of transparency in the decisions the business was making. Kelly also introduced a higher sense of empowerment to the development teams...”

PETER SILVA-JANKOWSKI
IPC MEDIA

“Kelly’s a leading program director with the ability to take charge from day one and keep strong momentum at both a program and project level driving prioritisation, resourcing and budgeting agendas. Kelly operates with an easy-going style and possesses a strong facilitation skill set. From my 5 months experience working with Kelly, I would recommend Kelly to program manage large scale, complex, cross company change programs both from a business and IT perspective.”

LUKE SHARKEY /STRATEGY & IMPLEMENTATION LEADER
SUNCORP

“Kelly is an extremely talented and visionary leader. As such he manages to inspire all around him to achieve their best. He is passionate about agile and has a wealth of experience to bring to bear in this area. If you're 'lucky' he might even tell you all about his agile blog. Above all this, Kelly is great fun to work with. He is always relaxed and never gets stressed - and trust me, he had plenty of opportunity here! If you get the chance to work with Kelly, don't pass it up.”

GILES BENTLEY, DEVELOPMENT & OPERATIONS DIRECTOR
TIME INC

“Kelly is an Agile heavy-weight. He came in to assess my multi-million $ Agile development program which wasn’t delivering the right throughput. He interviewed most of the team and made some key recommendations that, when implemented, showed immediate results. I couldn’t ask for more than that except he’s a really nice guy as well.”

DAN PULHAM, DIGITAL DIRECTOR
TELSTRA

“Kelly and I worked together on a very large project trying to secure a new Insurer client. Kelly had fantastic commercial awareness as well as his technical expertise. Without him I would never had secured this client so I owe a lot to him. He is also a really great guy!”

GINA MILLARD
GLASS'S INFORMATION SERVICES

“Kelly came to the department and has really made a huge impact on how the department communicates, collaborates and generally gets things done. We were already developing in an agile way, but Kelly has brought us even more into alignment with agile and scrum best practices, being eager to share information and willing to work with us to change our processes rather than dictate how things must be done. He is highly knowledgable about agile development (as his active blog proves) but his blog won't show what a friendly and knowledgeable guy he is. I highly recommend Kelly to anyone looking for a CTO or a seminar on agile/scrum practices - you won't be disappointed!”

ANDY JEFFRIES/TECHNICAL LEAD
IPC MEDIA

“Kelly was a great colleague to work with - highly competent, trustworthy and generally a nice bloke.”

HANNAH JOYCE
GLASS'S INFORMATION SERVICES

“Kelly was engaged as a Program Director on a complex business and technology transformation program for Suncorp Commercial Insurance. Kelly drew on his key capabilities and depth of experience to bring together disparate parties in a harmonised way, ensuring the initiate and concept phases of the program were understood and well formulated. Excellent outcome in a very short time frame. ”

BRUCE WEIR/EGM
SUNCORP

“I worked with Kelly on many projects at IPC and I was always impressed with his approach to all of them, always ensuring the most commercially viable route was taken. He is great at managing relationships and it was always a pleasure working with him.”

BEATRIZ MONTOYA/CONSUMER MARKETING DIRECTOR
IPC MEDIA

“I worked with Kelly whilst at Thoughtworks and found him to be a most inspiring individual, his common-sense approach coupled with a deep understanding of Agile and business makes him an invaluable asset to any organisation. I can't recommend Kelly enough.”

PETER THATCHER, SENIOR ACCOUNT DIRECTOR
ThoughtWorks

“Kelly was a brilliant CTO and a great support to me in the time we worked together. I owe Kelly a great deal in terms of direction and how to get things done under sometimes difficult circumstances. Thanks Kelly.”

JULIE PEEL
GLASS'S INFORMATION SERVICES