Measuring Progress and Performance in Large Agile Developments

Large agile programmes can suffer from the worst of all worlds - management managing with "on-time-on-scope-on-budget" in mind with a methodology which is designed to allow all three of these variables to change during the project. Having spent the past 18 months grappling with this problem in one of the largest agile projects in Europe, I'm off to Denmark for the GOTO Aarhus conference next week to share some of my conclusions with delegates - and hopefully learn from others' experiences.

The concept of measurement in software projects is not new. But nor, over decades of trying, has it it been hugely successful. While it is tempting to draw up very long lists of what can and should be measured to improve team performance, I am very much of the opinion that less is more, and that the place to start is the basics. Every project must measure cost; and if they look at the calendar they can measure time. Agile methods use story points to measure size, which although it is a relative metric, only providing an estimate to the story's size compared to others in the team's backlog, it is possible to build a picture across many teams of the backlog size for a whole programme. These 3 metrics of cost, time and size are the foundation metrics that all teams need. While Earned Value Management (EVM) is a traditional method for measuring the degree to which a team is following a plan, it can be modified for use with agile projects where the scope of the backlog is dynamic. It's potentially a very powerful tool for management to home in on problem areas and it's one of the techniques that I recommend for hybrid life-cycle projects.

The 3 metrics (cost, time, size) provide the basis for moving on to more interesting but much less measurable parameters such as quality, business value and productivity. These are the real quantities we'd like to know definitively. Unfortunately "interesting" maps to "elusive". As W. Edwards Deming said, "The most important figures that one needs for management are unknown or unknowable... successful management must nevertheless take account of them".