In search of unambiguous terminology for Flow systems

Confused by terms like cycle time, throughput, throughput time, lead time, value-adding time, utilisation, flow efficiency, resource efficiency and delivery rate? You are not alone!

One of the fundamentals of agile processes like Kanban and Scrum is that they are about the flow of work. Where older project management approaches have tended to focus on the single batch (the project), processes influenced by the Lean movement (and Toyota's production philosophy) emphasise the flow of work within a project, and of multiple projects. So agilists need to be able to understand, capture data about and improve flow.

One problem in getting good communication between the many experts, teams and sources in this area is common and unambiguous terminology. Just one example from 2 books on my desk at present... how do you express Little's Law (i.e. the law explaining the influence of work in progress on flow):

1. Throughput time = (flow units in process) * (cycle time) 
2. Delivery Rate = (work in progress) / (lead time)

When you realise that some sources use cycle time and lead time interchangeably, meaning the time taken from one point of the process (usually the "start") to another (usually the "end"), you can see this is a big mess. Both equations are correct provided you understand the definition of the terms.

So here's my goal: I'd like to start using terms that, even if they are not universally used, they do not get used by authoritative sources to mean something completely different!

Immediately that means cycle time is out, because for some (those using equation 1 for example) it means the time between the completion of units (the reciprocal of throughput or delivery rate), and for others it means the time between points in the process.

Lead Time (rather than Throughput Time, another alternative) is the generally preferred term for the time between points in the process in the Kanban community, but this can also be problematic since the natural language use of this term refers to the time between placing an order (as a customer) and receiving the goods. This may include parts of the process which,the designer of a Kanban system has no control over (e.g. agent web sites, queues before entering the system, third party delivery systems, and so on).

We could redefine Lead Time to mean just the time in our system (which suffers from the "other authoritative sources" problem), or - better in my opinion - qualify the term whenever we use it in a formal context: for example System Lead Time; QA Lead Time; Development Lead Time. This means the term is unambiguous (provided the prefixes have been defined for the system) and it also means we can slice the system temporally to view the Little Law effects within subsets of the process.

Term 1 - Lead Time: the time taken for a unit of work to move from one point in the process to another. Used informally to mean either System Lead Time (i.e. the time for a unit to move from the start to end of the system under consideration) or Customer Lead Time (i.e. the time from customer order to customer delivery). Otherwise the term should be qualified to determine the start and end point of the measurement. (Measured in: days, hours, seconds, etc.)

Term 2 - Work In Progress: the number of units of work currently within a specified part of the system or the whole system. Prefer this term to flow units in process or similar.(Measured in: "units".)

Which term should be used for the rate at which units pass through the system or part of the system? Velocity (in Scrum), Delivery Rate and Throughput are all used frequently - probably Delivery Rate is more common in the Kanban community, though I have a slight preference for Throughput. It is only one word, and it applies equally to a subset of the system as to the final delivery part. [Post-publishing Note: The Kanban Leadership Retreat, June 2013 confirmed Delivery Rate as the preferred term.] So...

Term 3 - Delivery Rate: the rate at which units of work pass through the system or part of the system. As with Lead  Time the term may be qualified with a phrase that determines context - e.g. System Delivery Rate or Development Delivery Rate. (Measured in: "units" per day/hour/second.)

So we can now translate the two versions of Little's Law above to preferred terms:

1. System Lead Time = (Work In Progress) / (System Delivery Rate)
2. System Delivery Rate= (Work In Progress) / (System Lead Time)

("System" could be omitted in these equations if the context is clear.)

Finally let's define value-adding time and the 2 efficiency terms for good measure. I'm indebted to Modig and Åhlström's excellent "This is Lean" for these definitions.

Term 4 - Value-adding Time: the total time spent on value-adding activities for one unit of work. Value-adding activities exclude waiting and superfluous work .

Term 5 - Resource Efficiency: a measure of the utilisation of a given resource, i.e. the ratio between the time working on adding value in the system to the total time available.

Term 6 - Flow Efficiency: a measure of time-utilisation on a given unit of work, i.e. the ratio of the Value-adding Time to the (System) Lead Time.

Lean approaches emphasise Flow Efficiency over Resource Efficiency, since maximising Resource Efficiency, so often the first concern of accountants and managers, eventually leads to the traffic-jam state of no flow (see graph above). The efficiency paradox is that the most effective use of resources is achieved at a point where both Resource Efficiency and Flow Efficiency are less than one.

See also: What's the difference between Cycle Time and Lead Time... and why to just use Lead Time in Kanban.

Scaling Kanban: Scale-Free or by "Not Scaling"

Nearly all agile processes started life as processes for single teams. Turns out this was very useful in getting software development processes that were appropriate to the domain (unlike single iteration waterfalls). It has brought a transformation in efficiency for small and - perhaps surprisingly - large projects alike. But
dealing with large projects always was the real problem. So the current interest in more formal definition of scaled agile processes is timely if not overdue.

I'm interested particularly in how Kanban scales. I think there are two separate threads to this:

1. Scale-free Kanban
2. Scaling Kanban by not scaling it

Scaling by not scaling is the idea - only touched upon in chapter 13 of +David Anderson's Blue Book - that multiple Kanban systems can be linked and balanced in an analogous manner to service-oriented architectures. The corporate operations review is the key feedback mechanism for the balancing process. Rather than expand on this idea here, it's maybe the topic for another post. Suffice to say it's the potentially "self-balancing" nature of such joined systems that make it such a powerful idea.

Scale-free Kanban on the other hand uses the fact that the definition of Kanban does not specifically reference the size of the items flowing, or the size of the context in which they are flowing. Hence Kanban can apply at different levels (see separate blog post "Could Kanban be defined in a "Scale-free" manner?"). At least three scales of Kanban are already in use:

1. Portfolio Kanban
2. Blue Book Kanban
3. Personal Kanban

The standard or "reference" scale corresponds to the method as described in the Blue Book. It concerns a small to medium size project (1-4 teams perhaps), with items flowing through each stage of the process between 0.5 and 5 days. (I'm not trying to be precise btw!)

Small scale would correspond to Benson and Barry's Personal Kanban. A single worker, pair or small team working on tasks that usually take a day or less.

Large scale concerns the flow of projects, releases or major features - it has been labelled Portfolio Kanban by some, including Pawel Brodzinski who has brought a new focus on this area.

At least at these 3 scales the principles and practices of Kanban can apply without modification. Some may  require less stress (for example the reference to levels of the organisation is not particularly relevant at the small scale), but all are applicable.

What is even more striking is that we already know how to link these different levels - several Kanban tools even use this property of Kanban to support boards at different scales out of the box. 

The flow-items are different at each scale. Portfolio flow-items may be projects, releases or major features (an interesting separate discussion: what do/should businesses track at the portfolio level?). We might refer to these items for example as "funded work packages". The business level Kanban board has a collection of these moving from the proposal stage, through approval and development to released. The (dynamic) decomposition of such items into minimum-marketable-features, epics, stories, etc. eventually leads to the items that will flow across the project/team level boards. At this level the items are often referred to as stories, features, defects or backlog items. Blockers at this level can reflect upwards, as can forecasting based on flow/throughput, and eventually completion. Precisely the same kind of levelling can apply, if desired, to take project flow items (stories say) down to personal tasks tracked on a personal Kanban board.

An important aspect to note here is that concurrent use of these three levels depends on the flow-items at one level decomposing into smaller flow-items at the next level down. This is why this hierarchical approach is an alternative to the balanced service-oriented model of the not-scaling approach, where the flow-items are "related peers" (for example a blocker in one system spawns a backlog item in another).

While it's good news that these scales can be unified in principle, practices may already be diverging across the 3 levels - for example it seems to be popular practice in Portfolio Boards to reverse the direction of flow so that items that will be delivered later are shown to the right of the board rather than on the left (see Pawel Brodzinski's example here). However I believe it is still worth preserving the commonality of method definition and to embrace Kanban's consistency over multiple scales moving forward..

Selecting Backlog Items By Cost of Delay

Selecting the right items for an agile project to work on next is vital to ensure your team is delivering the maximum value possible to the business. Rather than "MoSCoW" (see recent post) or similar prioritisation schemes, +David Anderson's Kanban book recommends using the opportunity-cost of delay as the way to decide which items to pull next (or which items to put into the next sprint if you're using Scrum). He suggests 4 archetypes for categorising what the cost of delay might look like, when the impact (opportunity cost) is plotted against the time of release:

Expedite items have a high and immediate cost of delay. They should be started at once and given priority over other items. Fixed date items have a high cost of delay once a threshold is passed. They should be started before there is a significant risk this date will not be met. Standard items' cost of delay follows a fairly shallow S-curve, in other words, there is no reason to expedite them - they should be handled normally. Intangible items apparently have no immediate cost of delay, though it is expected to rise at some more distant point in the future. Such items are useful background tasks to be handled when there are no more urgent tasks.

At a recent Kanban Masterclass I attended, David introduced an example to demonstrate the difference between Fixed Date Items and Standard Items:

A hotel/resort site wishes to offer two special promotions: one for the Spring/Easter holiday; one for Valentine's Day. Because people tend to book much later for Valentine's Day, the opportunity-cost of delay is likely to be much "steeper" - more like the Fixed Date case, compared to a more "Standard" shape for the other promotion.

This led to consideration of what the revenue projections for different release dates might be in these 2 cases, so I undertook to produce some sample data for discussion. Here they are.

The graph above shows the imagined additional sales generated by launching the Easter promotion on the dates shown. There's no point in launching before Christmas and New Year are over (zero cost of delay), so the first release date is January 1st. There's an initial bump when the first ads go out, then a growth in revenue, and in this case a rapid drop off when it is anticipated all places will be sold. Later releases follow a similar pattern but after 5 or 6 weeks' delay the cost in lost sales becomes much higher as sales go to the competition or simply have insufficient time to close.

The graph above plots the loss of revenue of the early February release (5 weeks delay) compared to 1st January release. Sales are always behind - even after the slight recovery in the period after the January release has sold-out.

Taking all the potential release dates and plotting the opportunity-cost for each delay produces the graph above - an S-curve showing little impact initially, rising more rapidly after the first couple of weeks.

Let's compare that with the simulated data for the Valentine's promotion.

In this case we are expecting nearly all the sales to be made in the last couple of weeks before the event. It really doesn't help to launch early because people won't book early any way!

So plotting the cost of delay in this case we get a curve which is much closer to the archetypal step function of the Fixed Date items. If there are items with higher opportunity cost of delay early on in the year, it is more beneficial to do these items, provided we don't miss the late January release for the Valentine's promotion, i.e. the point where the curve hits the "hockey stick" upwards.

Hopefully these examples help the understanding of the concept of using cost of delay for deciding which item an agile team should do next. Let us know if you have any real data examples to demonstrate this concept. I'd be most interested to hear about them.

MoSCoW - is it a sensible way to prioritize requirements?

Back in the mists of time (some time in the last century) DSDM came up with a neat acronym for classifying the importance of requirements, MoSCoW - meaning a requirement was one of:

• Must-Have
• Should-Have
• Could-Have
• Won't Have (even if you Want it!)

The problem with this scheme has always been getting stakeholders to specify their requirements as anything other than Must-Have and Project Managers to resource their projects so that there's anything other than barely enough time for the mandatory requirements. Just like my old boarding school - if it wasn't compulsory it was forbidden. Many agile practitioners have concluded that schemes like this which give user stories a priority value or category, are not worth the effort - just get the product owner to identify the next most important stories and we'll prioritize the other requirements later. I sympathise with that view but on a previous project I needed a way to introduce the idea of a "scope range" for each release of the software, and to do that I redefined the meanings of the MoSCoW categories.

Firstly Must and Won't:

Must-have (this release): If this functionality is not available by the release date, the release will be delayed.
Won't-have (this release): If all the other functionality is complete before the release date, we'll release early rather than start this work.

Should-have and Could-have both mean they will be attempted for the release if there's time. The difference between them is whether our forecasting predicts that it is more or less likely that they will make it. Thus:

Should-have (this release): Our forecasting shows a greater than 50% probability that the functionality will be included.
Could-have (this release): Our forecasting shows a less than 50% probability that the functionality will be included.

I felt this was a stepping stone for the client, to move away from deterministic and towards probabilistic planning. The main problem with the approach was that I was re-using a scheme that already had a publicly available definition and so in a large organisation there was no guarantee people would be using the definitions I had introduced.

A better approach now is to look to schemes based on the cost of delay of requirements. This again emphasises the probabilistic nature of forecasting and uses flow data from the process to optimise costs.

"What's your favourite technique for estimating stories in Scrum?"

My number one favourite? Not estimating them at all! 

Instead, ensure stories do not exceed an agreed size that ensures they fit easily within a sprint (ok - I agree that's estimating, but it's much simpler estimating than tee-shirt sizes (S, M. L or/and too big) or points (1, 2, 3, 5, 8,  and too big). Then spend the time you've saved estimating the size of stories on estimating the number of stories in the various epics in your product backlog. Measure velocity in stories not points (or use only two sizes: 1 point and too big) and forecast the completion of epics and minimum-marketable-features based on the throughput of stories (velocity) and the number of stories. 

As well as saving a ton of time, it turns out your forecasting will be just as accurate as if you'd spent ages agonizing over each story! See for example +Vasco Duarte's blog about the evidence for improved forecasts from this approach.

The key question - as +Neil Killick points out in his recent "People Need Estimates" - is what problem are we trying to solve with our estimating? 

I was working with a team recently who were concerned that they had missed their forecast for the second sprint in a row. The retrospective meeting came up with several suggestions about how they could improve their forecast for the next sprint. I asked the team whether there was anyone (outside the team themselves) who was interested or concerned about their forecasting ability. The last few sprints were running at a throughput that was over double the average for the previous year. If the team continued to focus on the improvements they were making, the business could easily adjust to this higher efficiency. The business did not need (or probably believe) the over-optimistic forecasts that the team were making. Shown the evidence of actually completed work, the business will adjust to the new opportunities the improvements open up.

There are 6 general practices of Kanban... and I'm happy with that!

There are 6 general practices of Kanban, and in spite of my preference for groups of 3 (see "There are 3 ... Principles of Kanban"), I'm really happy with that. (I guess 7 would have a nice golden ring to it... but now I am being silly.)

The core practices are:

1. Visualize
2. Limit Work-in-Progress
3. Manage Flow
4. Make Process Policies Explicit
5. Implement feedback mechanisms
6. Improve Collaboratively, Evolve Experimentally (using models & the scientific method)

Nevertheless I've found it useful to start a team off by emphasising three of these (no really):

1. Visualize 
2. Limit Work-in-Progress
3. Improve

Before we can manage the flow we need to see it, and so getting a Kanban board with items moving across it really is the first step. Limiting work in progress is the essence of Lean and fundamental to all agile processes. Surprisingly to most teams, it nearly always brings an immediate impact in increased throughput and reduced lead time. And finally, as the foundational principles tell us, continuous improvement is the whole point. Let's start improving from day one.

There are 3, no make that 4... no really, shouldn't it be 3 Principles of Kanban?

+David Anderson first formulated the Foundational Principles of Kanban a few years ago:

1. Start with what you do now
2. Agree to pursue incremental, evolutionary change
3. Respect the current process, roles, responsibilities & titles

I like that. It makes clear that Kanban is about changing your process (it isn't a process) and also that it's about changing your process in an evolutionary way - step by step, with a viable generation after each significant change. There's no known destination to the ideal, forever-great process, that we can produce a plan for getting to. Unlike the old farmer who replied to a request for directions with "You can't get there from here!", we have to get there from here, so let's start here.

I also like it because there's three! So much easier to remember.

But wait there's more. Recently the principles have been updated and there are now four. I'm sure someone will tell me when and by whom it was added, but I confess I don't know. I just know the fourth one's arrived. Even Wikipedia (2013-04-16) knows it, so it must  be true! So here are the four:

1. Start with what you do now
2. Agree to pursue incremental, evolutionary change
3. Initially, respect current roles, responsibilities & job titles
4. Encourage acts of leadership at all levels from individual contributor to senior management

Yes number four's a good one. It should definitely be there. But do we really need four? Three is much easier to remember, quicker to share, and - when two of the principles say almost the same thing - three could be a potential improvement.

So here's my suggestion for when someone next updates the Principles of Kanban. That is, when they update the three foundational Principles of Kanban:

1. Start with what you do now (including the current roles, responsibilities & job titles)
2. Agree to pursue (incremental) evolutionary change
3. Encourage acts of leadership at all levels (from individual contributor to senior management)

If you want the snappy version, omit the phrases in parentheses.

See also "There are 6 core practices...".

Better Daily Planning Meetings: Asking the Right Questions

+Joe Vallone provides a very useful insight into the right emphasis for daily stand-ups. In his blog Better Daily Planning Meetings: Asking the Right Questions he suggests slightly modifying the well-known standard questions (what did you do yesterday? ... etc.) to:

1. What did you get done yesterday?
2. What will you get done today?
3. What is preventing you from getting work done?

This moves the emphasis from the activities to goals and helps a team focus on finishing work rather than simply doing it.

There's a similar purpose behind the standard Kanban practice of "walking the wall" from right to left, finding where tickets are stationary and seeking to unblock them. Flow only happens when things get done!