Midway through Sprint 01

Time to revisit our example Scrum Project started a couple of weeks back. How's our team doing? Here's the burndown as it was 2 weeks ago.

When we left it, the Sprint was red indicating an overcommitment based on current estimates. Either the team would need to negotiate a slight reduction in scope for the Sprint or risk disappointing the project stakeholders at the end of the period.

Here's the situation today (click for an enlarged view).

The status of the Sprint is now amber. Comparing the charts carefully one can tell from the reduced size of the work that there is a reduction in scope (also visible from the task lists) but there is good progress visible from the closed tasks to date. Other things to note in the screenshot are:

• Alerts (showing where some open tasks have exceeded their "best-case" estimates - they are prompts that the 3-point estimates for these tasks may need review);
  
• Closed tasks in the Explorer view (show with check mark); and
  
• The task list for the Sprint (sorted by end date so that the at-risk backlog item is highlighted at the top).

Participating in the project

The key thing about defining processes and project in xProcess is that the participants on a project can interact with the plans day by day, making them not just a view of what someone once hoped might happen, but an up to date record of the most likely outcome given progress so far.

The easiest way for project members to update their estimates, progress and plans is to use the web client of xProcess. By installing the xProcess server, this becomes accessible to team members using just a browser. They log in using their Subversion password and can then see their task assignments and update time bookings, which tasks are active, task estimates and descriptions, and other aspects of the plan relevant to their work. Here's a typical screen shot taken from our Scrum Project example (click on the image to enlarge).

Different filters can be used to show either just tasks scheduled for this week or all tasks assigned to the participant. Clicking on a task bring the task details in to the lower panel where they can be reviewed and edited. If there are documents or other artifacts associated with the task these can be seen and downloaded from this screen.

This article is part of a series of posts on Using the pre-defined Scrum Process. Check out the next article in the series here: Midway through Sprint 01.

Adding structure, artifacts and gateways to task patterns

Task patterns can be very simple (just one task with no extras) or a quite complex template controlled by several parameters and providing the implementer of the task with template documents, quality checklists (gateways), variable estimates and a family of subtasks. It's important to note that often simplest is best! Or at least (following Einstein) "as simple as possible but no simpler". We already discussed this issue in the blog with regards patterns for FDD (see Comparing big patterns and small ones). In Scrum, we've shown that different patterns can be used in addition to or instead of the Backlog Item pattern (see previous posting). Now let's look at what additional features might be useful to add to our Enhancement and Defect patterns.

Depending on the size of Scrum team you may have specialist roles that go beyond those of

• Product Owner
• ScrumMaster
• Team Member

For example the specialist role of Tester is useful on nearly all but very small projects. Developers must also write tests and test their code but Testers are generally more focused on end-to-end tests, user acceptance tests and full system tests. They may also have a role to play in accepting enhancements and defect-fixes. So for example we could identify subtasks in the Enhancement pattern with these different roles.

The alternative to multiple tasks, each requiring a single person with a single role, is to specify one task with multiple people, possibly with multiple roles. This would mean that each of these people would book time to the same task. Although this results in fewer tasks ,it's not necessarily simpler since the completion of the task is less well defined - basically the last one to finish closes the task. Experiment with what works best in your team - and let us know the outcome. We find such feedback invaluable.

In subsequent posts we'll look at:

• Adding gateways
• Artifact templates
• Using parameters

These are all ways to enhance the task patterns and make them easier for managers and team members to use and be productive.

Different patterns for Backlog Items

If you've followed through the series of articles on Scrum (see Using the pre-defined Scrum Process) you'll know they've covered making a Scrum project in xProcess and setting up a backlog of tasks that you can prioritise, forecast and display in Burndown or other charts. Participants on the project can access their tasks in the plan and update progress or modify estimates to complete them. But the pre-defined "Basic Scrum" process is just that - basic. It doesn't take into account the type of project you are doing with Scrum (it may not even be a software development project), nor the specifics of your company's or your team's process. With a little process engineering we can make things much more specific, and sometimes that's very useful.

The types of things you might want to consider once you're up and running with the basic process include:

• Adding different role types if there are areas of specialisation within the team
  
• Adding artifact templates for essential documents
• Adding gateways to tasks (quality checklists)
• Adding structure to the "Backlog Item" pattern, for example subtasks of a standard form
• Adding different types of Backlog Item with different roles, estimates, gateways, structure and so on.

For example you might want to distinguish between a defect and an enhancement and use a slightly different structure of subtasks and roles for each. If you want both patterns to be used as backlog items you need to modify the Scrum Project pattern so that they can be added by Project Managers / Scrum Masters to Sprints and the Unscheduled Backlog.

This diagram shows the hierarchy in the Scrum Project pattern. It consists of a project, named in the pattern as "$Name$" (this string will be substituted with the name the user supplies for the project). It has a General Overheads... task and a Scrum administration... task, both of which cover background work not related to backlog items. The Backlog Item pattern appears 3 times within the project pattern: in the first sprint, Sprint 01; within the Sprint pattern (which can be instantiated within the Sprints parent task as many times as required for Sprint 02, Sprint 03, etc.); and within the Unscheduled Backlog task. In order to add additional or alternative patterns, all 3 of these locations will need to be modified slightly so that the new patterns appear in place of, or additional to, the Backlog Item pattern.

Here's the situation after adding 2 new patterns for Defect and Enhancement as alternatives to the general Backlog Item. This will give the Project Manager a choice of the types of backlog item she wants to make in each case.

One word of confession here: to add extra pattern shortcuts like this means that the composite task that contains them (Sprint 01 for example) must be a selection composite rather than a collection composite. In version Basic Scrum 2.9.0 they are collections so you either need to change this or, to save hassle, use version 2.9.0a or later of the process (if this is not yet available for download, email support@xprocess.eu.com to get it.)

Next we'll look at adding structure, artifacts and gateways to our new patterns. Or check out the next article in the Using Scrum series: Participating in the Project.

Agile 42 at the Scrum Gathering

It was great to get together last week with old friends and colleagues Alex Aptus and Andrea Tomasini, who were over in London for the Scrum Gathering 2007. They are both doing Scrum consulting and training in Germany. Andrea's company is called Agile42 and if you think about it the name has some logic to it - "agile" is the undisputed answer to all the major questions of software engineering (mmm...?) in the same way that 42 is the answer to that only slightly larger chestnut: life, the universe and everything. Actually I'd recommend Alex and Andrea's work precisely because they understand that the answers to large questions are not trite or formulaic. If you manage to engage them for your project you'll not only get excellent training in standard methods but thoughtful and experienced consultants that will help you solve the hard questions too.

Another highlight of our meeting was seeing the newly opened St Pancras station on the day that the first Eurostar train arrived there from Paris. The queues at Europe's longest champagne bar meant that we weren't able to order their most expensive bottle at (literally) £2800 a pop!

By the way Andrea has a blog on Software Development Process in Practice. Why not check it out. He has a thoughtful article for example on requirements management. Here's the link: Software Development Process in Practice: Agile Product Development... How to manage Requirements? Effective requirements management has been shown time and again to be the most crucial aspect of successful projects so it's a worthy topic for discussion. I think it's an area that all of us involved with agile processes should be examining. My feeling is that agile methods stop too readily at the level of "features" without also trying to capture other aspects of requirements. 3 Types of Requirement tries to explain this - it summarises the results of some consulting in this area for a client adopting agile methods on very large projects.

Top-down or bottom up planning for Sprints

xProcess supports two ways to specify the required resources for a parent task: top-down or bottom-up.

The simpler approach is bottom-up. A parent task has subtasks with their own estimates of size and effort. The parent task size is therefore just the sum of subtasks' sizes.

What if you don't yet know what all the subtasks are? In this case - or if you want to specify some contingency or slack without inflating the estimates for the subtasks (Parkinson's Law would suggest this is a good idea) - then you can supply an independent estimate for the parent task by specifying it as top-down.

A previous entry, Parent Tasks and Top-down Planning, has already discussed this feature and shows the user-interface for using top-down estimates. The question here (since this is part of the series on Using the pre-defined Scrum Process) is how to use it for Sprints in Scrum.

Let's take a look at the current state of "Sprint 01" in our example Scrum project. Although this is quite a detailed screenshot, just look particularly at the red figures in the middle and the Alerts at the bottom of the screen (click on image to make larger).

The negative numbers for the estimates of size and effort for the "REST" of this task, and the Alerts, tell the same story. The backlog items in this sprint exceed the size of the top-down estimate. (See Prioritising the backlog for how and why these items were added to the Sprint.)

In this state the top-down estimate really is of no use since the larger amount of effort required for the child tasks will determine the time it takes and the top-down estimate of the parent task will effectively be ignored by the auto-scheduler. The auto-fix for the Alerts provides 2 ways to solve this situation:

You can either increase the estimates so that once again the top-down estimate is greater than (or equal to) the sum of the subtasks, or simply convert it to bottom-up so that the estimate for the Sprint just reflects the size of its parts.

So Sprints which already have a complete set of tasks added to them should usually be set to bottom up. However Sprints which do not yet have their backlog items defined should be top-down. That's why when you create Sprints for the first time they are top-down, and as a result you can see the forecast assignments for people and the projected cost of each Sprint of the project, based on its top-down estimate.

The general rule for how to decide whether to use top-down or bottom-up is this:

• if you think that you have captured all the work required in the subtasks, then use bottom-up
• if you think that having completed all the identified subtasks that there may be some others, then use top-down and provide an estimate for how large these other subtasks are in the top-down estimate.
  

These other subtasks are what is referred to in xProcess as the "REST" of a parent task, or the Remaining Essential Sub-Tasks. When you discover them you can create them in the parent task and task-by-task the size of the REST will be reduced, Or you may become confident as time progresses that you have indeed identified all the subtasks so you can then convert the parent task to bottom-up estimating and the REST will be removed.

One more point about parent tasks: team members cannot book their time directly to a parent task, only to a leaf-level task. So if they find that they are carrying out part of the parent task, but not one of the identified subtasks, they may need to create a subtask so they can book to it. This is another way in which the REST of the parent task may be reduced.

The next article on using the Scrum process in xProcess concerns configuring your process with Different Patterns for Backlog Items. Or if you're more interested in how participants on your project can use the xProcess web client, see Participating in the Project.

Prioritising the backlog

The key point about priority-driven planning approaches like Scrum - common in fact to all agile processes - is that the set of tasks undertaken by the team are ordered by priority rather than by a pre-determined procedure. (Priority is usually assessed based on business benefit and risk.) This article focuses on this particular aspect of using Scrum with xProcess and it covers:

• Adding backlog items to specific Sprints
• Showing a Sprint in Burndown and Gantt charts, and finally
  
• Prioritising the backlog

Having created a Scrum project with resources and a set of backlog items (see the previous post in this series) our next step is to prioritise the backlog. Our starting point here is a project which has three Sprints defined and a number of backlog items that have been estimated for approximate size. For simplicity in this example I've created 26 backlog items (with single character names from 'a' to 'z') with sizes of between 1 and 5 "points" (points being nominal units of size corresponding approximately to an ideal day's work for the average worker - see earlier post for more about this topic). Here's a view of my project at this point (click on the image to make it larger).

For the backlog items to be scheduled they must be moved from the "Unscheduled Backlog" task to one of the Sprints. Maybe you have an idea at this stage about the most important tasks, or maybe you want to see how long they would all take given the current estimates and resources. In either case just select the backlog items in the task list and drag-and-drop them to the first sprint ("Sprint 01"). As you'll see in the screenshot, I've opened the Project Explorer pane to do this. It's a view that can be selected from the "Window" menu - or simply click on the project icon next to the project name on the "Tools" pane. Once the backlog items are in a Sprint you'll begin to see the forecast start and end dates for them. You can now hit the "Priorities" button on the Tools pane and begin to put them in priority order. First though let's look at the Gantt chart for "Sprint 01" having added backlog items 'a' to 'v' to it. Double-click on "Sprint 01" to see its details and click on the "Gantt Chart" button. You can view other charts for the Sprint in a similar way, for example by clicking on the "Burndown Chart" button. In the screenshot below you'll also see that I've put the task list in reverse End Date order (by clicking on the column heading) and on the Gantt chart I've scrolled down to show the tasks at the end of the Sprint.

The Gantt Chart shows the forecast start and end dates of the tasks based on resources and estimates so far defined. As time progresses and tasks are started, re-estimated, completed or changed this chart will also change of course - it gives us a view of the likely outcomes based on information currently available. The risk element of the estimates is shown by the 75/95% extensions on the Gantt bars (see "Why I use 3 estimates..." for a discussion of this feature). Given that the target end date for this Sprint is December 19th, we can see that 2 items ('n' and 'a') are in the red zone and not forecast to finish in time and several others are in the amber zone and at risk of not finishing. This would therefore prompt review with the team and the client to whom commitments are being made so that expectations are set appropriately.

Now what if 'n' and 'a' are actually the most important items to get done in this Sprint, or what if there are dependencies between some of the tasks? How are these handled? Well dependencies can be added simply by dragging-and-dropping a task on the task it is dependent on. But it's important for flexibility to keep such dependencies to a minimum. The main way to order tasks in the Scrum process is by priority. To see and change priorities click on the "Priorities" button on the Tools pane. Here's what you might see...

The priorities of all the tasks in "Sprint 01" are all the same (they're all priority one!). As a result the scheduling of the tasks is effectively random. Let's prioritise then in alphabetical order. Here's what the priorities list looks like then...

Changing priority changes the order that the tasks are scheduled, so the Gantt chart now reflects the alphabetical priority ordering. Here's the resulting Gantt chart for "Sprint 01".

And here's the projected Burndown chart for the same Sprint. It still shows a red status as we've not removed tasks by prioritising, but it reflects when the newly ordered tasks are forecast to close. For more on how burndown charts are used as the Sprint progresses, see "Don't burn out - burn down!".

The next article in this series looks at the significance of Top-down or Bottom-up planning for Sprints.

Using the pre-defined Scrum process

I've recently been asked for more guidance about using the pre-defined Scrum process in version 2.9 of xProcess. It uses some of the new features for process definition (like Composite tasks) and also has "top-down" estimating for projects built-in, which is useful for resource planning and financial reporting. So I'm planning to post a series of blog entries on the process and in this one I'll focus on getting started with this process "out of the box". Here's what it covers:

• Creating a Scrum project
• Adding resources to the project
  
• Adding Sprints to the project (a Sprint in Scrum is like a "timebox" in other methods)
• Adding Backlog Items to the project (Backlog Items are the tasks the project must achieve)

So step one is to select the Project Manager perspective (the default) and hit the "New" button so we can make a Scrum project. The New project dialog shows the processes that are available for import, so if you've not already done so this is the point to import "Basic Scrum 2.9.0" and then select "Scrum Project" as your type of project. You can fill in the name and other details of the project (such as the number of 4-week Sprints and a location of a Wiki if you are using one) on this dialog.

At this point take a look at the set of tasks that are already in the project (hitting the "Tasks" button is the quickest way to do this. Here's what you might see (click on the image to make it full size).


There are:

• some overhead tasks (green because they have fixed start and end dates and are within the target dates of the project)
• A "Sprints" parent task containing one sprint "Sprint 01" (these tasks are red since they have NEVER end dates, clearly after the project target)
• An "Unscheduled Backlog" (which is grey as this task is specified as not to be scheduled, also resulting in NEVER end dates).

In the pane below the project details, you can also see there are some alerts displayed telling us that this project needs resources with particular role types. That's a clue to our next step - add resources. Hit the "Resources" button on the "Tools" pane and you'll see an "Add/New Project Resource" button. If you've already defined or imported a set of people you can select those who are going to be on the project, or you can create new persons at this point. Add six team members (Participant role) and one team leader (Scrum Master role) to the project.

At this point we should see actual end dates appearing for the Sprints and the overall project and we can even see a Gantt chart and projected Burndown chart. The Burndown chart is slightly more interesting if we add 2 more Sprints to the project (hit the "New" button and select "Sprint"). Then hit the "Burndown Chart" button on the project pane and you should see something like this:

Now it's important to note that as yet we've entered very little information about this project other than the number of Sprints and the people on the project and their availability. The projected burndown is only reflecting this simple data showing when the work in each Sprint (based on estimates for the amount of work in a Sprint which is modifiable in the process definition or, for a specific Sprint by modifying the estimates in its "Required Resources"). To use Scrum in earnest we need some actual "Backlog Items". Backlog Items are identifiable tasks that in a software development project for example could correspond to features to be built or defects to be corrected. The next step therefore is to add such items.

Hit the "New" button again and select "Backlog Item" (and then "Next"). We have a choice of the 3 Sprints we've defined or the "Unscheduled Backlog" as valid parent tasks for these items. Setting the parent task as Unscheduled Backlog means that we can create a set of tasks without them changing the scheduling of the project. We can then decide later in which Sprint specific tasks should be completed. So with Unscheduled Backlog as the parent task you can give the backlog iten a name and a size (size is measured in "ideal days" by the way), and hit "Finish" and the task will created. If you already have a list of tasks you can copy and paste a comma-separated list into the name field of the "New" dialog, check the "Create Multiple" check-box and several items will be created in one go.

Now we have a project some resources and set of tasks to be completed. The next step is prioritising. We'll look at that in the next posting in this series.

Searching the blog by topic

Blogspot offers some useful facilities for finding blog entries by topic. Below most entries there are a series of labels. Clicking on any of these will bring up other related articles on the same topic. Try it our with the labels below this post.

xProcess Europe

On November 6th Ivis announced that it had demerged its European operation (see Press Release) . The new company is known as xProcess Europe and it's owned by former members of the UK team including Andy Carmichael and Paul Kuzan both seminal thinkers in the evolution of xProcess.

Andy Carmichael: "Paul Kuzan and I are both very pleased to be involved with the on-going evolution of the xProcess product and brand, and we believe xProcess Europe provides a great opportunity, particular for developing the services offering that customers adopting xProcess are often looking for. Providing better support and expertise for companies wanting to improve the processes behind their project and portfolio management will be one of the main focuses of the new company. While our product sales and support work will primarily be targeting the European market, we will continue to offer process improvement services world-wide, wherever the needs exist."

xProcess Europe and Ivis Technologies are both supporters and contributors to this blog. From time to time you'll find links and references to both companies here. For more details try their web sites: www.ivis.com and www.xprocess.eu.com.