Improving projects

New Release for xProcess - feedback request

2012-01-20T15:18:00.002Z

We're planning a new full release of xProcess in the next few weeks, which will incorporate all the features released in the beta versions last year and several additional fixes and changes. If you have any feedback from previous versions which have not been shared through the forums, or if you are able to download and run the latest source code version and provide feedback, please do so as soon as possible as such feedback is invaluable prior to the release. Thanks to all our users - we really appreciate your feedback and help.
To download the latest version use this link: https://sourceforge.net/projects/xprocess/.

The Last Shall be First - Continuous Delivery

2012-01-05T14:44:00.001Z

One of the key lessons of agile processes is that the final step in your process - delivering software to your customers - is the first in importance. The best way to optimize the process is to work backwards compared to a traditional software lifecycle. When Dan Haywood and I laid out the outline of our "Better Software Faster" book, we decided to emphasis this by putting the chapter on build and deploy at the beginning - the first after the introduction. It's where agile teams should start in new projects or process transformations. Ensure that you can automatically build, deploy and deliver a tested change to your system, before you spend a great deal of time in building anything. If you don't do this you'll end up with a great deal of wasted effort further down the line, or worse still a waterfall process struggling to free itself from inappropriate practices and vaguely agile terminology!

The Seven Habits of Spectacularly Unsuccessful Executives

2012-01-03T14:02:00.000Z

Eric Jackson

Eric Jackson's article for Forbes under this title is illuminating. I'm afraid it put me in mind of some CEO's I have worked with (and I hope there are enough of those over my long career in the business to keep the libel lawyers at bay!). More than anything I think the tendency - not just in CEOs of course - to attribute success simply to one's own genius / hard work / insight / inspiration, and all failures to the work of others, is the destructive habit I would pick over all others. When circumstances and good work conspire together to provide outstanding rewards, one does well to recognise that it is not simply one's own talent that has produced it. When CEOs and management boards believe all the success was down to them, their companies are in serious danger!

Kanban vs Scrum - less shootout more engagement needed!

2011-10-31T10:59:00.001Z

David Anderson's book on Kanban provides a real insight into the practices and essentially the motivations for Kanban, particularly applied to process improvement. Since my reading happened to coincide with Ken Schwabers's dismissive posting about the method - saying that in contrast to the serious nature of Scrum it was a "nice distraction, for distracted people", it was helpful for me to look at all the examples, practices and theory in the book with as critical an eye as possible. I am a supporter of Scrum and believe it is the best starting point for team's starting out with a mandate to choose their process. However my recent clients have over 100 scrums working in parallel with a lot of other process surrounding them. They need methods for analyzing the waste in the processes and incrementally improving, based on sound metrics and evidence.

Kanban provides this kind of framework for improvement and I would strongly recommend such clients use it, as the basis for informed process development. As Henrik Kniberg points out in his companion volume to Anderson's, "Kanban and Scrum making the most of both", the methods are not incompatible. The fewer constraints that Kanban imposes on processes means that from a Scrum starting point, some of the constraints of scrum could be relaxed if benefit was deemed to flow from it. I presume this is why Schwaber objects to the method so strongly - if the starting point of the process is waterfall, one might end up with a slightly improved waterfall but not the major benefit that would flow from a true agile process. However if Kanban is followed effectively changes to a Scrum starting point would only be made if evidence was available for incremental improvements in the key business metrics. Yes there are risks involved in moving away from tried and tested agile processes. But that's where rewards lie too.

I'm not convinced by arguments between methods that focus on the character of those proposing them, when the substance of the methodologies are not examined. I expect more from the proponents in this case, more substance, evidence... and maybe a little more humility. Process improvement is not a one shot change from waterfall to agile. It requires insights from many different quarters to produce the best possible result for the business.

Why?

2011-10-13T16:31:00.000+01:00

If you've not already picked up Simon Sinek's "Why?" video it's worth a watch. Thanks to Jean Tabaka for highlighting and expanding this thought in her presentation at GOTO Aarhus. Inspiring.

Google announces new programming language for the web at GOTO Aarhus

2011-10-10T15:42:00.001+01:00

Gilad Bracha and Lars Bak, designers of Dart, Google's potential replacement for JavaScript, came on stage at GOTO Aarhus this morning both dressed as Darth Vader, proclaiming they had gone over to the Dart side. While replacing the lingua franca of the web is not going to happen any time soon (even in the opinion of these guys), the language does look interesting. It has optional strong typing claiming both the flexibility and ease of running of a language like Smalltalk, with type safety (to a point), modularity / team programming features and better code completion tools of languages like Java.

Interestingly the second presentation of the day was also from a Googler (Alex Russell) whose contention was that the problem for web developers was not the language (!) but the platform, with most of delays and impediments to powerful web programs coming from the need for network roundtrips.

Anyway, it's definitely worth taking a look at the early prototype release of Dart which is available here: http://www.dartlang.org/

Finding tasks in your Personal Planner

2011-10-06T18:36:00.000+01:00

If you can't see the tasks you expect in xProcess Personal Planner check the filters. If all the boxes are unchecked, all tasks assigned to you should show up. You can then reduce the list shown using the filters.

Another thing that can make tasks harder to manage is if you are assigned to several projects. In this case consider using just one xProcess project and moving the other tasks to parent tasks with this project. The auto-scheduler in xProcess handles each project separately so if you are assigned to several you have to divide your availability between the projects first. On the other hand if you have just one xProcess project you can control the order in which the various parts of the work are schedule using priorites (see "Driven by priority").

Measuring Progress and Performance in Large Agile Developments

2011-10-03T16:33:00.002+01:00

Large agile programmes can suffer from the worst of all worlds - management managing with "on-time-on-scope-on-budget" in mind with a methodology which is designed to allow all three of these variables to change during the project. Having spent the past 18 months grappling with this problem in one of the largest agile projects in Europe, I'm off to Denmark for the GOTO Aarhus conference next week to share some of my conclusions with delegates - and hopefully learn from others' experiences.

The concept of measurement in software projects is not new. But nor, over decades of trying, has it it been hugely successful. While it is tempting to draw up very long lists of what can and should be measured to improve team performance, I am very much of the opinion that less is more, and that the place to start is the basics. Every project must measure cost; and if they look at the calendar they can measure time. Agile methods use story points to measure size, which although it is a relative metric, only providing an estimate to the story's size compared to others in the team's backlog, it is possible to build a picture across many teams of the backlog size for a whole programme. These 3 metrics of cost, time and size are the foundation metrics that all teams need. While Earned Value Management (EVM) is a traditional method for measuring the degree to which a team is following a plan, it can be modified for use with agile projects where the scope of the backlog is dynamic. It's potentially a very powerful tool for management to home in on problem areas and it's one of the techniques that I recommend for hybrid life-cycle projects.

The 3 metrics (cost, time, size) provide the basis for moving on to more interesting but much less measurable parameters such as quality, business value and productivity. These are the real quantities we'd like to know definitively. Unfortunately "interesting" maps to "elusive". As W. Edwards Deming said, "The most important figures that one needs for management are unknown or unknowable... successful management must nevertheless take account of them".

The New Action Mechanism

2010-08-02T18:57:00.007+01:00

With the release of xProcess v3.5 we have implemented a new way of collecting data through Actions:
http://sourceforge.net/apps/mediawiki/xprocess/index.php?title=Reporting (the Recursive Solution). Existing actions can still be used but those that have a tendency to generate large amounts of data may need to be rewritten in order to get the benefit of the new mechanism.

The following example reworks the Work Log for Person action.

In which ever Process you want to, create a new Action called 'Get DailyRecords for Person'. This will return a set of daily records in a date range and is the FetchSet Action.

The first difference we see in v3.5 is that there is a new Editor for Actions. The Editor is split into three tabs:

Editor tab – a text area for the OGNL Expression

Details Tab:

Action Name - the name of the action
Applicable to - What it is applicable to (any element is the default)
UI Action - should it appear on context menu for element its applicable to?
On this tab are a further two tabs; Parameters: add the Parameters that will be presented to the user when the Action is run. These parameter values are sent into the OGNL expression when run. We will come onto the Export tab later.

Test Action Tab:

On the Test Action Tab you can choose an Element to run the Action on to test it and enter the parameters for that action. It is recommended that you create test Elements for this, as the Action may change that Elements data.
The lower part of this tab shows the output when the test it run

For the 'Get DailyRecords for Person' we just need the OGNL for retrieving the daily records, so in the Actions Editor we enter in:

#project = projects.isEmpty() ? null : projects.iterator().next(),#from
= (#project != null) ? #project.getDayProperty("fromDateForReport") :
null,#to = (#project != null) ?
#project.getDayProperty("toDateForReport") : null,
#dailyrecords = getWorkLog(#from,#to,null)

Hit Ctrl+S to save the Action.

The next Action to create is the 'Generate Work Log Details'. This Action is the Subordinate Action. You can have one or more Subordinate Actions, and these Actions are run against the data set that the FetchSet Action returns. The OGNL to enter in this Actions Editor is:

#output = '"' +
day + '","' +
assignment.task.project.label + '","' +
assignment.task.label + '","' +
assignment.requiredResource.roleType.label + '","' +
time/60.0 + '","' +
logEntry + '","' +
confirmed + '"',
#output = #output + '\n',
#output

Hit Ctrl+S to save the Action.

The final Action is the 'New Work Log for Person'. It's OGNL gets called before the FetchSet and Subordinate Actions, so we use that to provide the column header that will be written to file:

#project = projects.isEmpty() ? null : projects.iterator().next(),#from
= (#project != null) ? #project.getDayProperty("fromDateForReport") :
null,#to = (#project != null) ?
#project.getDayProperty("toDateForReport") : null,
#output = 'Work Log for: ' + label + '\n'
+ 'From: ' + ((#from==null) ? 'earliest booked time in data source'
: #andfrom) + ' To: ' + ((#to==null) ? #$today : #to) +'\n'
+ 'Date,Project,Task,Role,Hours,Log Entry,Confirmed\n',
#output

Then go to the Details tab and:

Make the Action applicable to Person
Click on the Export tab and tick the Export Action checkbox. Give it a .csv extension
On the FetchSet Field click on the '…' and select the ' Get DailyRecords for Person' Action
Now click on the Add button in the Subordinate section, and select the ' Generate Work Log Details' Action
Hit Ctrl+S to save the Action

We can now test the Action via File | Export | xProcess Export | Custom Reports. When the Action runs it uses the FetchSet Action to retrieve a set of Daily Records on Projects for the selected Person. This Daily Record set is then iterated over passing in the Daily Records to the Subordinate Action.

Since each time a Daily Record is iterated over a line is written out to file, it prevents the building up of large data to be written out to file, and possible memory issues.

Reporting in xProcess

2010-04-05T11:16:00.004+01:00

The mechanism for defining custom reports in xProcess is straightforward. It uses "Actions" (expressions which use OGNL as the language for navigating the xProcess model) and the string returned by the Action is exported to a file with whatever file extension is appropriate. There are several examples delivered with the Simple Process which are worth exploring and maybe modifying if you need a specific type of report for your project. See "How to define a custom report in v3" for more details.

Recently with users defining larger reports on projects that have been running for multiple years a problem has emerged with this mechanism - out-of-memory errors. As the text for the report is held in its entirety in memory, prior to any of it being written to file, it is clearly not too difficult to define an Action that will fail in this way if you have a reasonably large data source. To see and comment on the technical solution proposed for this problem, see the wiki for the open source project here: https://sourceforge.net/apps/mediawiki/xprocess/index.php?title=Core_Design. The new mechanism allows records to be written out one by one, but it will require reporting Actions to be rewritten to take advantage of the mechanism.

Watch this space for more details.

Update! See Tim's article on the new mechanism - http://xprocess.blogspot.com/2010/08/new-action-mechanism.html

Powered-by-xProcess Applications

2010-04-01T10:20:00.004+01:00

As discussions continue about opening up the xProcess world to other applications through the web services interface, it's worth considering the types of application this could enable. From an automatically scheduled to-do list on an i-Phone through to case-management support applications, the scope is of course boundless. But where are the quick wins? Most people who have seen the potential of xProcess, recognise that it's firstly in the context of supporting standard software engineering and project management processes such as Scrum, FDD, Prince and UP. The process definition gives teams access to standard patterns of tasks, if necessary with standard templates for associated artefacts and quality gateways. But processes are everywhere in the business landscape and if they are processes that need planning, resource allocation and forecasting, it may well be that the xProcess model provides just the foundation for powerful on-line support. Furthermore alternatives in this space are much less prevalent.

The web services interface means that full access to the underlying model will no longer be restricted to the Eclipse-based RCP client. Mash-ups which combine multiple sources of information can use the xProcess data source as the process and scheduling engine while application-specific views and data can be integrated seamlessly. If you've ideas for applications or would like to know how you can use the xProcess model for your project, do get in touch. The link to the Source Forge project is here: https://sourceforge.net/projects/xprocess/.

Cycle time and cashflow

2010-03-26T13:24:00.014Z

Cycle time is not merely an academic concern. In times like these it may involve your job security, your project's viability and your company's future.

Clarke Ching amusingly points this out in his latest biztech parable "Rocks into Gold" with a tale of a back-from-the-brink project that found a simple way to justify its existence and solve the cashflow problem. It a quick read that's well worth the effort.

Graph's like the one above also bring home the message that even a small reduction in cycle time - specifically the time to external release of new features of a product - can result in major improvements in economic performance measured by return on investment (ROI), time to pay back, and profitability.

Agile processes - influenced for example by Goldratt's Theory of Constraints (TOC) - stress the need for rapid turn around and frequent releases, and the key drivers for this are the financial benefits from early release. The inhibitors to frequent and early release of new features are usually lack of automated testing and incomplete continuous integration. Using the financial arguments, particularly improving ROI, is the best way to justify and assess investment in this area. Automating acceptance tests in particular involves considerable effort and expense, but if it results in reduced cycle times and improved quality the financial justification is clear.

Driven by priority - practical tips

2010-03-25T10:48:00.008Z

Agile processes are designed to support changing activities when the business environment changes. This after all is the natural understanding of the word agile - the ability to change direction quickly. What this means in practice is the tasks in plans - backlog items in a release say - have to be prioritised and changed when priorities change. This has always been a feature of xProcess, but in the very earliest releases the priorities list was a single list. We soon found when we had up to 1000 stories, features requests and defects to prioritise against each other it was an almost impossible task to get your brain round it all. This led to the current model in the product where groups of tasks can be defined and prioritised relative to one another, and then the groups themselves can be given a "weight" relative to other groups so that the overall priority list can be derived. This allowed us for example to prioritize bugs relative to one another and then decide on the balance between bug-fixing and new features in any given release. It made the whole process much more manageable.

The screen shot below shows a single "Priority Group" being prioritised, either by dragging and dropping tasks or by giving them a priority number.

Recently a user of the xProcess "Basic Scrum" process raised the issue that the relative priorities of user stories get lost if stories are moved from one Sprint to another (for example if they are incomplete at the end of the Sprint). Here's my reply:

xProcess has a very flexible mechanism for defining priorities using "Priority Groups" which allows any Folder or Parent Task which has been "prioritised" to support ordering of tasks within it. Each Priority Group can be given a "weight" so that all the priorities can be combined into the ordered list that the scheduler uses. As with many of the very flexible mechanisms in the product, when you actually tailor the product for a given process (Basic Scrum in this case), you have to make some decisions about the normal way to use the features.

Basic Scrum defines Sprints as prioritised Parent Tasks with diminishing weight (later Sprints therefore have lower weight and are scheduled after the earlier ones). As you've observed when you move a task from one Sprint to the next the priorities they had in the previous Sprint gets lost. So if this is a normal part of your process - I can understand why it could be - why not define another prioritised group (say a Folder called Release Backlog) where you can give your stories a longer lasting relative priority. The weight for this group does not need to be high because it will just be the "tie-breaker" when stories have the same priority in Sprint. I think this should achieve what your aiming for.

There are many different way to use Priority Groups, some of which have been incorporated into the standard processes released with xProcess. For example the Simple Process has the facility to add "High", "Medium" and "Low" categories to the project so that just by categorising tasks they get prioritised appropriately.

Accessing xProcess via Web Services

2010-03-19T10:04:00.006Z

Wouldn't it be great if you could build your own web interfaces which integrate project process and scheduling information from the xProcess model?

We thought so too. That's why we've started an inititative in the open source project to provide access to xProcess via Web Services. This will be a different server to the current xProcess web server (see screen shot) which uses Tapestry and has facilities for supporting participants on xProcess projects (booking hours to tasks, closing tasks with gateways, access artifacts and assigning tasks). There will be a considerable amount of common code however as the authentication, integration with Subversion, scheduling and multi-threading will all follow the same design. However the web services interface is intended to give full flexible access to the complete underlying data model of xProcess. The first services being defined allow you to access and update project data, but the mechanisms of data exchange and update are generic so almost anything is possible!

If you think this project might be of interest to your needs, or if you're interested in participating in it, please contact us. Your input will be most welcome. There are further details on the Wiki about how to use the source code once downloaded: http://sourceforge.net/apps/mediawiki/xprocess/index.php?title=WebServices

Precision and accuracy in estimating

2010-03-12T09:13:00.003Z

The difference between precision and accuracy has always been important to engineers and it is vital to understand too in the context of estimating for software projects. One approach to estimating a project is to create a detailed work breakdown structure and estimate each task, say in person-hours. A friend told me recently she had consulted on a project which had estimated its size as 135,213 person-hours! The precision in this case is 6 significant figures. As to the accuracy however, previous projects had errors or between 10 and 200%. The precision actually gets in the way of understanding the estimate, particularly as crucial business decisions involving significant proportions of a company's revenue may be based on such numbers. While three-point estimating (3pe) provides one way of capturing risk estimates along with the estimate themselves, the main contribution of agile methods in this area is the tight feedback loop from measured velocity timebox by timebox. Check out some of the links below to see how xProcess supports these concepts.

Naked Objects

2010-03-09T11:29:00.010Z

I recently received and re-read Dan Haywood's excellent "Domain-Driven Design Using Naked Objects" (I'd read a review copy some months ago). It's definitely worth the effort and I'm hoping to get some of the exercises done too as this means I can get up to speed with where the Naked Objects framework currently is these days.One of the first pieces of prototyping we did with xProcess (back in 2003) was to create a domain model (using TogetherSoft's "Together") and then connect it to Naked Objects so we could dynamically interact with those early models of Project, Task and Person. Naked Object effectively animates a domain model so you have an instant user interface to manipulate the domain objects. I believe the approach helped us then and I would certainly recommend projects consider doing the same - particularly in early stages of a project - as getting good domain models early pays off many times over at the end of the day.

CSV Import in xProcess

2010-02-19T12:46:00.004Z

We're currently working on new import mechanism in xProcess that is both more powerful and more flexible. It builds on the recently released facility to define custom columns in tables for tasks and other elements in the "rich-client" implementation. This allows you to display information about elements, say a calculated value based on standard or customised fields of a task. You can export values displayed in a table into a CSV (comma-separated values) file. The new facility will allow CSV files to be imported and the values in columns to be imported as properties of the elements imported. If you'd like to know more about this facility or be involved in beta testing the feature do let us know. You can contact the development team via the forum in the SourceForge project: http://sourceforge.net/projects/xprocess/develop.

Video on xProcess

2009-11-27T16:10:00.004Z

There is a new training video available for xProcess developed by Dr Vic Uzumeri of Interactive Point of View. Click on the link below to try it.

xProcess Tutorial

It may take a minute because of the file size, depending on connection speed. Also as it's embedded in a PDF you need a compatible version of Adobe Acrobat. We may get a more generic and faster playing web-based version later so please let me know if you have any problems with this link.

Training graduates in Software Engineering

2009-09-22T18:09:00.004+01:00

When gaps in the Blog occur it's a clue something rather distracting is going on! For me it was a long (10 week) assignment in North Carolina training software graduates in the induction programme of a major financial institution. Both challenging and fun, it was also an opportunity to think about the future of the industry in a challenging business environment. To me it is clear that business models are changing so rapidly - frequently totally overturning only recently established major successes - that new approaches to business and software process will sweep through many traditional firms. We taught both agile and iterative-incremental lifecycles (with more emphasis on the latter). However I expect agile to be the only option on such programmes in years to come.

Although I've had less opportunity to participate in xProcess development, we've seen healthy activity on the project over the summer and new interest from projects. For more details check out the SourceForge site here: http://sourceforge.net/projects/xprocess/.

Randomness

2009-05-20T14:55:00.006+01:00

It seems that nearly everything that we are formally "taught" turns out to be useless. Yet a few of the so-called "useless" things we do turn out to be invaluable - incredibly useful. And without the "useless" things we do, that do indeed turn out to be useless, we would never have found those incredibly useful ones!

I was struck by Nassim Taleb's story in The Black Swan about the inventor of the laser. Far from having eyesight correction, compact disks, fibre optics, data storage or microsurgery in mind when he invented it, he was just satisfying his desire to split light beams. As Taleb observed "We build toys. Some toys change the world."

The moral of the story to me is simple: learn play! Rather than seeing education as the process of acquiring knowledge, we should see it as the first steps in learning to think. The knowledge we acquired may be dangerous, particularly as it displaces the unacquired knowledge that may turn out to be more useful in a particular real-life situation. Where it is useful it can often be acquired when needed, provided our education gave us the means to learn and to inquire.

What is an "overhead" task?

2009-05-14T13:08:00.005+01:00

When starting to use xProcess there are a number of terms that may be unfamiliar. What for example is an "overhead" task?

In general terms an overhead task is one which does not directly result in the modification or qualification of a required project deliverable. Under this definition most management activities, meetings, planning and monitoring are overheads, while specifying, designing, coding, documenting and testing are "payload" activities.

In xProcess there are 2 ways of specifying how a task will be scheduled: date-based or effort based.

Date-based tasks are generally more applicable to overheads. They run from a date to a date (the date may be derived from other dates such as project start and end or target dates) and have a specified time (or percentage of available time) to determine the effort required per day. If there are no resources available to carry out the task the start and end date are not changed - they will just consume less time in the schedule.

Effort-based tasks are the more normal type of task in xProcess. They have a specified size and required effort (see this link for the difference between size and effort). Their start and end dates are derived from the effort required and the availability of suitable resources (people who have matching role types and skills).

You don't have to use overhead tasks in your process - for example you could include in your estimates the time that will be required for these other activities. However it is useful to get a view from the time booked to overheads of the efficiency of your process and where improvements might be made.

Nominate xProcess for the Source Forge Community Choice Awards

2009-05-07T16:58:00.007+01:00

Source Forge today opened nominations for its Community Choice Awards. Why not nominate xProcess now in any or all of these categories:

Vote now - it only takes a click!

Resourcing your Scrum Project

2009-05-07T10:22:00.008+01:00

In a previous article I explained how to set up a Scrum project in xProcess and the steps to go through to set up a Scrum project in xProcess:

Import the Scrum process into your data source.
Create a project (setting the various parameters in the New Project dialog)
Add project resources and give people the appropriate Scrum role (e.g. Product Owner, Scrum Master or Team Member)
Create the first Sprint
Create backlog items
Create a Delivery
Ensure Team Members update their tasks regularly.

Here I want to look in a bit more detail at adding project resources.

Clicking the Resources icon on the Project Toolbar (see image above) takes you to the resources tab for the project where you can add people to the project and also change their roles. There are 3 roles defined in the Basic Scrum process: Product Owner, Scrum Master and Team Member. Adding in people with these role types is the next step.

The tasks for Product Owner and Scrum Master are defined under "Administration and overheads" in the task hierarchy and they are date-based tasks. This means they run between specific dates (in this case the dates of the project) and are assumed to take a fixed amount or proportion of time each day. You can change these amounts by double-clicking on the task and adjusting the Required Resources. Some teams don't need to track time spent on Product Owner tasks and only wish to record work on stories and technical tasks in Sprints. If so you can can remove this task from the plan and leave the Product Owner out of the resources. It just depends what information you need, or might use later to improve your process.

Once you've defined your resources, here's what the "Available Resources" tab of the project might look like:
You can see the hours available for each person on the project and also the roles. If you need to change availability either at the company level, an individual level or for the project you can do this by opening the Availability tab of the organization or person, or by using the "Manage Project Availability for..." button on the project's "Available Resources" tab.

The Scrum configuration has one other field which uses the resource levels on the project. It's on the "Set up" tab which is found at the bottom of the project window. The field is called "Number in team member role" and it is used to size Sprints correctly when they are first created. You should enter the number of teams members (assuming they are all full time). If some are part-time, or some like the Scrum Master working on other tasks as well, remember to allow for the "Overheads level", which by default is 80%.

The next task then is to define Sprints. More later...

Using the Workflow Server

2009-05-06T15:58:00.004+01:00

A number of people have enquired about the xProcess Workflow Server recently and whether they can download it from SourceForge and run it. Though you can download and build the source code (see instructions on the Wiki to find out how to do this - and which projects here), we’ve not included the Workflow Server in the pre-built packages for download. Instead we are supplying it as part of a support package for clients in order to ensure that we can put all the components in place for any specific implementation of workflow.

The xProcess Workflow Server runs on Tomcat and is designed to respond to events that occur within the xProcess Data Source (detected by the DataSourceMonitor) or in external systems (detected by custom Monitors). Typical uses for the Workflow Server are

to integrate with other systems, such as bug tracking systems, that are running in your environment
to notify users by email of changes to plans or processes
to update external systems with changes that have occurred in xProcess.

There are some useful blog articles (referenced below) that help understand more about the Workflow Server:

The last article in the list discusses the use of the in-built STD for definition the actions triggered by internal changes (e.g. sending notifications when the state of tasks, targets or resources change). Only this mechanism allows workflow mechanisms to be changed without writing Java code. However even this mechanism requires a significant amount of set up, for example to include email servers and to define OGNL actions, which are triggered on state changes. This is why it is best to download the source code itself if you want to really unleash the full potential of this feature of xProcess. You can build the Workflow Server directly, set up email servers and other configurations, and integrate with your own code for external system monitors.

Also included in the source code are a number of examples of workflow usage, including and integration with the “Flyspray” bug tracking system. When running, this automatically creates tasks to correspond with confirmed bugs in Flyspray, and also updates the status of the issues when the xProcess tasks are closed for example.

Open Source Announcement for OpenXprocess

2009-05-01T00:55:00.006+01:00

This press release is hot off the press. You read it here first!

Date: 1st May, 2009
Location: SOUTHAMPTON, UK.

Ivis Technologies and xProcess Europe announced today that xProcess will now be available under an open source license, expanding its already substantial user base to a much wider community.

Also announced today was the setting up of a new company, OpenXprocess Ltd, to own, manage and support xProcess and supply training mentoring and maintenance services to users worldwide.

xProcess allows users to define project management processes (for example task patterns, artifacts, gateways and workflows) and then run live projects while monitoring targets, deliveries and process compliance. Agile methods for software development such as Scrum, FDD and variants of Unified Process are particularly well supported by the technology, which provides priority-driven ordering of activities and resource balancing. The free download of the product includes pre-configured processes for Scrum and other methods. Read more…

Scrum-xProcess: step by step

2009-04-02T13:12:00.009+01:00

I always tell people who are starting with xProcess to try the simplest things first, and in particular to try out "Simple Process" before embarking on processes which have more complex features configured, like the Scrum and FDD processes. This is just so you get the idea of what the scheduler does, how to reprioritise tasks and the effects of adding resources, manually assigning tasks, adding constraints (dependencies) and using the gantt and burndown charts. So it's a good idea to read The Simplest Possible Way to Get a Project Plan before reading this article because that takes you very quickly through those first steps.

So I'm assuming you've had a play with those first steps of setting up a project in xProcess and now you're ready to use a more configured process, in fact to use Basic Scrum 3. (Note processes can be easily changed in xProcess so it's very possible further changes have been made to this process by the time you read this article! The principles should hold though.)

These are the steps to go through to set up a Scrum project in xProcess:

Import the Scrum process into your data source.
Create a project (setting the various parameters in the New Project dialog)
Add project resources and give people the appropriate Scrum role (e.g. Product Owner, Scrum Master or Team Member)
Create the first Sprint
Create backlog items
Create a Delivery
Ensure Team Members update their tasks regularly.

The first two steps can be done separately or together. Firstly ensure you are in the Project Manager perspective (if you see either "Project Participant" or "Process Engineer" on the toolbar, click the icon and change to "Project Manager"). Then click on the "Projects" button on the Project Toolbar (on left usually) and then select "New". This brings up the dialog shown at the top of this page. Here you can select the Basic Scrum process and import it. Then at the top of the same dialog select this process and hit next. This brings up the dialog shown here.

More to come...

Getting started with Scrum-xProcess

2009-04-02T12:48:00.003+01:00

There are several useful articles on Scrum-xProcess in the blog and if you're starting using the process it's worth having a browse of these to guide your initial set-up.

Using the pre-defined Scrum process - this is the first of a series of articles on using Scrum-xProcess. You can follow them sequentially from this article.
Don't burn out... Burn Down! - discussion of burndown charts.
Calculating team velocity - discussion of velocity and productivity.
Rescheduling tasks to the date you want - what to do when then auto-scheduler doesn't schedule a task when you want it!

I'm going to start a new article now to take you through step by step of setting up a Scrum process. Here's the link to that one... Scrum-xProcess: step by step.

Opening the source...

2009-04-01T16:56:00.003+01:00

The source code of xProcess is finally being released today after several months discussion and negotiation concerning the ownership and licensing terms for the code. We settled on the GNU Affero General Public License and having dotted all the i's and crossed all the t's in the legal documents, it's finally out there. Enjoy!

In search of a definition of uncertainty in three point estimates

2009-03-26T09:55:00.012Z

I've written elsewhere in this blog about the value and theory of three-point estimating (3PE). See "3PE - why I use three estimates where one might do!". The main practical snag with using 3PE in an agile context is the additional overhead of thinking of 3 numbers every time you want to confirm "effort to complete" - ideally something people can quickly update even on a daily basis.

So I'm working on a mechanism which shows the level of uncertainty in an estimate to complete which takes into account the effort completed to date (T) and the three points of the estimate: best case (b), most likely (m) and worst case (w). If you're interested in the intricacies of such things, please read on!

The composite estimate (E) is the estimate is the derived "median" which is used when one number is required (as for example in the "Total" field in the screenshot above). It is derived from best case (b), most likely (m) and worst case (w) as follows based on assumptions about distribution of cases between best and worst:

E = (b + 4m + w) / 6

Various formulae are possible for our level of uncertainty or estimate risk (R). Here's a starting point expressed in terms of just b and w as follows:

R = (w - b) / (w + b - 2T) ..................................[1]

This expresses the average "error", (w-b)/2, as a proportion of the average time to complete, (w+b-2T)/2. However since it ignores the most likely estimate, it doesn't take into account that for example the worst case may be much further away from the most likely than the most likely is from the best case. An alternative formula would therefore be:

R = (w - m) / (m -T) ........................................[2]

But this formula ignores the best case completely. One could say the "error" should be defined as whichever is greater out of (w-m) or (m-b), but actually it's not this aspect that worries me practically. The worst case is always more significant from a forecasting viewpoint, and from an estimating viewpoint it is the one that can be much further in error than the best case, which can never go lower than T and in practice will always be a little bit higher than T (or you can forget about estimating and just finish it!).

So a better approach is to define the "error" in the formula for R relative to the median estimate, E, all three points are then taken into account. It's much more satisfactory.

Following this approach, here's the formula for the uncertainty in the estimates (estimate risk) that I'll be using:

R = (w - E) / (E - T) ........................................[3]

For those interested in the mechanisms within xProcess to use this formula, there'll be a discussion on that project's wiki about how users can set estimate to complete and level of uncertainty rather than worrying about 3pe every time they update time to complete.

Inaugural meeting of the Southern UK Scrum User Group

2009-03-19T09:40:00.006Z

Several scrum enthusiasts in the South of England have decided that, rather than spending a few hours on the train to get to the London Scrum meeting, it would be a good idea to get together more locally to talk about Scrum. The inaugural meeting was last night at the Inn on the Furlong in Ringwood (easy reach of Southampton, Bournemouth, Winchester, Salisbury and so on). Apart from some excellent discussions about people's experiences with Scrum we were able to savour the very fine selection of real ales from the Ringwood brewery. If you're in the South of England and interested in joining us next time check out the group's forum discussions on LinkedIn here.

How to define a custom report in v3

2009-03-17T18:49:00.017Z

There are 2 ways to generate a report on a project. One is to right-click on a project and select "Reports", and the other is to select from the File menu: File -> Menu -> Custom Reports and then select the type of report you want and its subject. For example using File->Export we can select a "Work Log" report for a person or a task.

What if the report we want isn't in the list? Can you write your own report template? Yes you can - that's what this article is about.

Let's say I want a report on a task in the project. I need to define an "Action" in the process I'm using, identify that action as an "Export Action" (checkbox in the Action dialog) and define the file extension for the report. Let's say in this case we want an html file. You can see in the screenshot this action being defined. Note that the "Applicable to:" field has been filled in as "Task" and also the "Expression:" field has been given the value "name". This won't be a terribly fascinating report but even so let's run it!

Use the File-> Export-> Custom Reports menu and select one of the tasks in your project as the subject of the report. Once you've supplied a filename and pressed "Finish" you should find that a browser is launched and a page generated containing the name of your task. Congratulations you've generated your first xProcess custom report!

So what about doing someting a bit more interesting. As you've probably guessed by now that will mean looking in a bit more detail at the "Expression" field. The script that goes in this field must be written in OGNL so you need to know a bit about that language - if you'd like to read the (quite small) reference manual click here - and a little bit about the xProcess Java API. (You'll also find the OGNL reference manual in the xProcess Help documentation.)

So let's try a slightly larger bit of OGNL to generate our task report. How about this:

'<h1>' + name + '</h1>' + '<br>' +
description + '<br>' +
'Start: ' + start + '<br>' +
'End: ' + end50 + '<br>' +
'Closed?: ' + closed

You could even turn this into a report on all tasks in your project. First change the "Applicable to:" field to "Project" (this will also mean you can invoke it by right-clicking on your project) and turn the above code into a subroutine that can be called for every task. Like this...

#taskReport = :[
#output = #output +
'<h1>' + name + '</h1>' + '<br>' +
description + '<br>' +
'Start: ' + start + '<br>' +
'End: ' + end50 + '<br>' +
'Closed?: ' + closed
],
#output = '',
allTasks.{#taskReport(#this)},
#output

Browse other Actions in the predefined processes to see other examples of using OGNL. Then if you make a report you think others will find useful - or if you need help with syntax or the API - post it to one of the xProcess Forums on SourceForge. See xProcessForums.

Define the process behind your projects

2009-03-11T10:18:00.004Z

To get started with xProcess it's sensible to start with a pre-defined process - the Simple Process for example (see "The Simplest Possible Way to Get a Project Plan?").

However defining your own process in xProcess needn't be complicated. First thing is to switch to the Process Engineer perspective via the toolbar across the top of the screen (from the default Project Manager perspective). The explorer view in this perpective shows you the processes that have already been imported into your data source and you can create a new process simply by clicking on the process icon in the task bar on the left hand side.

Here are some of the other things you can create in a process:

Project pattern(s)
Task patterns
Role Types
Category Types
Gateway Types
Workflow Packages
Artifacts and Artifact Types

Start by looking at some of the Task Patterns in pre-defined processes like Basic Scrum and Basic FDD which come included in the download. You can see the structure of the task and project patterns by using the hierarchy or process diagram. When you define a parent task in a pattern you can define what kind of child tasks can be created in the parent by right-clicking on the task and selecting New -> More Patterns... This makes it easier for Project Managers to build sets of tasks that match the process you are defining.

There are many other aspects of Process Engineering with xProcess to discover. If you are interested in taking the on-line course for the Certified xProcess Engineer qualification, contact us now through the xProcess site on SourceForge.

Theory of Constraints and Agile Project Management

2009-03-03T21:05:00.008Z

Recently on the LinkedIn forum PM Toolbox, Arash Sadati asked for comments about tool support for how Goldratt's Theory of Constraints (TOC) or Critical Chain Project Management can be integrated as part of the Agile Project Management. Here are my thoughts on the subject.

One thing all agile projects share is that - because of a conscious removal of dependencies between features wherever possible - they will be resource constrained rather than critical path constrained (a generalisation but broadly valid). Methods like Scrum-XP try to avoid specific role constraints (e.g. we can't make progress because we don't have a BA or a GUI expert or a middleware expert available) by specifying only 3 role types (Product Owner, Scrum Master, Team Member). So the team as a whole must to some extent be generalists (or at least prepared to learn / fill in specific roles) to reduce the risk of role constraints. In any real application of Scrum of course such role constraints may be real and teams will have to pay attention to them. The default condition though is worth understanding because it's a means to keep the effect of role constraints to a minimum.

FDD defines many more roles and I think as a result there is a greater risk of role constraints. Availability of specific class owners for example is a typical problem which Palmer and Felsing discuss in their book on FDD. Flexibility, especially in small teams, for people to wear multiple role "hats" is crucial. The really interesting observation in comparing agile to waterfall is the requirements process where detailed specification of features is ideally "just in time". David Anderson and others point out that it is a serious waste of precious resources to allow too much work to be done on future requirements when the critical constraint is the development team.

When we were specifying the xProcess product which is designed to support any priority-driven process (any agile process in other words), we did want to be able to give visibility to problems arising from role or resource constraints. Patterns of tasks can be defined in custom processes that require specific or general roles - or even allocation to specific team members - so that forecasting can take into account not only the total development effort available, but in the cases where specialised roles are required. It can therefore show the impact of limited availability of critical roles. Custom reports can be generated to show resource utilisation, and where these show less than 100% utilisation, the situation is likely to be one where specific roles are "critical", requiring other roles to wait for them.

The Simplest Possible Way to Get a Project Plan?

2009-02-18T11:47:00.009Z

I want some work done - so make a new project...

Ok. Here are the tasks that need doing (I can enter them as a comma separated list).

My task list forecasts that these tasks are never going to complete. I guess that's because I don't have anyone to work on them. So add myself as a resource to the project.

Good. They're now being scheduled but I don't think the most important things are being done first. I need to prioritise...

That's fine. They're in order and I can create a Timebox to show what will be done in the first two weeks. Let's have a look at the forecast Gantt and Burndown charts...

Seems ok but all the tasks have the default estimate of 2 days. Next job is to estimate those tasks properly. Oh and I also need to add in the other project resources, and when they're going to be available to the project. Once that's done I'll have a plan I can show my manager.

Mmmm... That didn't take too long!

FDD sub-processes in context

2009-02-13T10:55:00.005Z

Like most agile processes FDD doesn't pretend to cover every aspect of the software development life-cycle. Just the essentials for establishing a common approach. In particular FDD doesn't explicitly state how and when releases of software occur during or following an FDD iteration.

Here's a diagram that attempts to put the releases in context and also shows the distinction between the first 3 sub-processes which define the project (or iteration), and the multiple instances of the 4th and 5th sub-processes which develop the project (iteration). There can be just one release at the end of an FDD iteration or (more normally?) several releases which deploy the developed software.

What's different about Folders?

2009-02-04T10:12:00.004Z

When you're deciding on the structure of project patterns for a new process, often the first thing to consider is how to represent the different hierarchies of elements that make up projects.

In FDD for example there is the hierarchy of features, normally described as being Features, which are contained in Feature Sets, which are contained in Business Areas or Major Feature Sets. This logical grouping of functionality is independent of priority and time-ordering. It may well correspond to the structure of the user manual for example or the Functional Specification should such a document be required. It doesn't give us any time view. On the other hand the hierarchy from Releases -> containing Timeboxes -> containing Features is a time-based structure. Similarly the hierarchy based on the five subprocesses of FDD also will have a time correspondence: The Definition Stage -> containing FDD#1 Develop Overall Model, FDD#2 Build Feature List, and FDD#3 Plan by Feature; The Build Stage -> containing many instances (for each feature) of FDD#4 Design by Feature and FDD#5 Build by Feature; and the Release/Deployment Stage following completion of a release.

Each of these hierachies is an important view of the tasks and artifacts in the project. However when defining a process in xProcess you must decide which hierarchy will define where a task "lives". This is the parent-child hierarchy of tasks and is analogous to a directory structure which tells you where a file is located. All the other relevant hierachies can be represented with Folders which contain explicit or implicit (defined by rule) references to the tasks. This means that different features may appear in different Timeboxes and Releases, and indeed different subtasks of a feature may appear in different folders (design and build for example may appear in different timeboxes; build and certify a feature may appear in different stages). Since Folders may contain Folders in xProcess as many hierarchies can be provided as are required to understand the process, each to several containment layers if this helps understand the view.

So Folders provide an alternative hierarchy of tasks to the main parent-child structure. Their membership is defined either explicitly (because the process has put them there or the user has dragged them into the Folder) or implicitly (because the task has a matching category to the category of the Folder). In some methods it is sensible to use the parent-child hierarchy for time-based structure (Basic Scrum 3.0.1 follows this scheme for example). In others using a more static structure such as Business Areas/Feature Sets in FDD or a Work Breakdown Structure in traditional methods, gives the preferred structure for parents-children with more dynamic hierachies being represented with Folders.

Description of the xProcess product

2009-01-30T16:55:00.003Z

xProcess allows projects to define their own processes... and make and monitor plans that conform to those processes.

Projects may start from predefined processes such as those for Scrum, FDD, Iconix, Unified Process or Prince 2, or define their own. Using their process definition, projects can be defined, resources added and tasks defined, estimated and prioritised. xProcess then provides forecasts for the project against which targets can be set.

Once a project is started the participants on the project can access the tasks they have been assigned and record hours worked against tasks, adjust estimates of time to complete, add artifacts (such as documents or links to managed files) and close tasks, including (if the process requires it) completing task "gateways" to show compliance with intermediate steps of the task.

Thus xProcess supports process definition (including several graphical views of the process patterns), project management (including defining and estimating tasks, forecasting, timeboxing, target definition and monitoring, gantt, pert and burndown charts) and project execution.

The main client for accessing xProcess information is an Eclipse RCP application. There is also a web server which provides browser access for project participants. The RCP client has 3 perspectives for: Process Engineers (defining processes); Project Managers (defining and monitoring projects); Participants (reviewing and recoding their work on the project).

xProcess project on Source Forge

2009-01-30T16:46:00.002Z

With various announcements, activities and releases in the pipeline, you may be interested to know that there is now an xProcess projects on SourceForge. The site is http://xprocess.sourceforge.net and althought as of this moment there are no downloads available there, the forums, announcements, wikis, etc. of the project will all be moving up there soon. Why not post a message there now?!

Calling Software Development Process experts!

2009-01-08T17:20:00.004Z

Did you know that from January 2009 the xProcess project is going open source?

This means that projects worldwide will be able to download and use free process definition and project management software and, if they are so inclined, configure and improve it for their own use. At the same time we are launching a new services company OpenXprocess. This company will incorporate xProcess Europe (and xProcess US), and manage the ongoing enhancement of the product, while offering services to users of xProcess such as: training in Scrum, FDD, Agile, Iconix Process, Unified Process, Prince and other configured processes of xProcess; configuration of the product to companies’ own development processes; and a support package which includes the web server for access to project data from browsers.

This is where the software process experts come in. Would you be interested in learning about xProcess and potentially offering services to configure processes or consult with clients using existing configurations? The Scrum configuration in xProcess for example provides a number of unique features which are not available in other Scrum tools (task matching, forecasting, three-point estimating, target management, multi-dimensional prioritisation) as well as familiar Scrum tools such as burndown charts (including burndown forecasts), Gantt charts (auto-generated) and backlog management. If you would like to be involved I’d very much like to hear from you.

The process configuration capabilities of xProcess, particularly now it’s an open source product, offer important innovations for the industry, and I’m particularly keen to involve experienced software consultants in this area. In the past process definitions have been of only passing interest to project managers, useful guidance but completely separate from the plans they work on day by day. xProcess changes this relationship since it is now possible to define processes that instantiate directly into project plans and aid conformance with the process (or highlight differences) as plans are implemented.

Project processes are often described with Activity diagrams or similar notations. But these are notoriously difficult to map to the actual activities of a project. With xProcess the process is described in terms of task patterns which, when instantiated within the initial project pattern, show the trackable tasks that are required to complete that part of the process. Where aspects of the process involve activities that are below the level of trackable tasks – for example a development task may involve say, coding, testing, integrating and building – these steps may also be captured in the process definition through xProcess’s Gateways, which are defined in task patterns and are completed as questions and answers on closing a trackable task. And because task patterns may themselves contain task patterns the process definition language gives as much flexibility to define optional, repeating or iterating structures as activity diagrams do, but with the important difference that they map directly into the plan structure that projects implement.

If you’ve read this far I hope that means I’ve at least aroused your curiosity to investigate further. You can download the product from www.ivis.com (soon to be www.openxprocess.com) and I very much hope you’ll do this and encourage colleagues and clients to do likewise. But you might also be interested in getting directly involved. In Q1 2009 I’ll be conducting some training workshops for potential xProcess consultants or process engineers (probably in Denver and London). Please let me know if this might be of interest. The training will last 2 days with the opportunity for a further 3 days collaborative work on a process of your choosing – this can be carried out either immediately following the training event or remotely at your convenience, with access to me via phone, skype or teleconference. Following completion of this we will offer certification as “Certified xProcess Consultant” so that you can offer services to clients directly or through OpenXprocess. OpenXprocess will only use certified personnel for training and consulting services and will support certified consultants in the field to help ensure your success.

If you know of others who may be interested in this, or companies who you feel might want to adopt xProcess, please let me know.

Wishing you all the best for 2009.

Andy Carmichael.

Folders are changing

2008-12-04T15:48:00.003Z

There's a new spec for Folders in xProcess that should be released shortly in the v3 programme. The new feature let's you define a hierarchy of Folders and it clarifies the difference between folders with "explicit" members (the tasks are in the folder because someone put them there) and "implicit" members (the tasks are there because they fulfil the criteria for the folder - usually that they are in the appropriate category.

Email me or post a comment below if you would like a preview of this spec.

Rescheduling tasks to the date you want

2008-11-01T13:23:00.012Z

I was asked by someone recently to help them get their task to be scheduled on the date they wanted. One thing xProcess doesn't give you (except for overhead tasks whose size is defined by their duration rather than the estimated effort) is the ability to state when a task should start and stop. This is what the auto-scheduler does, based on your estimates, constraints (e.g. dependencies on other tasks or specific dates), your priorities and your available resources. So what if you have a task which is not being scheduled when you think it should?

Here's the checklist I gave my colleague:

If the task being scheduled too soon you can add a constraint to the task by opening its editor (double-click) and selecting the “Constraints” where you can add the constraint (e.g. a dependency on another task or a “start after < date >” constraint).
It may be though that the task is being scheduled too late and you want it scheduled sooner. A quick and easy way to achieve this is to make it “active” as then it will be scheduled immediately. (Just right-click on the task and select “Make active”. This can also be done from the web client too and it’s worth checking that the team always make the tasks they are working on active so that they are always scheduled immediately and not postponed when higher priority tasks are defined unless they make them inactive.)
If it’s not currently being worked on (so not “active”), but you want it scheduled earlier (for example as soon as current active tasks are complete) then increase it's priority. Here's how:

If the task is not in the current sprint (assuming you're using the Scrum process) it will be scheduled after this sprint, so drag it into the current sprint from its current location
If it is in the current sprint but still not being scheduled soon enough, use the “Priorities” button on the tool bar, which takes you to the “priority groups” tab for the project. Maximize the window so you can see it all and select the current sprint. In the lower list you can see the relative priorities of all the tasks in the sprint. Change them so your task is near the top.
Check in the first tab of this window “All tasks in Schedule Order” that your task is high enough. (You will see for example all the active tasks near the top of this list.)

Finally one more thing to check. Make sure the task doesn't have constraints on it's start or end, for example a dependency on another task that won't complete for some time. This will also cause it to be scheduled later.

IT&T Conference Galway

2008-10-27T11:40:00.008Z

Ireland seems to be a second home for me at present with a number of clients using my services for migrating to agile practices through partners Neueda Technologies. So it was a pleasure to be back in sunny Galway (in the gaps between the gales!) for this conference at GMIT. My tutorial was on "Agility: driving software projects for value and quality" and apart from giving an overview of agile methods, particularly Scrum, it looked at some of the key issues for scaling agile processes including design, continuous integration and team structures.

Project Home Page

2008-10-20T13:23:00.014+01:00

xProcess allows you to define a wiki, blog or other site as your project's home page. If you can see this page from with your project in xProcess you are pointed at the default page. Why not change this to a site where you can add details of your own project and invite team members to contribute.

Some processes are configured to expect a Wiki site rather than a blog. These sites generally allow pages which do not yet exist to be referenced by url. In such processes a reference to a new page is generated when a new task or project is created, allowing the user to create the corresponding page, at the same time as and already linked to the task or project.

New Scrum Process for v3

2008-10-09T11:45:00.006+01:00

With the release of xProcess v3 drawing closer here's an overview of some of the improvements that are in the pipeline enabled by new features in the product. In this article we look at the steps to create Scrum projects and the backlog, then to prioritize them and allocate them to Sprints. In subsequent articles we'll consider how you can use the same features used in the Scrum process to tailor development processes for your organisation.

Here's the dialog for a new Scrum project. It shows the parameters used to set up the project including the location where Wiki pages may be created (including a home page for the project) and the Sprint duration which is used when Sprints (the timeboxes of the Scrum process) are created.

As the next screenshot shows there are now more patterns to choose from when you hit the "New" button in the project toolbar. We'll be looking at exactly what all these patterns are for in subsequent articles. For now let's just look at the "backlog items". The patterns for these are User Story, Technical Task, Defect Fix and Delivery. In fact you can see just this menu of items if you right-click on the "Backlog..." task (or a Sprint if you've created one) and select "New" there.

Defects kill Sprint Planning

2008-09-16T14:40:00.003+01:00

Burning down user stories sprint by sprint at a nice constant velocity is what we hope to be seeing on agile projects. What happens though if there are defects in the work we've delivered and we have to start planning the defect fixing as well as the new work? How should this situation be approached?

Firstly, don't deliver defects! Easy to say but we know that all teams deliver defects at some points. If you start getting a lot of defects back though it's a pretty sure sign that you have a false velocity - you're reporting work as done when it isn't really. It simply hasn't been tested enough. So the second point is that you have to face reality about the velocity of your team. Get the testing done and automated before you deliver the next set of stories.

All well and good but what about planning the next few sprints? How do we plan reliably now when we are not sure how much defect fixing we're going to have to do along side the new user stories. Here's one approach that seems to work... 2 velocities!

The overall velocity of the team is how many "points" the team burns down in a sprint. We want to ensure this is pretty constant, always assuming that the team stays constant and we're not carrying over too much work in progress at sprint boundaries (see here for a previous discussion of that problem). The second important velocity though is "effective velocity" - the amount of new client-required work that is burned down per sprint. This excludes work on defects (which means the higher velocity reported in sprints when you delivered the defects is balanced) and work on refactoring and process improvement (necessary but not what the client is paying for). You can see the effective velocity sprint by sprint by creating a new folder each sprint to contain just the new work completed/targeted).

Giving the team visibility of these metrics is important so they can see the impact of defects and also appreciate improvements when the effective velocity is restired.

Running xProcess with a console

2008-09-15T10:55:00.004+01:00

It seems I missed out a vital piece of information in the previous blog - how do you create a shortcut that runs xProcess with a console. Actually it's pretty simple: add "- console" to the target command. So on a Windows machine the Target field of the shortcut should look something like this...

"C:\Program Files\Ivis\xProcess 2\xProcess\xprocess.exe" -console

Reporting and exporting from xProcess 2.9

2008-09-12T18:32:00.019+01:00

A project I've been working with recently has been needing to get reports and data out of xProcess to integrate with other management reports. This hasn't proved as easy as I would have hoped with the product's standard reports, custom reports through BIRT, exporting and other options failing to give us what we needed. I decided to do some more experimenting with "UI Actions", which use OGNL to navigate over the data. Turns out this is a pretty good option with writing the required scripts being quite straightforward. The hardest thing turned out to be actually getting the export file which was a bit more of a hassle. I'm told all this will be great in v3 but for now here's my solution.

First you need to create a new Action in your process, and set it applicable to "Task" (or whatever type of object you want to report on) and "UI Action to "true". The only other crucial bit in this stage is writing the OGNL Expression. My script is below so you can always paste that in, see what it does and then change it!

We wanted to extract a set of key attributes of tasks (like dates and size) and display them in a spreadsheet.The script produces a set of comma-separated values (csv format). Unfortunately v2.9 doesn't give you an easy way to save this in a file. Instead the script outputs the values to the console (so you have to start xProcess with a console window). Once the output has been produced you then have to cut and paste the data into a csv file -- yes that's the "clonky" bit. Still it doesn't take a moment and you then have the data in a spreadsheet where you can mess with it to your heart's content. Brill!

All I want now is v3 so I can get this sort of report through a nice clean user interface. Here's the script...

"-- Output to console action - summarises task information --",

#categoryName ='Importance',
#category = #this.getProcess().getCategoryTypeByName(#categoryName),

#output='\n\nTabular output for: ' + getName() +'\n'+
'Name,'+
'Parent Task,'+
'Start,'+
'End,'+
'Target End,'+
#categoryName + (#category==null ? ' (WARNING - no such category)' : '') +','+
'Size (points),'+
'Estimate (person-hours),'+
'Actual/Planned (person-hours),'+
'Actual to date (person-hours),'+
'Closed?,'+
'Assigned to' +
'\n',

"-- First define the different functions for Leaf and Parent tasks --",

#leafOutput = :[
#output = #output +
name +','+
parent.name +','+
start +','+
end50 +','+
(targetEnd==null ? '' : targetEnd) +','+
(#cat=getCategoryInType(#category), #cat ==null ? '' : #cat.label) +','+
size + ','+
estimateOfEffort/60.0 +','+
bookedTime/60.0 +','+
bookedTimeToDateIncludingConfirmedTimeToday/60.0 +','+
closed,
getCurrentManualAssignments().{#output = #output + ',' + person.label},
#output = #output + '\n'
],

#parentOutput = :[
#output = #output +
'REST OF ' + name +','+
(parent==null ? '' : parent.name) +','+
start +','+
end50 +','+
(targetEnd==null ? '' : targetEnd) +','+
(#cat=getCategoryInType(#deliveryCategory), #cat ==null ? '' : #cat.label) +','+
(topDown ? (#result = size - sizeRolledUpFromChildren, #result>0 ? #result : 0) : '') + ','+
(topDown ? (#effort = estimateOfEffort - estimateOfEffortRolledUpFromChildren, #effort>0 ? #effort/60.0 : 0) : '') + ','+
bookedTime/60.0 +','+
bookedTimeToDateIncludingConfirmedTimeToday/60.0 +','+
closed,
getCurrentManualAssignments().{#output = #output + ',' + person.label},
#output = #output + '\n'
],

"-- Then process all the tasks, including the one selected --",

#tasks = getAllTasks(),
#tasks.add(#this),
#tasks.{
#this.isDesignatedAsParent() ? #parentOutput(#this): #leafOutput(#this)
},
#$sysout.println(#output),

"-- Inform the user where the output is",

#$dialog.informUser('Output to console: ' + getName(), 'The result of this action has been sent to the console.\nCut and paste the output into a Spreadsheet and use Data->Text to Columns... to create the table.\n\n'+
'Note: If you do not have a visible console, close xProcess and restart it from a command line window, or run xProcess with a console by adding "- console" to the target command. (E.g. on a Windows machine the Target field of the shortcut should look something like:\n\n"C:\\Program Files\\Ivis\\xProcess 2\\xProcess\\xprocess.exe" -console')

Note: one of the columns this scripts outputs is a category value ("Importance"). You could use the same pattern to display other relevant categories - "Target Delivery" for example if this was a category defined in your data.

How long should a Sprint be?

2008-08-06T11:43:00.003+01:00

Generally I'd I'd say between 2 and 4 weeks. How long can the business leave the team alone to get on with what they've planned (the "no interruptions"rule) is one constraint. The other is how frequently do we need to update the release plan - we only really get feedback on velocity at sprint boundaries.

I've been working with a team recently where establishing a constant velocity has been very difficult. They are basically dealing with a large backlog of defects from previous iterations where they weren't using an agile process. Until this backlog is cleared it's largely academic how long the sprint is - they still get interrupted with a significant number of urgent defect fix tasks every week. Perhaps going to a weekly sprint, at least until this backlog is effectively cleared would help. That way the defects being worked on in that week would be known, with analysis only being done on new defect reports during the week. The sprint planning for the following week can then re-prioritize the work for the following week. This is an example of where the sprint needs to be short in order to handle rapidly changing priorities. However it also demonstrates the difficulty of trying to apply an agile process when a clear quality baseline has not yet been established.

In other circumstances I favour a longer sprint rather than a short one - at least 3 weeks. It gives the team a chance to establish it rhythm of delivering backlog items without the overheads of sprint planning and retrospectives interrupting.

Requirements, requirements, requirements

2008-06-12T10:08:00.003+01:00

Is it possible to implement an agile project without overhauling your requirements process? I think not. The key economic benefits flow from agile methods ability to reduce "work in progress" compared to traditional methods [Anderson]. This means specifiers must take a much lighter approach to distant requirements and have much greater involvement, for example as "on-site customers" or product owners, in requirements currently being implemented.

[Anderson] Agile Management, Prentice-Hall (2003)

Availability - how much and who to?

2008-05-30T16:22:00.004+01:00

When you're specifying your availability in xProcess there are 2 parts to the information required: when are you available? and to which projects? If you are wanting to change availability for people in your project from the assumed defaults (usually 8 hours each weekday), then here are a few hints to get you started.

The first part - your overall availability - can be specified for both organisations and for individuals Open an editor for either a person or organisation and you'll find (at the bottom of the editor pane) a tab labelled "Availability". This shows the availability records that have been defined. Each record has a from and to date ("..." means "forever" in this context, either in the past for the "from" date, or in the future for the "to" date). For each day of the week you can either specify a specific number of hours or inherit the hours specified on another person or organisation.

By default organisations at the top level (that is organisations with no parent organisation) are created with availability of 8 hours on Monday to Friday. Sub-organisations or persons within an organisation are created to inherit availability from their parent organisation. So for example public holidays can be defined by adding availability records at the top level and personal holidays defined on the individual person in the same way.

The second part of defining availability is to divide your time between one or more projects. When the Project Manager adds a resource to his team he must specify the percentage availability (default 100%) and the from and to dates for this person (defaults to the start and end of the project). To change the percentage of time available to any one project, hit the "Resources" button in the Project Manager perspective "Tools" panel, or open the project editor and go to the "Available Resources" tab. You can then select the relevant person and use the "Manage Project Availability for..." button. Again availability records can be added for a period specifying a percentage of the overall time the person is available on each day of the week. (The default in this case is 100% each day.)

Process patterns: what are "composites"?

2008-05-28T12:38:00.015+01:00

The key to defining the task structure in a process definition is the task pattern. Task patterns consist of prototype tasks, optionally contained in a prototype project. Here's an example:

This is the task pattern for a "Feature" in the FDD process and it results in the creation of a set of subtasks and artifacts when the Feature pattern is "instantiated". This is quite a simple pattern and it doesn't include any composite tasks, which are structures to allow multiple tasks, optional tasks and iterations of tasks (repeated tasks that follow on one after the other). It is composites that allow arbitary process diagrams, drawn say using UML's activity diagram format, to be translated into xProcess processes. Here's an example of a task pattern, this time shown in the Hierarchy Diagram view, which does contain a composite. It's the "Feature Set" pattern from the same process.

The composite task in this example is called "Features: $Name$" (this is a pattern so the $Name$ will be substituted by the user supplied name when the pattern is instantiated). It contains a choice of 3 patterns: "Defect", "Feature" and "Task". Any of these patterns may be instantiated multiple times within the composite task.

Composite tasks are parent tasks, in other words they do (or will) contain child tasks, and they also define constraints on what type of child tasks can be created through the patterns they contain. In the latest version of xProcess composites can be instances (only one pattern can be instantiated in the composite), collections (an arbitrary number of patterns can be instantiated in them) or iterations (similar to collections but each pattern will be constrained to start after previously instantiated tasks have completed). Composites contain one or more patterns which define the options available when creating new tasks in the composite. In the example above, Defects, Features or Tasks can all be instantiated within the composite.

So the composite mechanism gives the Process Engineer the ability to define task patterns with loops, sequences and optional paths. More or less any pattern that can be represented in a flow chart or activity diagram can be created in an xProcess pattern by using composites. Composites go further though. Because of xProcess' support for top-down estimation, composites have their own estimates and resource requirements so that they can be scheduled even before the detailed decisions about how many tasks, iterations or options have been made.

To understand how these top-down estimates work in practice, see this blog article: Latest Patterns for Scrum which shows how the top-down estimate for a Sprint is used for scheduling resources, up to the point that all the backlog items have been created within it.

Footnote Question (for the techies among you!):
Are all parent tasks also composites as well as vice versa?
Well firstly all composites are parent tasks. Even if they as yet contain no child tasks they are classified as parent tasks because they allow task patterns to be instantiated in them. On the other hand parent tasks are only composites if they contain a set of patterns. This set defines the patterns that can be instantiated in the composite /parent task. So a parent task without such a set of patterns is not a composite and (in version 2.x) any non-hidden task pattern maybe instantiated in it.
[Note in version 3 this behaviour is likely to be changed, effectively removing any distinction between parents and composites. If the parent task does not contain any patterns, no patterns will be able to be instantiated in it. This will allow Process Engineers to create project patterns where, for example, certain tasks have a fixed set of children which cannot be added to.]

Not just project management - not just process improvement

2008-05-27T14:29:00.008+01:00

Explaining xProcess's position in the market of agile management products is something I never find straightforward. On the one hand it's important to emphasize the way you can simply define a set of tasks a team will be working on and immediately generate a plan (which you can then monitor in real time as tasks are re-prioritised, started and completed). On the other hand there's all that cool stuff you can do defining the task patterns, artifact templates, gateways and workflows that make up the process definitions behind different project approaches. I guess it all depends who you're talking to. If they're project managers needing something to get them started quickly, emphasize the simplest possible agile approach (prioritizing a list of tasks, defining the resources available and generating the plan). If they're process gurus - it has to be said these guys are thinner on the ground! - try impressing them the process modelling facilities xProcess provides for you. Producing the graphical task patterns and even workflows is a great starting point.

If you're going for the "simplest possible" approach, try these steps:

Create a new Data Source (or open an existing one) and switch to Project Manager perspective
Create a new Project (hit the "New" button in the "Tools" panel)
Define your tasks (hit "New", select "Task" and create multiple tasks in a comma-separated list)
Define your resources (hit the "Resources" button, then "Add/New Project Resource" and select yourself as the project resource
Hit the "Tasks" button to see forecast dates for your new tasks
View Gantt and Burndown charts for your plan
Add further resources if available and see the forecast dates and charts change
Hit the "Priorities" button and make one of the later scheduled tasks highest priority (drag-and-drop or type "0" in its priority field and save the change using Ctrl-S)
View the change to the task order in the Gantt chart.

SpringSource Application Platform released

2008-05-02T19:19:00.007+01:00

An interesting announcement this week that you may have missed is the release of the SpringSource Application Platform. Effectively this is a lightweight but high-performance platform providing most of the functionality of existing application servers in a much simpler, small-footprint package. SpringSource CEO Rod Johnson has been quoted as saying that it will provide a "leaner and more powerful solution." He continued. "The SpringSource Application Platform, comprised of the dm-Kernel we developed, along with the proven Spring, Eclipse Equinox and Apache Tomcat technologies inside, provides the industry with the leanest, most flexible server for running enterprise Java applications". Paul Kuzan, a leading software architect and part of the team that put together xProcess, has been contributing to the open source code in the SpringSource platform. "It's a really exciting development in the app server world", said Paul, "and one that is likely to make a very big impact in the years ahead. Watch this space!"

UK Earned Value Conference announced

2008-04-14T15:14:00.008+01:00

For another opportunity to hear my thoughts on using Earned Value analysis within agile develop- ment processes, check out this conference in London in June. My presentation is on the Wednesday (11th June) entitled Applying Earned Value in Agile Projects. I'll be reporting on the work to integrate EV analysis into the xProcess project-process improvement environment, highlighting essential modifications that make it possible to apply EV within agile projects.

Sample project revisited

2008-04-10T12:33:00.014+01:00

The last time we looked at progress on the blog's sample xProcess project we were looking at the way that project velocity measured simply as "points per sprint" can vary significantly (Calculating team velocity, 24.02.08). If we can gather accurate data about team availability and overhead activity, using a simple measure of productivity produces much better forecasts. Here's the burndown chart as last seen on 24th February.

This sprint completed the target 69 points although as can be seen by the slightly changed positions of the steps on the burndown, some individual tasks completed on different days to the forecast.

The following sprint also completed 77 points, again showing the problem of "yesterday's weather" forecasting, albeit within a reasonably acceptable 10% margin of error. However the productivity measure is much more stable, only varying from 0.91 to 0.90.

Here's the state of the current sprint. In this case we're forecasting 64 points to be completed.

To follow through a series of articles on Scrum in xProcess start with this article on "Using the pre-defined Scrum process".

Converting Eclipse RCP applications to Web apps

2008-04-07T17:42:00.005+01:00

We're exploring a new technology called the Rich AJAX Platform (RAP) at present which converts Eclipse Rich Client Platform (RCP) applications into AJAX-enabled Web applications. It looks very interesting. For those of you interested to find out more here's a link to a useful webinar...

http://live.eclipse.org/node/450

Since xProcess is an Eclipse RCP application you might gues the reason for my interest. The main advantage of this technology is the ability to share the same Java code base for the RCP and web applications.

Whitepapers available for download

2008-04-02T17:06:00.006+01:00

A number of whitepapers relating to process modelling and agile development are now available for download via the xProcess Europe web site. These include:

The last of these is a report into the BT experience of transforming a traditional CMMI-oriented culture with the adoption of agile techniques. This paper was published in 2005 and since that time more experience papers relating to the cross-over between software engineering focused organisations and the agile community have appeared. It remains though a gulf that is only rarely bridged by companies and communities - generally people prefer to stay one side or the other. I heard recently of a major bank that had implemented a major new project management methodology, standardising the approach for all its IT projects with the exception of - wait for it - agile projects! Surely it behoves us to understand the strengths and weaknesses of each other's approaches rather than merely excluding whole classes of projects. On the other hand I expect major gains to be made in the next few years by organisations who look to embrace both the lessons of agile methods and the best software engineering practices that have evolved over the lifetime of the discipline's existence. Thesis, antithesis, synthesis: it's a good pattern for progress.

Modelling processes to scale

2008-03-28T22:09:00.002Z

Tasks in process models come in varying sizes. There is a key distinction that divides tasks in process models into two classes: is this task at the "plannable" or "sub-plannable" level. Plannable tasks appear on plans, Gantt charts, burndowns and crucially time-sheets. They are the kind of tasks that take from several hours to days or even weeks. Plannable tasks are the trackable planned, estimated and costed elements in xProcess. However in some processes we wish to define (and monitor) tasks at the sub-plannable level. For example if my tasks is to distil a certain compound from ingredients (in a chemical process), the steps in the process may need to separately defined and tracked to ensure quality compliance. xProcess provides a mechanism to support this in the "Gateway Type". While these steps form only a simple sequence in v2, we've recently been experimenting with a richer model for gateways that will allow the same kind of forms - selections, collections, iterations and sequences - that are supported in plannable tasks through the composite structures. Together these facilities make a continuum of process modelling from from large to small scale.

Better Faster Audit

2008-03-22T09:52:00.001Z

I've recently been asked to outline the structure for our process audit service offering at xProcess Europe. The product is called the "Better Software Faster Process Audit" and is based on the structure of Dan Haywood's and my book of the same name(Better Software Faster). It covers 7 crucial aspects of any software development process:

The last step - Build and Deploy
The first step - Modelling and design
The stakeholder step - Requirements
The controlling step - Management
The continuous step - Quality measurement
The micro step - Development
The macro step - Architecture

Starting with the last step usually throws people, but given one of the key criteria for improvement is agility, it makes sense to start with seeing how close a team is to fully automating their build and test suite. There's no excuse for not fixing this area if you're using languages like Java or C# but with some older languages it can be tough to provide continuous integration for every change made by a developer.

Quality Measurement is another related area which is sometimes overlooked. Every team has some means to assess quality. For some teams "release the product and monitor the bug reports" seems to be their preferred way of working. Not recommended!

EVM and Agile Processes

2008-03-06T12:45:00.004Z

I am presenting a paper on EVM and xProcess next week at the 2nd EVM Conference in Birmingham, UK. I’d love to see you at the conference but if you’re not able to make it and would like a copy of the paper you can download it now from the xProcess Europe or Ivis web sites. Here's the abstract...

EVM and Agile Processes – an investigation of applicability and benefits

Earned Value Management (EVM) is a technique for understanding the degree to which a project is following a plan. A prerequisite for EVM as traditionally applied is therefore a stable baseline plan with a fixed scope. Agile processes for software development by contrast are designed to be responsive to a changing environment and enable discovery of requirements through their lifecycle. Agile plans are rarely stable or of fixed scope. However both approaches share the goal of producing plans that provide stakeholders with immediate feedback on the daily progress of projects and they share common needs in understanding and reporting progress and productivity. In addition the fact that agile processes map requirements to plannable tasks means that requirements may change without invalidating the baseline for progress reporting. This paper reports on work to develop tool support based on the xProcess process and planning product to apply EVM to agile processes. It shows how the metrics for schedule and cost efficiency can be modified for compatibility with agile approaches, and how they help effective management of agile projects, particularly when supplemented with tool support for project forecasting and monitoring.

Calculating team velocity

2008-02-22T10:34:00.024Z

Why is it that agile methods use timeboxes containing broadly similar sets of activities - as opposed to phases favoured by more traditional methods, which contain different types of activity in each phase through the lifecycle? A key reason is that stakeholders in the project get quicker and more effective feedback on the progress and direction of the project. As a consequence planners can judge more accurately the effectiveness of the project and what it is likely to achieve over its full duration. One of the most useful metrics planners get from each timebox is the velocity of the team. Scrum projects usually usually use the velocity from previous sprints to forecast their likely progress - a technique often referred to as "yesterday's weather" since it effectively assumes the next period will be broadly the same as the previous. In this article we discuss what velocity is, how to calculate it in xProcess, and how to go beyond "yesterday's weather" by calculating and using simple productivity for improved forecasting.

Velocity is defined as the amount of required work delivered per time period. In the Scrum process for example, required work is called the Backlog, and its component parts, the Backlog Items, are individually sized and prioritised. The size of an item is not the same as the effort required to complete it, though size and effort are related and may - once the velocity of the team is known - be derived one from the other (see here for further discussion of size and effort). The size of backlog items is usually estimated in a team-specific measure called points (sometimes called Story Points, Feature Points or, if related to the amount of work a team member could carry out in a full uninterrupted day, Ideal Days). The size estimates therefore really only give a relative size for the backlog items. However once a timebox has been completed - in Scrum timeboxes are called Sprints - the team has a measure of how many points they were able to complete and so have the first reading for velocity in Points per Sprint.

When using xProcess for Scrum you can see the velocity of completed Sprints from the burndown chart. For example in the Sprint shown below you can see that a total of 91 points were completed, giving the team a velocity of 91 points per sprint.

Using the principle of yesterday's weather it might be natural for the team to commit to a similarly sized set of backlog items in the next sprint. Before doing so they might be wise to look at all the history on the project. Here are the burndown charts for the previous two months, first Sprint 02 and then Sprint 01...
As we can see from the respective velocities of 40 points per sprint and 73 points per sprint, velocity is not always constant and, if we look at the reasons behind this, we may be able to forecast the velocity for the next sprint more effectively. Why did Sprint 02 for example do so badly?

Well one clue is in the dates for this Sprint. This won't be the only team to have discovered their productivity was lower between mid-December and mid-January! All the team were on holiday for at least some of the Sprint, and when they were in they probably discovered someone they needed to speak to was away, or that there were other activities they had to attend to which were not related to the sprint backlog. Another factor was that this sprint had a number of items that had been worked on but not completed at the end of the sprint. Since they are not finished they cannot be counted in the size of work "done". Backlog items like this do not necessarily get done in the following sprint since other priorities may be introduced to the team at that point. However if they are carried forward (as they were in this case), there will be considerably less work left to complete in the following sprint. This at least partially explains why Sprint 03 was such a productive one.

So what velocity should the team predict for Sprint 04? This is where other information from xProcess comes in very handy in the planning process. xProcess allows team members to record when they were working on overhead tasks (like management meetings, email and admin, training courses and office parties) as opposed to backlog items themselves. xProcess also give us information about the amount of time spent on backlog items that were not completed in the sprint.

The key metrics we want to derive in order to better predict the next sprint are these:

simple productivity (size of work done in points per ideal person-day)
overhead percentage (what proportion of people's time is applied to non-backlog-item tasks)
availability (when are people available to the project)

We'll leave discussion of overhead tasks and availability to another occasion, though they can be set simply in xProcess. The simple productivity measure of Size/Effort can be calculated automatically by xProcess. I say simple productivity measure since, as Putnam and Myers point out, it is more complex to derive a true process productivity for a project, taking into account that size and effort do not follow a linear relationship when the duration under consideration varies. This non-linearity can be broadly discounted however if the duration of the sprint does not vary. (One of the reasons why agile methods recommend fixed durations for timeboxes is to ensure that feedback from one timebox is applicable to the next.)

Simple productivity for a given period (a sprint for example or the whole project) takes into account all the non-overhead tasks booked to during the period. It also takes into account tasks which were incomplete at the start and end of the period. Once the simple productivity factor has been calculated for previous sprint (or for the project so far), the value can be use to set new effort estimates based on size using the Set effort to match size UI Action described in a previous article.

The simple productivity currently being used on this project is 1.0 - in other words a backlog item estimated as size 1 will be scheduled with an estimated required effort of 1.0 ideal person-days. Using this factor the current state of the burndown chart for Sprint 04 is as shown below.

However using the history from the previous 3 sprints, xProcess can calculate the actual productivity in points per ideal person-day. In this case it was found to be 0.91 - in other words historically a task of size 1 has taken on average about 8.8 person-hours to complete. Using the Set effort to match size action, we can quickly adjust the estimates of all the open tasks in the project to reflect this factor. Once all this is applied (see burndown chart below), the state of Sprint 04 now reflects the true situation that certain tasks are at higher risk of not completing.

Developing your own workflow monitor

2008-02-21T16:04:00.009Z

Just occasionally I'm tempted to post a really techie article that will probably only be of interest to hard-core xProcess enthusiasts! This is one of those occasions. I've written elsewhere about some of the aspects of workflow support in xProcess (see for example the article on State Diagram support which explains how to define triggers for internal events graphically, and "From tracking to planning and back" which discusses how tracker products can be linked to xProcess via workflow). This article though will go a bit further and give you some hints about how the Workflow Server works and how you can link in your own programs to detect workflow events and trigger actions on them.

The Workflow Server sits in a Tomcat instance watching the outside world (and the inside world of the xProcess Data Source) through a set of monitors. When these monitors discover events that might be of interest, they tell the Workflow Server which checks whether any object in the xProcess Data Source has registered an interest in that type of event. If it has it invokes the defined action(s) for that event, potentially committing changes to the xProcess data or indeed to external systems. Several monitors are delivered with the xProcess Workflow Server. There's the DataSourceMonitor itself which monitors state transitions within the Data Source.There are also monitors for the Flyspray bug tracking system, for CollabNet's Project Tracker and a general purpose email monitor. However if you want to integrate external systems with xProcess this is exactly the way to do it - write your own monitor. In spite of my introduction to this blog article, this is quite straightforward to do. There are 3 key elements:

A monitor class (inheriting from com.ivis.xprocess.workflowserver.monitors.MonitorImpl)
Methods for this class which initialize it, find the events
[optionally] An event class (inheriting from com.ivis.xprocess.workflowserver.ExternalEvent) and possibly useful methods that may be called when the event is triggered.

Here (below) is a code snippet for a sample monitor which show the key methods init(), which is called when the Workflow Server starts up, and findEvents(), which is called periodically by the server in order for this monitor to query its external system and provide the events to the server.

public class SampleSystemMonitor extends MonitorImpl {
SortedSet events;
public static String EXTERNAL_SYSTEM_NAME = "My External System";
public static String EXTERNAL_EVENT_NAME = "My event name";

public String getSystemName() {
return EXTERNAL_SYSTEM_NAME;
}

public String[] getEventNames() {
return new String[] { EXTERNAL_EVENT_NAME };
}

@Override
public void init(DataSource dataSource) throws WorkflowException {
super.init(dataSource);
// TODO initialize connection with external system
}

@Override
protected SortedSet findEvents() throws WorkflowException {
events = new TreeSet();
// TODO query external system for new events, e.g.
// loop
ExternalEvent xevent = new SampleSystemEvent(new Date(), EXTERNAL_EVENT_NAME);
Map map = xevent.getMap();
//TODO set map values so they can be referenced in the triggered action, e.g. ...
map.put("name", "New Task");
events.add(xevent);
// end loop
return events;
}
}

The monitors are singleton objects and are instantiated within the server by Spring. So having written your monitor you can inform the Workflow Server that it should be called simply by modifying the xprocess.xml in the Tomcat WEB-INF directory (following the pattern of other monitors). Specific objects in the xProcess Data Source can respond to the events and trigger specific actions through their "WorkflowPackage" - which is declared in the process definition.

Set effort to match size

2008-02-06T21:59:00.012Z

A previous entry provided the script required to define an action shortcut for setting the Size of tasks based on existing estimates for the effort required (see Set size to match effort). In some cases the inverse of this operation - Set effort to match size - is the useful operation. Here's the code you need for that one. Once again you should make it a "UI Action" by setting this field to True so that it appears on the right-click menu when you select a task or set of tasks.

Action Name: Set effort to match size
Expression:
#factor = getProject().getFloatProperty( "CurrentProductivity_pointsPerIdealPersonDay"),
#factor = (#factor < 0.01) ? 1.0 : #factor,
#size = getSize(),
setBest(#size/#factor*240.0),
setMostLikely(#size/#factor*480.0),
setWorst(#size/#factor*720.0)
Applicable to: Task
UI Action: True
Parameters: (none)

Note that the operation uses a factor to scale the mapping between size points and effort. If this factor is 1.0 the mapping is 8 hours effort (480 minutes) per size point.

Perhaps - like it's inverse operation, this will one day appear in the preconfigured operations of the Simple Process. Note that is very similar to the Set estimates action which is delivered in this process. However that operation is designed to take Size as a parameter (usually during the instantiation of a pattern) and so it is not suitable to make that one into a UI Action for operating directly on existing tasks.

Traceability through a simple metamodel

2008-01-30T14:48:00.001Z

When people start defining processes with total traceability between every aspect, the sight is often not very pretty! When traceability adds multiple layers of complexity it rarely pays for itself in terms of improved quality or maintainability.

This is one of the reason why we defined what we termed the "minimum metamodel" in the Better Software Faster book on agile development. The diagram shown here is this model in its simplest form. The questions traceability should answer are:

How is this requirement tested?
How is this requirement implemented?
How is this part of the design tested?
Why is this part of the design needed?

It's a useful starting point when defining the artifacts required in a software development process, and the references they should contain to enable these questions to be answered.

The Role of the Process Engineer

2008-01-24T12:31:00.000Z

Since xProcess provides configurability of processes, it also introduces a new role to projects - that of the Process Engineer. Of course the role may be played by a project manager or process consultant, or simply by one of the members of the project, but it's an important role and one that can greatly enhance the success of projects using xProcess.

Several Process Engineers have already created the processes that come bundled with xProcess (Basic Scrum, Basic FDD, Simple Process) and there are others such WPM Prince 2, Web Development, Issue Management and Basic UP that are available in the community. So the likelihood is that most Process Engineers will start with one of these processes as the baseline. As experience with a process is gained, the details of the way it is being used by the projects can be understood and the sorts of useful modifications or additions to the patterns, actions and workflows can be specified.

Using parameters in task patterns

2008-01-22T11:14:00.001Z

Processes are defined in xProcess through Patterns - patterns for sets of tasks, for projects, for priority groupings (e.g. Folders) and patterns for artifacts. By constructing sets of patterns, including patterns within patterns (Composite Tasks), a complete definition of the process behind a family of projects can be built up. A key part of pattern definition is the specification the pattern's parameters. This is what this article looks at.
The diagram above shows the editor for a typical pattern from the Simple Process, showing the 4 parameters that this pattern has: Name, Project Home Page, Start Date, Duration. These are the values that the user can supply when creating an instance of the pattern. Parameters always have default values which are used if the user does not change them, but you can add an action to a parameter - a DefaultValueGetter - which will set the default value at run time based on teh context or time at which the pattern is being instantiated. You can see in this case that Start Date has such an action attached which sets the start date to the current date. Parameters also have a type which can be one of: Integer, Floating Point, Boolean, String or Rich Text.

Having set up the parameters of your pattern you can then use these values to set values within the elements that are created by your pattern. Typically these will be projects, tasks and artifacts any or all which can be modified from the state of the prototype elements in the pattern to the real elements made when a user uses the pattern.

Th first way to use a parameter is by string substitution. Any text field in your prototypes, for example the prototype project's name, can include a parameter substitution. Simply enclose the parameter name with 'Simple Project pattern, the name of the prototype project is $Name$, showing that when the pattern is used the name of the project comes from this parameter. It's also used in the case of the Project Home Page parameter, which sets the description field to point to this url.

You can even use string substitution with artifacts, provided that the artifact uses a text-based, rather than binary format (e.g. rtf, xml, csv formats). In this case just put the substitution string, say $Name$ inside the file attached to your pattern. You'll find when you use the pattern that the corresponding artifact has this string substitution applied. Several patterns in the Basic FDD signs in the field, and when the pattern is instantiated the parameter value will appear at this point instead. You can see an example of this type of parameter use in nearly all the pre-defined patterns delivered with xProcess. In our example here, the Simple Project pattern, the name of the prototype project is $Name$, showing that when the pattern is used the name of the project comes from this parameter. It's also used in the case of the Project Home Page parameter, which sets the description field to point to this url.

You can even use string substitution with artifacts, provided that the artifact uses a text-based, rather than binary format (e.g. rtf, xml, csv formats). In this case just put the substitution string, say $Name$, inside the file attached to your pattern. You'll find when you use the pattern that the corresponding artifact has this string substitution applied. Several patterns in the Basic FDD process have artifacts that use this feature.

The second important way that parameters are used is in actions. As well as DefaultValueGetters, patterns have another very useful type of action - InstantiationActions. These actions are called when a pattern is used. When you define InstantiationActions you specify which of the prototype tasks you want to be transformed by the the action, and which particular action you wish to be called. Here's the InstantiationAction used in the Simple Project pattern.It uses the Set project dates action which sets a number of dates (targets and scheduler dates) associated with the project. It is this code which ensures projects you create start on the date provided by the Start Date parameter. One thing to note about parameters appearing in OGNL code is that spaces in parameter names are replaced by underscores and non-alphanumeric codes are ignored. Also following OGNL conventions the parameter is preceded by a hash (#). Note that the Set project dates action uses both the Start Date and Duration parameters.

When you're starting to use parameters in your patterns the best approach is to look at patterns delivered with the pre-configured processes in xProcess and to re-use actions that these patterns use. The Simple Process for example provides the following actions that you can re-use in your own patterns:

Next timebox date (Project): finds the latest finishing folder/timebox and adds a day to its end date
Set required role type (Task): finds a RoleType matching the supplied string and sets this as the required role type for the task
Set target dates (Task): sets targets
Get today with offset: returns a date relative to the current date
Set max participants (Task): sets the number of people for a task
Set project dates (Project): sets target and schedule dates
Assign to account name (Task): searches for a matching user and assigns to the task
Get earliest start or end date (Task): returns either today or the task's project's scheduleStart if this is later
Set target dates and weight (Folder): sets the folder's targets and priority weighting based on that of preceding folders/timeboxes
Next timebox name (Project): finds the name of the latest finishing folder/timebox with the appropriate name and returns a name with its number incremented
Set estimates (Task): sets three-point estimates based on Size
Set importance (Task): categorizes task into High, Medium or Low categories.

Note: there are also some UI Actions in the set of actions supplied by the Simple Process. See for example this article: Set size to match effort.

Artifact templates

2008-01-22T11:13:00.000Z

When you're defining the process for your project don't forget that a major part of this is defining the standards for the files, documents and records (artifacts) that are created, modified and qualified as a result of the process. xProcess provides excellent support for defining the types of artifacts that will be used in a process. Furthermore it stores both the document templates and the resulting documents within its version control system so you can not only reliably access the most up to date version, but also compare it with previous versions if necessary. Here's a task pattern showing artifact templates that have been added to each of the subtasks in the pattern.
You can do this simply by right-clicking on the task and adding a "reference" to a new artifact - an attached file, form or hyperlink. (To learn more about the different types of artifacts supported in xProcess see Creating Artifacts in the Help system). Once you've done this, every time you make an instance of this pattern a clone of your artifact template will be created for use with that instance. Another nice feature is that you can substitute text in the file when you instantiate it, using the values that the user enters into the parameters of the pattern.

Actually parameters of patterns are discussed in the next article in this series, so see Using parameters in task patterns for more details.

Adding gateways to task patterns

2008-01-22T11:12:00.001Z

An important part of defining the process for your project is defining how to know when a task is complete. Is a development task complete before unit testing for example? Almost certainly not. But if acceptance testing for an enhancement is a separate task in your process, at what stage does the developer hand it over for acceptance testing?

One way to define this in xProcess is to define a gateway for the task so that the user is prompted by a checklist to ensure he has completed the required work.
In the task pattern shown above you can see that a gateway called "Feature complete" has been added to the "Develop" task. To do this in one of your patterns simply switch from the Project Manager perspective to the Process Engineer perspective, you can thenopen the pattern diagram for your pattern (right-click on the pattern). You can then select "Gateway Type" from the palette and drop it on the task in the pattern that you want to have a gateway.

The content of the gateway is defined by creating or editing a Gateway Type in your process. Right-clicking on your process and selecting New -> Type of Gateway opens an editor like the one shown here on the right. This is where you can enter in the questions / checklist items that are presented to the user when he closes one of these tasks. In this case the questions are:

Tests complete and pass?
Documentation complete?
User approved?

In each case we must also specify the set of possible answers and whether the answer represents a "pass" or "fail" condition.

Let's now see how this shows up when a participant on the project closes one of these tasks using the web client. Here's the gateway screen that a user would complete before closing one of these tasks (click on the image to enlarge).
You can see that for questions where a "fail" response is selected a comment is required in order that a reviewer can understand why this was done. You can learn more about gateways in xProcess from the help files. See for example Answering Gateways.

The next article in this series concerns Artifact Templates.

EV for Agile

2008-01-08T11:50:00.001Z

Earned Value analysis is really designed for showing how closely you are tracking a pre-defined plan. Schedule efficiency and cost efficiency give separate measures of the degree to which you are "on-time" and on-budget". The problem with agile plans is that the scope of the project is expected to change as it progresses, usually with the aim of keeping schedule and cost more or less constant. Is EV analysis simply not applicable in this case? Actually in the context of a forecasting environment like xProcess, EV can still provide some most valuable metrics to agile plans. I've been invited to present a paper on EV for agile projects in a few months time and so I'd be most interested to hear of others applying EV with plans that must vary over time. Drop me a line if that's you!

If you're interested in receiving a copy of the paper btw click here.

The headaches of centralised data

2007-12-28T12:59:00.000Z

The UK government's woes concerning the potential loss of personal data has received a lot of publicity in recent months. It calls into question the whole strategy adopted by governments to build ever larger centralised databases containing more and more integrated data from multiple applications such as passports, social security, driving licences and health service. An insurmountable security nightmare arises from developing such databases. Just who can you trust absolutely to control access to such universal data sources?

Here's a radical thought - instead of centralising the data and building multiple applications around the same centralised database, why not build applications that can safely access multiple data sources, allowing departmental control of the more localised data sources with full version control and audit of all data access. Interestingly when we built xProcess, the data architecture we adopted lends itself to just such an approach. We've since named the framework XAPPA (eXtensible APPlication Architecture) and its continued development promises much in terms of rapid, model-driven application development for distributed systems.

Key benefits include:

Lets multiple “centres” manage the data they own and freely link to data owned by other centres.
Each centre manages access and audit
All data versioned
All access auditable
Every change recorded
Uses industry standard protocols for versioning
Uses XML files (optionally encrypted)
Transfer objects support Web2 applications and tools
Applications built with MDA/DSM
Applications can share data sources.

Frameworks like XAPPA may yet provide the means for both the integration and distributed management of data that future personal data systems will need. While all data storage and retrieval systems suffer from the risk of penetration, distributed systems at least limit the risk of total data penetration, which is the doomsday scenario few seem to realise is only too likely with conventional database approaches.

Document-driven processes

2007-12-01T12:12:00.000Z

Many processes are driven by documents. Documents are the resulting artifacts from tasks of the process, as well as being the inputs or informing artifacts for subsequent tasks. Examples of such processes range from drug administration, planning applications, legal processes of many kinds and financial control. Since many of these processes are also subject to regulatory control and audits it makes sense to define and monitor them within an environment like xProcess, which not only provides feedback on the forecasts and targets for task completion, but also can handle the version control of the document templates, and the documents themselves.

Modelling these processes requires understanding of what the key documents are, their format (which can be captured as document templates) and the transformation work which takes place through the process (which can be captured as task patterns). The quality control to be executed at each stage can be captured as gateways.

Here's a simple example.

Midway through Sprint 01

2007-11-30T18:15:00.000Z

Time to revisit our example Scrum Project started a couple of weeks back. How's our team doing? Here's the burndown as it was 2 weeks ago.

When we left it, the Sprint was red indicating an overcommitment based on current estimates. Either the team would need to negotiate a slight reduction in scope for the Sprint or risk disappointing the project stakeholders at the end of the period.

Here's the situation today (click for an enlarged view).

The status of the Sprint is now amber. Comparing the charts carefully one can tell from the reduced size of the work that there is a reduction in scope (also visible from the task lists) but there is good progress visible from the closed tasks to date. Other things to note in the screenshot are:

Alerts (showing where some open tasks have exceeded their "best-case" estimates - they are prompts that the 3-point estimates for these tasks may need review);
Closed tasks in the Explorer view (show with check mark); and
The task list for the Sprint (sorted by end date so that the at-risk backlog item is highlighted at the top).

Participating in the project

2007-11-30T16:22:00.000Z

The key thing about defining processes and project in xProcess is that the participants on a project can interact with the plans day by day, making them not just a view of what someone once hoped might happen, but an up to date record of the most likely outcome given progress so far.

The easiest way for project members to update their estimates, progress and plans is to use the web client of xProcess. By installing the xProcess server, this becomes accessible to team members using just a browser. They log in using their Subversion password and can then see their task assignments and update time bookings, which tasks are active, task estimates and descriptions, and other aspects of the plan relevant to their work. Here's a typical screen shot taken from our Scrum Project example (click on the image to enlarge).

Different filters can be used to show either just tasks scheduled for this week or all tasks assigned to the participant. Clicking on a task bring the task details in to the lower panel where they can be reviewed and edited. If there are documents or other artifacts associated with the task these can be seen and downloaded from this screen.

This article is part of a series of posts on Using the pre-defined Scrum Process. Check out the next article in the series here: Midway through Sprint 01.

Adding structure, artifacts and gateways to task patterns

2007-11-24T12:39:00.000Z

Task patterns can be very simple (just one task with no extras) or a quite complex template controlled by several parameters and providing the implementer of the task with template documents, quality checklists (gateways), variable estimates and a family of subtasks. It's important to note that often simplest is best! Or at least (following Einstein) "as simple as possible but no simpler". We already discussed this issue in the blog with regards patterns for FDD (see Comparing big patterns and small ones). In Scrum, we've shown that different patterns can be used in addition to or instead of the Backlog Item pattern (see previous posting). Now let's look at what additional features might be useful to add to our Enhancement and Defect patterns.

Depending on the size of Scrum team you may have specialist roles that go beyond those of

Product Owner
ScrumMaster
Team Member

For example the specialist role of Tester is useful on nearly all but very small projects. Developers must also write tests and test their code but Testers are generally more focused on end-to-end tests, user acceptance tests and full system tests. They may also have a role to play in accepting enhancements and defect-fixes. So for example we could identify subtasks in the Enhancement pattern with these different roles.

The alternative to multiple tasks, each requiring a single person with a single role, is to specify one task with multiple people, possibly with multiple roles. This would mean that each of these people would book time to the same task. Although this results in fewer tasks ,it's not necessarily simpler since the completion of the task is less well defined - basically the last one to finish closes the task. Experiment with what works best in your team - and let us know the outcome. We find such feedback invaluable.

In subsequent posts we'll look at:

These are all ways to enhance the task patterns and make them easier for managers and team members to use and be productive.

Different patterns for Backlog Items

2007-11-23T16:35:00.000Z

If you've followed through the series of articles on Scrum (see Using the pre-defined Scrum Process) you'll know they've covered making a Scrum project in xProcess and setting up a backlog of tasks that you can prioritise, forecast and display in Burndown or other charts. Participants on the project can access their tasks in the plan and update progress or modify estimates to complete them. But the pre-defined "Basic Scrum" process is just that - basic. It doesn't take into account the type of project you are doing with Scrum (it may not even be a software development project), nor the specifics of your company's or your team's process. With a little process engineering we can make things much more specific, and sometimes that's very useful.

The types of things you might want to consider once you're up and running with the basic process include:

Adding different role types if there are areas of specialisation within the team
Adding artifact templates for essential documents
Adding gateways to tasks (quality checklists)
Adding structure to the "Backlog Item" pattern, for example subtasks of a standard form
Adding different types of Backlog Item with different roles, estimates, gateways, structure and so on.

For example you might want to distinguish between a defect and an enhancement and use a slightly different structure of subtasks and roles for each. If you want both patterns to be used as backlog items you need to modify the Scrum Project pattern so that they can be added by Project Managers / Scrum Masters to Sprints and the Unscheduled Backlog.

This diagram shows the hierarchy in the Scrum Project pattern. It consists of a project, named in the pattern as "$Name$" (this string will be substituted with the name the user supplies for the project). It has a General Overheads... task and a Scrum administration... task, both of which cover background work not related to backlog items. The Backlog Item pattern appears 3 times within the project pattern: in the first sprint, Sprint 01; within the Sprint pattern (which can be instantiated within the Sprints parent task as many times as required for Sprint 02, Sprint 03, etc.); and within the Unscheduled Backlog task. In order to add additional or alternative patterns, all 3 of these locations will need to be modified slightly so that the new patterns appear in place of, or additional to, the Backlog Item pattern.

Here's the situation after adding 2 new patterns for Defect and Enhancement as alternatives to the general Backlog Item. This will give the Project Manager a choice of the types of backlog item she wants to make in each case.

One word of confession here: to add extra pattern shortcuts like this means that the composite task that contains them (Sprint 01 for example) must be a selection composite rather than a collection composite. In version Basic Scrum 2.9.0 they are collections so you either need to change this or, to save hassle, use version 2.9.0a or later of the process (if this is not yet available for download, email support@xprocess.eu.com to get it.)

Next we'll look at adding structure, artifacts and gateways to our new patterns. Or check out the next article in the Using Scrum series: Participating in the Project.

Agile 42 at the Scrum Gathering

2007-11-19T18:03:00.000Z

It was great to get together last week with old friends and colleagues Alex Aptus and Andrea Tomasini, who were over in London for the Scrum Gathering 2007. They are both doing Scrum consulting and training in Germany. Andrea's company is called Agile42 and if you think about it the name has some logic to it - "agile" is the undisputed answer to all the major questions of software engineering (mmm...?) in the same way that 42 is the answer to that only slightly larger chestnut: life, the universe and everything. Actually I'd recommend Alex and Andrea's work precisely because they understand that the answers to large questions are not trite or formulaic. If you manage to engage them for your project you'll not only get excellent training in standard methods but thoughtful and experienced consultants that will help you solve the hard questions too.

Another highlight of our meeting was seeing the newly opened St Pancras station on the day that the first Eurostar train arrived there from Paris. The queues at Europe's longest champagne bar meant that we weren't able to order their most expensive bottle at (literally) £2800 a pop!

By the way Andrea has a blog on Software Development Process in Practice. Why not check it out. He has a thoughtful article for example on requirements management. Here's the link: Software Development Process in Practice: Agile Product Development... How to manage Requirements? Effective requirements management has been shown time and again to be the most crucial aspect of successful projects so it's a worthy topic for discussion. I think it's an area that all of us involved with agile processes should be examining. My feeling is that agile methods stop too readily at the level of "features" without also trying to capture other aspects of requirements. 3 Types of Requirement tries to explain this - it summarises the results of some consulting in this area for a client adopting agile methods on very large projects.

Top-down or bottom up planning for Sprints

2007-11-19T10:58:00.000Z

xProcess supports two ways to specify the required resources for a parent task: top-down or bottom-up.

The simpler approach is bottom-up. A parent task has subtasks with their own estimates of size and effort. The parent task size is therefore just the sum of subtasks' sizes.

What if you don't yet know what all the subtasks are? In this case - or if you want to specify some contingency or slack without inflating the estimates for the subtasks (Parkinson's Law would suggest this is a good idea) - then you can supply an independent estimate for the parent task by specifying it as top-down.

A previous entry, Parent Tasks and Top-down Planning, has already discussed this feature and shows the user-interface for using top-down estimates. The question here (since this is part of the series on Using the pre-defined Scrum Process) is how to use it for Sprints in Scrum.

Let's take a look at the current state of "Sprint 01" in our example Scrum project. Although this is quite a detailed screenshot, just look particularly at the red figures in the middle and the Alerts at the bottom of the screen (click on image to make larger).

The negative numbers for the estimates of size and effort for the "REST" of this task, and the Alerts, tell the same story. The backlog items in this sprint exceed the size of the top-down estimate. (See Prioritising the backlog for how and why these items were added to the Sprint.)

In this state the top-down estimate really is of no use since the larger amount of effort required for the child tasks will determine the time it takes and the top-down estimate of the parent task will effectively be ignored by the auto-scheduler. The auto-fix for the Alerts provides 2 ways to solve this situation:

You can either increase the estimates so that once again the top-down estimate is greater than (or equal to) the sum of the subtasks, or simply convert it to bottom-up so that the estimate for the Sprint just reflects the size of its parts.

So Sprints which already have a complete set of tasks added to them should usually be set to bottom up. However Sprints which do not yet have their backlog items defined should be top-down. That's why when you create Sprints for the first time they are top-down, and as a result you can see the forecast assignments for people and the projected cost of each Sprint of the project, based on its top-down estimate.

The general rule for how to decide whether to use top-down or bottom-up is this:

if you think that you have captured all the work required in the subtasks, then use bottom-up
if you think that having completed all the identified subtasks that there may be some others, then use top-down and provide an estimate for how large these other subtasks are in the top-down estimate.

These other subtasks are what is referred to in xProcess as the "REST" of a parent task, or the Remaining Essential Sub-Tasks. When you discover them you can create them in the parent task and task-by-task the size of the REST will be reduced, Or you may become confident as time progresses that you have indeed identified all the subtasks so you can then convert the parent task to bottom-up estimating and the REST will be removed.

One more point about parent tasks: team members cannot book their time directly to a parent task, only to a leaf-level task. So if they find that they are carrying out part of the parent task, but not one of the identified subtasks, they may need to create a subtask so they can book to it. This is another way in which the REST of the parent task may be reduced.

The next article on using the Scrum process in xProcess concerns configuring your process with Different Patterns for Backlog Items. Or if you're more interested in how participants on your project can use the xProcess web client, see Participating in the Project.

Prioritising the backlog

2007-11-17T10:41:00.000Z

The key point about priority-driven planning approaches like Scrum - common in fact to all agile processes - is that the set of tasks undertaken by the team are ordered by priority rather than by a pre-determined procedure. (Priority is usually assessed based on business benefit and risk.) This article focuses on this particular aspect of using Scrum with xProcess and it covers:

Adding backlog items to specific Sprints
Showing a Sprint in Burndown and Gantt charts, and finally
Prioritising the backlog

Having created a Scrum project with resources and a set of backlog items (see the previous post in this series) our next step is to prioritise the backlog. Our starting point here is a project which has three Sprints defined and a number of backlog items that have been estimated for approximate size. For simplicity in this example I've created 26 backlog items (with single character names from 'a' to 'z') with sizes of between 1 and 5 "points" (points being nominal units of size corresponding approximately to an ideal day's work for the average worker - see earlier post for more about this topic). Here's a view of my project at this point (click on the image to make it larger).

For the backlog items to be scheduled they must be moved from the "Unscheduled Backlog" task to one of the Sprints. Maybe you have an idea at this stage about the most important tasks, or maybe you want to see how long they would all take given the current estimates and resources. In either case just select the backlog items in the task list and drag-and-drop them to the first sprint ("Sprint 01"). As you'll see in the screenshot, I've opened the Project Explorer pane to do this. It's a view that can be selected from the "Window" menu - or simply click on the project icon next to the project name on the "Tools" pane. Once the backlog items are in a Sprint you'll begin to see the forecast start and end dates for them. You can now hit the "Priorities" button on the Tools pane and begin to put them in priority order. First though let's look at the Gantt chart for "Sprint 01" having added backlog items 'a' to 'v' to it. Double-click on "Sprint 01" to see its details and click on the "Gantt Chart" button. You can view other charts for the Sprint in a similar way, for example by clicking on the "Burndown Chart" button. In the screenshot below you'll also see that I've put the task list in reverse End Date order (by clicking on the column heading) and on the Gantt chart I've scrolled down to show the tasks at the end of the Sprint.

The Gantt Chart shows the forecast start and end dates of the tasks based on resources and estimates so far defined. As time progresses and tasks are started, re-estimated, completed or changed this chart will also change of course - it gives us a view of the likely outcomes based on information currently available. The risk element of the estimates is shown by the 75/95% extensions on the Gantt bars (see "Why I use 3 estimates..." for a discussion of this feature). Given that the target end date for this Sprint is December 19th, we can see that 2 items ('n' and 'a') are in the red zone and not forecast to finish in time and several others are in the amber zone and at risk of not finishing. This would therefore prompt review with the team and the client to whom commitments are being made so that expectations are set appropriately.

Now what if 'n' and 'a' are actually the most important items to get done in this Sprint, or what if there are dependencies between some of the tasks? How are these handled? Well dependencies can be added simply by dragging-and-dropping a task on the task it is dependent on. But it's important for flexibility to keep such dependencies to a minimum. The main way to order tasks in the Scrum process is by priority. To see and change priorities click on the "Priorities" button on the Tools pane. Here's what you might see...

The priorities of all the tasks in "Sprint 01" are all the same (they're all priority one!). As a result the scheduling of the tasks is effectively random. Let's prioritise then in alphabetical order. Here's what the priorities list looks like then...

Changing priority changes the order that the tasks are scheduled, so the Gantt chart now reflects the alphabetical priority ordering. Here's the resulting Gantt chart for "Sprint 01".

And here's the projected Burndown chart for the same Sprint. It still shows a red status as we've not removed tasks by prioritising, but it reflects when the newly ordered tasks are forecast to close. For more on how burndown charts are used as the Sprint progresses, see "Don't burn out - burn down!".

The next article in this series looks at the significance of Top-down or Bottom-up planning for Sprints.

Using the pre-defined Scrum process

2007-11-16T10:56:00.000Z

I've recently been asked for more guidance about using the pre-defined Scrum process in version 2.9 of xProcess. It uses some of the new features for process definition (like Composite tasks) and also has "top-down" estimating for projects built-in, which is useful for resource planning and financial reporting. So I'm planning to post a series of blog entries on the process and in this one I'll focus on getting started with this process "out of the box". Here's what it covers:

Creating a Scrum project
Adding resources to the project
Adding Sprints to the project (a Sprint in Scrum is like a "timebox" in other methods)
Adding Backlog Items to the project (Backlog Items are the tasks the project must achieve)

So step one is to select the Project Manager perspective (the default) and hit the "New" button so we can make a Scrum project. The New project dialog shows the processes that are available for import, so if you've not already done so this is the point to import "Basic Scrum 2.9.0" and then select "Scrum Project" as your type of project. You can fill in the name and other details of the project (such as the number of 4-week Sprints and a location of a Wiki if you are using one) on this dialog.

At this point take a look at the set of tasks that are already in the project (hitting the "Tasks" button is the quickest way to do this. Here's what you might see (click on the image to make it full size).

There are:

some overhead tasks (green because they have fixed start and end dates and are within the target dates of the project)
A "Sprints" parent task containing one sprint "Sprint 01" (these tasks are red since they have NEVER end dates, clearly after the project target)
An "Unscheduled Backlog" (which is grey as this task is specified as not to be scheduled, also resulting in NEVER end dates).

In the pane below the project details, you can also see there are some alerts displayed telling us that this project needs resources with particular role types. That's a clue to our next step - add resources. Hit the "Resources" button on the "Tools" pane and you'll see an "Add/New Project Resource" button. If you've already defined or imported a set of people you can select those who are going to be on the project, or you can create new persons at this point. Add six team members (Participant role) and one team leader (Scrum Master role) to the project.

At this point we should see actual end dates appearing for the Sprints and the overall project and we can even see a Gantt chart and projected Burndown chart. The Burndown chart is slightly more interesting if we add 2 more Sprints to the project (hit the "New" button and select "Sprint"). Then hit the "Burndown Chart" button on the project pane and you should see something like this:

Now it's important to note that as yet we've entered very little information about this project other than the number of Sprints and the people on the project and their availability. The projected burndown is only reflecting this simple data showing when the work in each Sprint (based on estimates for the amount of work in a Sprint which is modifiable in the process definition or, for a specific Sprint by modifying the estimates in its "Required Resources"). To use Scrum in earnest we need some actual "Backlog Items". Backlog Items are identifiable tasks that in a software development project for example could correspond to features to be built or defects to be corrected. The next step therefore is to add such items.

Hit the "New" button again and select "Backlog Item" (and then "Next"). We have a choice of the 3 Sprints we've defined or the "Unscheduled Backlog" as valid parent tasks for these items. Setting the parent task as Unscheduled Backlog means that we can create a set of tasks without them changing the scheduling of the project. We can then decide later in which Sprint specific tasks should be completed. So with Unscheduled Backlog as the parent task you can give the backlog iten a name and a size (size is measured in "ideal days" by the way), and hit "Finish" and the task will created. If you already have a list of tasks you can copy and paste a comma-separated list into the name field of the "New" dialog, check the "Create Multiple" check-box and several items will be created in one go.

Now we have a project some resources and set of tasks to be completed. The next step is prioritising. We'll look at that in the next posting in this series.

Searching the blog by topic

2007-11-15T10:45:00.000Z

Blogspot offers some useful facilities for finding blog entries by topic. Below most entries there are a series of labels. Clicking on any of these will bring up other related articles on the same topic. Try it our with the labels below this post.

xProcess Europe

2007-11-15T10:06:00.000Z

On November 6th Ivis announced that it had demerged its European operation (see Press Release) . The new company is known as xProcess Europe and it's owned by former members of the UK team including Andy Carmichael and Paul Kuzan both seminal thinkers in the evolution of xProcess.

Andy Carmichael: "Paul Kuzan and I are both very pleased to be involved with the on-going evolution of the xProcess product and brand, and we believe xProcess Europe provides a great opportunity, particular for developing the services offering that customers adopting xProcess are often looking for. Providing better support and expertise for companies wanting to improve the processes behind their project and portfolio management will be one of the main focuses of the new company. While our product sales and support work will primarily be targeting the European market, we will continue to offer process improvement services world-wide, wherever the needs exist."

xProcess Europe and Ivis Technologies are both supporters and contributors to this blog. From time to time you'll find links and references to both companies here. For more details try their web sites: www.ivis.com and www.xprocess.eu.com.

The Simple Process... or is it?

2007-09-05T15:01:00.000+01:00

The Simple Process in xProcess is the first one you are likely to use. It contains basic patterns for a project, a task a timebox and so on, and it also has some useful actions in there that can be used in your own patterns or even exposed as "User Actions" so they can be called on your projects.

The Simple Process has several elements that you cannot delete as for example xProcess assumes that even if no other Role Types exist (such as Project Manager, Developer, Tester and so on) at the very least the Participant role type will exist. When you define your own process it's likely that you will inherit Simple Process patterns (though you can hide them by defining your own hidden patterns with the same name. So it's definitely worth taking a snoop at all the elements in there. You might even want to cut and paste some bits into your own processes and modify their behaviour.

Latest patterns for Scrum

2007-07-31T13:21:00.000+01:00

xProcess 2.8 is being released this week and among the exciting new capabilities are some new sample processes, including updates to the Scrum patterns. Here are some of the highlights.

Firstly the project pattern for the "Basic Scrum" process is now built from composite tasks so that there is a standard structure that is provided for all its projects. The Hierarchy diagram shows the structure, having top-level tasks for: General and Scrum administration overheads (overheads are unplanned or unplannable tasks that take up a proportion of everyone's time); the Sprints (a composite task which contains an iteration of Sprint instances); and the Unscheduled Backlog (a composite which contains a collection of Backlog Items).

You can also see from this diagram the structure of the Sprint and Backlog Item patterns. Sprint is a collection of the Backlog Items to be scheduled over the period, and consists of a single task. The beuty of xProcess of course is that these patterns can be modified to match the way your team is working or the particular application of Scrum that you are using. For example if you are using Scrum with XP you might want to rename the Backlog Item pattern to User Story and perhaps provide some structure to the set of tasks and role types that corresponds to0 the way you are using the method.

When you make a new Scrum Project in xProcess you'll be prompted for values of the parameters of the pattern. You can see these in this screenshot, where as well as defining the name of the project the start date, duration (the number of 4 week Sprints) and the Wiki location.
The Wiki location is not a required field but we've found many people find wikis are a convenient way to group documentation. By providing the location of the wiki in this dialog the descriptions of your backlog items are automatically transferred to a wiki page while also being browsable and editable in xProcess views.

Having created your Scrum project from this pattern, the next thing to do is add your resources. You need to add someone with a Scrum Master role and the others with Participant roles - let's say six participants in this example.

Because the tasks in the Scrum pattern are defined as top-down, even at this stage xProcess is able to provide a project schedule, based on the specified duration of the project. More usefully though, this is the time to create the first Sprint (right-click on the Sprints task and select New...) and then create some Backlog Items, either in the Sprint or in the Unscheduled Backlog if we don't yet know which Sprint we want them to be completed in. Backlog Items are prioritized, firstly from the particular Sprint into which they are created or moved, and secondly by the priority order of the tasks within a Sprint. You can edit this priority order by opening the Sprint editor, clicking on the priorities tab and either dragging and dropping or changing the priority number.

Once you have a Sprint which contains the items the team will be implementing, open the Burndown view. This provides both the history of how much work has been completed so far in the Sprint and a forecast of what should be completed. You'll see immediately for example if there's too much work in there and forecast dates are beyond the end of the Sprint. Here's an example of just one such Sprint... (click on the image to see more detail)Looks like some discussion needs to go on as to whether all the functionality is vital this time round!

Latest patterns for FDD

2007-07-31T13:20:00.000+01:00

The release of 2.8 this week sees some interesting updates to the FDD patterns. Like the Scrum updates for xProcess, the new FDD process uses "composite tasks" to define the structure of FDD projects, allowing them to be expanded top down as the details of feature sets and features are defined.

This hierarchy diagram shows the structure of the FDD project pattern. For those familiar with the method, the 5 stage process is visible in the top-level tasks:

Develop an Overall Model
Build a Features List
Plan by Feature and
combined with 5. Design by Feature - Build by Feature.

Note that the darker blue nodes are patterns, so these can be further expanded in the hierarchy diagram or viewed in the graphical pattern editor.

Here for example is the hierarchy view of the Feature Set pattern, which groups together a set of related features into a deliverable package which can be scheduled for a particular release. Note that here the "Features" composite task, as well as containing Features, may also group Defects (work on correcting issues from previous releases) and Tasks (any other work related to the Feature Set).

The Feature pattern has already been the subject of a number of postings on this blog (see for example...). The pattern diagram shown here uses a quite traditional view of the tasks required to deliver a feature with the emphasis on Specify (the "design by feature" part) and Develop (the "build by feature" part). We can see more detail of this task pattern by using "Go Into" in the pattern editor. Here the specific artifacts and quality gateways that are associated with each task, as well as the number and types of roles required to carry out the task, can be reviewed and modified.
Clearly other ways of breaking these tasks down are possible, and some of the FDD literature (for example Palmer and Felsing's book Feature Driven Development) provide alternative schemes that place stronger emphasis for example on design and code walkthroughs rather than the test-driven approach implied here.

The beauty of xProcess is that the task patterns are all easily and graphically editable so you can make this method match exactly what you want the FDD teams to carry out. As all plans, tasks, artifacts, processes and time records are stored in the xProcess versioned data stores, the compliance with your process can be monitored at any time, either to improve the process patterns where teams have discovered better ways of working, or to improve the teams' approach by following the best practice captured in your process patterns.

FDD has a three-level hierarchy of functionality: Features, Feature Sets and Major Feature Sets (also referred to as Business Areas). So far we've seen where Features and Feature Sets appear within the hierarchy of a project. Major Feature Sets (MFS's) are handled slightly differently. MFS's are created by an xProcess pattern in a similar way to the other patterns we've looked at, but they result in categorized folders rather than tasks which are part of the project hierarchy. This is so that the Feature Sets are visible within the Gantt charts and release schedules. Because of the scope of major Feature Sets, they generally do not have such a clearly delineated start and end date and so it would not make the project schedule clearer to include them in that way. So opening a MFS folder shows you all the feature sets and features in that category. You can then review and prioritize the features in just this one business area. Here's an example of a MFS in a particular project with its corresponding Gantt chart (click on the diagram to see more detail).
Finally in this brief review of xProcess FDD, the pattern for a Release is also instructive. Again a prioritized folder is used for the Release pattern, rather than a task in the project task hierarchy (see the Scrum method for an example of a process that uses this alternative). By default just one release will be created in the project (with a target close to the end of the project). However the Project Manager can create as many additional interim releases as he requires. The "scope" of the release is defined by moving the required features/feature sets/tasks into the release folder. The scheduler of xProcess uses the implied and defined priorities of these tasks to calculate completion dates and provide alerts if targets are unlikely to be met. As the project progresses, the scheduler uses the input from team members completing and possibly re-estimating the required effort for tasks to give immediate visibility of targets' status and costs providing all stakeholders of the FDD projects timely and detailed information to support decision making and further planning.

Set size to match effort

2007-06-20T10:42:00.000+01:00

I explained in another entry that size and effort are distinct metrics for tasks. However it's useful to keep the overhead of estimating two closely related items to an absolute minimum. This is why the patterns in the "Simple Process" uses the user's input of a size metric (in "ideal days") to automatically complete the effort estimates (usually displayed in person-hours or person-days). They can be changed subsequently though and if you want to re-synchronize them at some point - say when you a re baselining a plan - a utility for doing so is useful.

Here's a simple Action that you can define in your process (make it a "UI Action" by setting this field to True).

Action Name: Set size to match effort
Expression: setSize(getEffortIncludingChildren()/480.0)
Applicable to: Task
UI Action: True
Parameters: (none)

Having done this you can expand your task hierarchy in the explorer, select the tasks you wish to set size on, and then right-click and select UI Action -> Set size to match effort .

Note: From version 2.9 of xProcess this action is included in the Simple Process.

ITIL We Meet Again

2007-06-07T15:33:00.000+01:00

I was talking with Charlie Betz recently about ITIL, the best practices framework for delivery of IT services, and how xProcess might be used to support service delivery applying these practices. xProcess is well suited to "case management" type projects because of its ability to handle relatively small task patterns and create composite patterns of collections, iterations or selections of smaller patterns. With v3 of ITIL taking a more "lifecycle" approach, it brings a greater emphasis on process and stronger motivation to understand and improve underlying process models. That's why I'm particularly interested to find ITIL professionals wanting to model their processes with xProcess.

Any way I've been browsing Charlie's blog which has some interesting observations, and not just on ITIL. This entry highlights some frustrations with ITIL from an SDLC perspective: My first ITIL v3 presentation. It brought a smile to my face at least.

(Apologies to Diana Ross for the title of this entry.)

xProcess wins in the "Tools and Environments" category of the SD Times 100 Awards

2007-06-01T23:49:00.000+01:00

The 5th annual SD Times 100 recognizes the leaders and innovators of the software development industry. The 2007 SD Times 100 was posted on the sdtimes.com Web site today, and it names Ivis's xProcess as a winner in the Tools and Environments category. The award citation reads:

xProcess changes those lovely best practices into die-cast working frameworks for your developers. The easiest way to push some sanity into the development process.

David Rubinstein, Editor-in-Chief of SD Times, said "The winners of this year's SD Times 100 awards have demonstrated their leadership in shaping the software development industry. We took into account each nominee's products and services, its reputation among development managers, and the new ideas it brought out. These select individuals and organizations are the ones we've identified as helping to move the art of development forward."

The team at Ivis are proud to be a part of this select band creating the new breed of tools that will equip organizations and projects to deliver more, faster with less!

Parent Tasks and Top-Down Planning

2007-05-30T10:20:00.000+01:00

A key aspect of agile planning is that you expect the plan to change. That doesn't make it a bad plan, it's just the baseline from which you can judge whether to accept certain changes and what impact can be expected. To help you build flexible plans xProcess supports a very important feature - top-down planning. Parent or composite tasks can have their own effort and size estimates which are independent from the size of their children. Alternatively you can select Bottom-Up planning of the parent task and its size will be defined entirely by its children.The above screenshot is part of a task editor that is defined as a top-down parent task. It displays the size (15 points) and effort estimate (120 hours) but also shows how much of this is already allocated to subtasks and how much remains in the REST of the parent task.

The advantage of this approach is that parent tasks can be defined and estimated before the details of their subtasks have been defined. This works for both parent tasks and composite tasks which are collections, iteration or selections of task patterns. As Project Manager you can plan near-term activities in fine detail while leaving tasks that are further into the future as larger activities that will be planned in more detail as time progresses.

Outsourcing for quality

2007-05-02T09:19:00.000+01:00

Some people think that outsourcing is about minimizing the costs by moving work to wherever rates happen to be lowest at the time. The costs of distributing software teams can be high however and such a strategy may fail to reduce costs at all, and worst of all it can impact quality and agility.

There are good reasons to distribute teams however - for example to maximize quality by picking the best available expertise.
Ivis found such a partner in SwiftTeams who are based in St Petersburg. Their expertise in graphical editors for development tools is second to none, and the xProcess product has benefited enormously as a result.

How then do you overcome the difficulties of planning and coordinating teams that are geographically distributed? This is one of the key motivations of course for deploying xProcess. It provides the key visibility and communication layer and minimizes the negative impact of having a few thousand miles between the sponsors, planners and specifiers and the implementers. A number of key enhancements to xProcess in recent months have been motivated by improving our application of the Scrum process with SwiftTeams. We'll be looking for more in the coming months too.

Have you earned value today?

2007-04-24T17:42:00.000+01:00

There are many ways to track progress on projects - see "Don't burn out - burndown" for example. Burndown is particularly useful in agile projects with timeboxes where the targets are set at quite short notice - month by month for example. Earned Value (EV) analysis on the other hand is a great technique for tracking progress against a stable plan (ultra-agile teams go and read that burndown article instead of this one - stable plans are a luxury you rarely see!).

With EV tracking the value of completing tasks is calculated from its originally planned cost. There are some disadvantages of this - not least that the whole method relies on having the well-estimated plan to start from - however its advantage is that efficiency of execution can be measured day by day on the project.There are two relevant measures of efficiency that EV analysis provide: cost efficiency (how much better or worse you are doing than the baseline plan with regards the cost of the tasks you have delivered so far); and schedule efficiency (how much more or less value you have delivered at this point in time compared to the baseline plan).

xProcess supports EV analysis in the Executive Dashboards client (see screenshot). The Executive Dashboards provide an overview of multiple projects (or just one project if you like) so it's an ideal place to look at these efficiency levels in different projects. We may revisit how to use the views provided in later blog entries so watch this space.

Great resources for estimating

2007-04-20T15:30:00.000+01:00

There are two new books on estimating that I've been finding very useful recently, and one older one. The older one first; and here you understand I'm not referring to Larry Putnam Senior himself! It's his book, co-authored with Ware Myers, "Measures for Excelence: Reliable software on time on budget" that remains a classic, full of helpful advice and packed with actual measures from real projects. I met Larry in 1992 just after his book came out, when I was in Washington with Ed Yourdon presenting some research on software reuse from Japan. He kindly gave me a signed copy and it has been in frequent use since that time, and was a key source for my own book "Better Software Faster" published in 2002. Putnam and Myers' latest book "Five Core Metrics" is another excellent resource which highlights the essentials that all software projects should measure, with practical information about how to do it. In case you're wondering the five are:

Time
Effort
Quality
Productivity
Size

While Putnam and Myers experience covers projects ranging from the small to the truly massive, using a wide range of methodologies and covering five decades of software projects, Mike Cohn's book "Agile Estimating and Planning" is focused very much on the agile development processes of more recent history. It is similarly an excellent read with a mass of very practical advice for project managers of agile projects. I particularly liked the "Planning Poker" variant of the Wideband Delphi approach (!) but there are many other practical suggestions and insights, including discussions of burndown charts, velocity and prioritisation.

The challenge for me, working as I am on the product direction of xProcess, is to ensure that the full range of techniques and processes represented in these books is well supported by the underlying model and user interface of the product. xProcess has always been about agility, and in particular bringing powerful techniques from established project management methods and making them available to managers and participants of agile projects in such a way that they remain agile. These books similarly contribute to that drive to achieve controlled, productive and agile software projects.

Hierachy Diagrams

2007-04-19T13:43:00.000+01:00

Hierarchy diagrams in xProcess are a good way to see an overview of a project or of process patterns. The diagram on the left is a project that is following a Prince2 style project - actually using a process defined by the WPM Group, leading project management consultants and a company I've been doing some work with recently.

The Hierachy Diagram is a good way to show Work Breakdown Structures for projects, and they are also useful editor views, where you can expand/collapse regions of the hierarchy and edit tasks via their right-click menu.

One more thing to try with Hierachy Diagrams... turn on the constraints on the diagram via the filter button and (as in this example) you'll see all the dependencies between the various tasks. Very useful for understanding the constraints imposed on the ordering of tasks.

Estimating size and effort - Why are they different?

2007-03-30T10:03:00.001+01:00

A common question I get asked about estimating in xProcess (after three-point estimating of course, which is another popular question!) is: "What's the difference between a task's estimate for size and its (three-point) estimate for effort... and why do I need both?".

There are several ways to answer this, but perhaps a starting point is to ask why we think they're the same. If I ask you the size of your swimming pool you're more like to give me an answer in feet or metres that to reply with the number of person days it took to build the pool. (Building swimming pools is an xProcess application by the way - but that's another story!) When we are discussing the planning of a project the first thing to consider is the size (in whatever units are appropriate) of the deliverables of that project. Then we need a function (based on previous experience) to map from size to the estimate of the effort and consumables required to provide those deliverables.

Because the size of the things we want to deliver are measured in many different units, all of which we want to map eventually to an estimate of effort, it is common for projects to estimate directly in effort units such as person-days, person-hours or person-years. However if we do this, we lose a very important measure of the effectiveness of the project. Questions of team velocity or productivity become somewhat meaningless (except as a correction factor on original estimates of effort, compared to actual effort expended).

So if we can agree size is an important metric for projects, the next question is what units should it be estimated in, and can we compare the estimates with actuals at the end of the project. Software projects in particular are notoriously difficult to find an appropriate measure for size. Lines of source code is commonly used (because they are easy to count) but they don't readily map either to effective functionality delivered nor to effort required to develop them. Function points is another option and more recently "story points" (for an XP user story) or simply "points" have gained popularity where these are arbitrary units that are comparable only within the single project where they are being used. Another useful measure is "ideal-days" - in other words the amount that can be produced by a typical team member in a completely uninterrupted day. We rarely get such days (if ever) so it will take most of us more than a day to implement an ideal-day's worth of size. Nevertheless it is an understandable and reasonably verifiable measure of size.

In xProcess the units of size are common to the whole Data Source (they can be set in Data Source Preferences) and the value is used in several important places. When creating tasks the size value is often used in process patterns to set initial values for three-point estimates. The size units are used in burndown charts and earned-value plots to show progress of a project towards creating its deliverables. And size is also used to calculate productivity measures such as team velocity and software productivity.

Project Challenge! On your bike...

2007-03-20T10:41:00.000Z

The National Motorcycle Museum in Birmingham was the setting for last week's Project Challenge exhibition. It was the first opprotunity to see the new Executive Dashboard for xProcess in action, as well as to hear yours truly speaking about process modeling as a route to improve project performance. Needless to say the Executive Dashboard was clearly the highlight!

Guy's interest in the dashboard in this photo is not merely a pose for the camera. The show was also the first opportunity for the sales team to see this feature, which was turning a lot of heads (not just ours!). The ability to create additional views of interesting project data "on-the-fly" is a particularly impressive aspect of the facility.

Don't burn out... Burn Down!

2007-03-08T23:51:00.000Z

Following in the tradition of you heard it here first, here's another sneak preview of soon to be released functionality on xProcess. The burndown chart is a key visualisation that was developed in Scrum and now used with a number of agile methods. Currently you can use the reporting feature in xProcess to generate a burndown chart for your project or timebox. However there's a new feature in the wings that shows you the burndown instantly (history and forecast) on any project, parent task or folder. Here's an example:

This timebox (Timebox 01) runs for about 3 weeks and the scope of work targeted is forecast to complete right at the end of the timebox -- indeed a couple of the tasks are alread critical. (This diagram is looking forward, predicting when tasks will close, and thus when the work in each task will burn down). Look at the same time box a couple of days later.

Now -- perhaps because tasks have been re-estimated or resources changed -- we can see some tasks are forecast to finish beyound the end of the timebox. The chart is consequently shown as "red" status. Even at this early stage in the timebox we can predict that some deliverables are at risk and so take appropriate action. In this case some additional resources have been found and the some tasks excluded from the targets. Once the changes have been made, the effect can be seen and assessed immediately...

The timebox is "green" and all systems are go! Day by day as the timebox progresses we can monitor our progress and ensure the whole team stays on track!

When we visit the team half way through the timebox we can see from the burn down chart that they are still very much on track and confidence will be high that this team will deliver on its promises.

New project Pattern for FDD

2007-02-28T12:59:00.000Z

The project pattern for the Basic FDD process, which is delivered with xProcess is changing in version 2.7.2. Here's a preview.

This puts the emphasis back on the familiar 5 stages of FDD at the top level (Develop an Overall Model, Build a Features List, Plan by Feature, Design by Feature and Design by Feature). The first 3 stages are included in Timebox 00 which covers the start up period of the project and Features and Feature Sets are created as subtasks of combined final stages: Design by Feature - Design by Feature. Major Feature Sets are modeled as Folders with membership defined by Category.

3 Types of Requirement

2007-02-27T13:10:00.000Z

Priority-driven processes only work if you can model the process around task patterns for a single requirement. There is little point in prioritizing say, coding over testing, or design over specification, since all these activities must take place at some stage, even if they happen inside an activity that is called something else. But are all requirements the same (in the sense of using the same processes) or are some requirements more the same than others?!

Recently I was asked to advise on the requirements capture process for an organization wanting to apply agile techniques in a controlled environment. I came up with some slightly different names for types of requirement, though I think the concepts will be familiar to you if you've analyzed other software development methods. The requirements types (shortened to PCF) are as follows:

Business problem statements (Problems)
Solution contraint statements (Constraints)
Solution feature statements (Features)

I've written elsewhere about FDD, which provides a very good process for managing and prioritizing features. However often projects applying processes like FDD do not pay enough attention to the other types of requirement that precede features.

Problems, for example, should identify issues in terms of a measurable aspect of the current solution and the degree to which a performance improvement would overcome it. Constraints express aspects of the solution that the current designers are not expected to change (though they should also state the rationale behind the constraint, and how and by whom it may be changed if justification exists). Only by considering all of these three types of requirement can the essential requirements of a system be captured.

Action stations

2007-02-24T01:03:00.000Z

One of the most powerful aspects of xProcess is its flexibility and configurability. Processes are defined in terms of patterns: for example project patterns, task patterns and artifact patterns. Configurable enough. However you can do much more to enhance these and make them easier to use, by using xProcess Actions. These are bits of "OGNL" code that do things when xProcess is working. They can be triggered by user actions, by instantiating a pattern, by workflow or a number of other events.

For example Sam DerKazaryan of Isobar Global asked me recently for an Instantiation Action to increment the issue number on newly created issues. For those of you who like a spot of programming with their process engineering, here's the code snippet that does it...

#root = getPersistenceHelper().getRootExchangeElementContainer(),
#counter = #root.getIntProperty("currentCounterValue"),
#counter = #counter + 1,
setName("Issue " + #counter),
#root.setIntProperty("currentCounterValue", #counter)

Note: you need version 2.7.2 of xProcess or later to run this code.

Finalist for Eclipse Community Awards 2007

2007-02-21T14:45:00.001Z

Just received great news - xProcess has been shortlisted (1 of 2) for the prestigious Eclipse Technology Awards in the category Best Commercial RCP Application. The two Technology Awards aim to recognize the best Eclipse-based commercial and open source products in the world.

"Finalists were selected from a pool of 63 nominations by a panel of judges from the Eclipse community". Way to go!

New white paper on priority-driven processes

2007-02-21T13:12:00.000Z

Are your teams working on the most important issues to your business? If not you are losing potential earned value every day that goes by!

I've just finished a paper on this topic called Priority-Driven Processes, which you can download from the Ivis website. Such processes use the priority of deliverables as a key management control, enabling teams to focus on what provides greatest payback soonest. The paper is designed to help people to model and implement processes with priority, and draws on examples from FDD, XP and other agile methods.

If you do download, be sure to send us your feedback!