The New Action Mechanism

With the release of xProcess v3.5 we have implemented a new way of collecting data through Actions:
http://sourceforge.net/apps/mediawiki/xprocess/index.php?title=Reporting (the Recursive Solution). Existing actions can still be used but those that have a tendency to generate large amounts of data may need to be rewritten in order to get the benefit of the new mechanism.

The following example reworks the Work Log for Person action.

In which ever Process you want to, create a new Action called 'Get DailyRecords for Person'. This will return a set of daily records in a date range and is the FetchSet Action.

The first difference we see in v3.5 is that there is a new Editor for Actions. The Editor is split into three tabs:

• Editor tab – a text area for the OGNL Expression

• Details Tab:

• Action Name - the name of the action
• Applicable to - What it is applicable to (any element is the default)
• UI Action - should it appear on context menu for element its applicable to?
• On this tab are a further two tabs; Parameters: add the Parameters that will be presented to the user when the Action is run. These parameter values are sent into the OGNL expression when run. We will come onto the Export tab later.

• Test Action Tab:

• On the Test Action Tab you can choose an Element to run the Action on to test it and enter the parameters for that action. It is recommended that you create test Elements for this, as the Action may change that Elements data.
• The lower part of this tab shows the output when the test it run

For the 'Get DailyRecords for Person' we just need the OGNL for retrieving the daily records, so in the Actions Editor we enter in:

#project = projects.isEmpty() ? null : projects.iterator().next(),#from

= (#project != null) ? #project.getDayProperty("fromDateForReport") :

null,#to = (#project != null) ?

#project.getDayProperty("toDateForReport") : null,

#dailyrecords = getWorkLog(#from,#to,null)

Hit Ctrl+S to save the Action.

The next Action to create is the 'Generate Work Log Details'. This Action is the Subordinate Action. You can have one or more Subordinate Actions, and these Actions are run against the data set that the FetchSet Action returns. The OGNL to enter in this Actions Editor is:

#output = '"' +

day + '","' +

assignment.task.project.label + '","' +

assignment.task.label + '","' +

assignment.requiredResource.roleType.label + '","' +

time/60.0 + '","' +

logEntry + '","' +

confirmed + '"',

#output = #output + '\n',

#output

Hit Ctrl+S to save the Action.

The final Action is the 'New Work Log for Person'. It's OGNL gets called before the FetchSet and Subordinate Actions, so we use that to provide the column header that will be written to file:

#project = projects.isEmpty() ? null : projects.iterator().next(),#from

= (#project != null) ? #project.getDayProperty("fromDateForReport") :

null,#to = (#project != null) ?

#project.getDayProperty("toDateForReport") : null,

#output = 'Work Log for: ' + label + '\n'

+ 'From: ' + ((#from==null) ? 'earliest booked time in data source'

: #andfrom) + ' To: ' + ((#to==null) ? #$today : #to) +'\n'

+ 'Date,Project,Task,Role,Hours,Log Entry,Confirmed\n',

#output

Then go to the Details tab and:

• Make the Action applicable to Person
• Click on the Export tab and tick the Export Action checkbox. Give it a .csv extension
• On the FetchSet Field click on the '…' and select the ' Get DailyRecords for Person' Action
• Now click on the Add button in the Subordinate section, and select the ' Generate Work Log Details' Action
• Hit Ctrl+S to save the Action

We can now test the Action via File | Export | xProcess Export | Custom Reports. When the Action runs it uses the FetchSet Action to retrieve a set of Daily Records on Projects for the selected Person. This Daily Record set is then iterated over passing in the Daily Records to the Subordinate Action.

Since each time a Daily Record is iterated over a line is written out to file, it prevents the building up of large data to be written out to file, and possible memory issues.

Reporting in xProcess

The mechanism for defining custom reports in xProcess is straightforward. It uses "Actions" (expressions which use OGNL as the language for navigating the xProcess model) and the string returned by the Action is exported to a file with whatever file extension is appropriate. There are several examples delivered with the Simple Process which are worth exploring and maybe modifying if you need a specific type of report for your project. See "How to define a custom report in v3" for more details.

Recently with users defining larger reports on projects that have been running for multiple years a problem has emerged with this mechanism - out-of-memory errors. As the text for the report is held in its entirety in memory, prior to any of it being written to file, it is clearly not too difficult to define an Action that will fail in this way if you have a reasonably large data source. To see and comment on the technical solution proposed for this problem, see the wiki for the open source project here: https://sourceforge.net/apps/mediawiki/xprocess/index.php?title=Core_Design. The new mechanism allows records to be written out one by one, but it will require reporting Actions to be rewritten to take advantage of the mechanism.

Watch this space for more details.

Update! See Tim's article on the new mechanism - http://xprocess.blogspot.com/2010/08/new-action-mechanism.html

Powered-by-xProcess Applications

As discussions continue about opening up the xProcess world to other applications through the web services interface, it's worth considering the types of application this could enable. From an automatically scheduled to-do list on an i-Phone through to case-management support applications, the scope is of course boundless. But where are the quick wins? Most people who have seen the potential of xProcess, recognise that it's firstly in the context of supporting standard software engineering and project management processes such as Scrum, FDD, Prince and UP. The process definition gives teams access to standard patterns of tasks, if necessary with standard templates for associated artefacts and quality gateways. But processes are everywhere in the business landscape and if they are processes that need planning, resource allocation and forecasting, it may well be that the xProcess model provides just the foundation for powerful on-line support. Furthermore alternatives in this space are much less prevalent.

The web services interface means that full access to the underlying model will no longer be restricted to the Eclipse-based RCP client. Mash-ups which combine multiple sources of information can use the xProcess data source as the process and scheduling engine while application-specific views and data can be integrated seamlessly. If you've ideas for applications or would like to know how you can use the xProcess model for your project, do get in touch. The link to the Source Forge project is here: https://sourceforge.net/projects/xprocess/.

Cycle time and cashflow

Cycle time*, by which I mean the time to market for a feature, is not merely an academic concern. In times like these it may involve your job security, your project's viability and your company's future.

Clarke Ching amusingly points this out in his latest biztech parable "Rocks into Gold" with a tale of a back-from-the-brink project that found a simple way to justify its existence and solve the cashflow problem. It a quick read that's well worth the effort.

Graph's like the one above also bring home the message that even a small reduction in cycle time - specifically the time to external release of new features of a product - can result in major improvements in economic performance measured by return on investment (ROI), time to pay back, and profitability.

Agile processes - influenced for example by Goldratt's Theory of Constraints (TOC) - stress the need for rapid turn around and frequent releases, and the key drivers for this are the financial benefits from early release. The inhibitors to frequent and early release of new features are usually lack of automated testing and incomplete continuous integration. Using the financial arguments, particularly improving ROI, is the best way to justify and assess investment in this area. Automating acceptance tests in particular involves considerable effort and expense, but if it results in reduced cycle times and improved quality the financial justification is clear.

*Note: Unfortunately cycle time is an overloaded term that means different things to different people. I now trying to use a different term whenever I can. See "In search of unambiguous terminology for Flow systems" for further discussion of this issue.

Driven by priority - practical tips

Agile processes are designed to support changing activities when the business environment changes. This after all is the natural understanding of the word agile - the ability to change direction quickly. What this means in practice is the tasks in plans - backlog items in a release say - have to be prioritised and changed when priorities change. This has always been a feature of xProcess, but in the very earliest releases the priorities list was a single list. We soon found when we had up to 1000 stories, features requests and defects to prioritise against each other it was an almost impossible task to get your brain round it all. This led to the current model in the product where groups of tasks can be defined and prioritised relative to one another, and then the groups themselves can be given a "weight" relative to other groups so that the overall priority list can be derived. This allowed us for example to prioritize bugs relative to one another and then decide on the balance between bug-fixing and new features in any given release. It made the whole process much more manageable.

The screen shot below shows a single "Priority Group" being prioritised, either by dragging and dropping tasks or by giving them a priority number.

Recently a user of the xProcess "Basic Scrum" process raised the issue that the relative priorities of user stories get lost if stories are moved from one Sprint to another (for example if they are incomplete at the end of the Sprint). Here's my reply:

xProcess has a very flexible mechanism for defining priorities using "Priority Groups" which allows any Folder or Parent Task which has been "prioritised" to support ordering of tasks within it. Each Priority Group can be given a "weight" so that all the priorities can be combined into the ordered list that the scheduler uses. As with many of the very flexible mechanisms in the product, when you actually tailor the product for a given process (Basic Scrum in this case), you have to make some decisions about the normal way to use the features.

Basic Scrum defines Sprints as prioritised Parent Tasks with diminishing weight (later Sprints therefore have lower weight and are scheduled after the earlier ones). As you've observed when you move a task from one Sprint to the next the priorities they had in the previous Sprint gets lost. So if this is a normal part of your process - I can understand why it could be - why not define another prioritised group (say a Folder called Release Backlog) where you can give your stories a longer lasting relative priority. The weight for this group does not need to be high because it will just be the "tie-breaker" when stories have the same priority in Sprint. I think this should achieve what your aiming for.

There are many different way to use Priority Groups, some of which have been incorporated into the standard processes released with xProcess. For example the Simple Process has the facility to add "High", "Medium" and "Low" categories to the project so that just by categorising tasks they get prioritised appropriately.

Accessing xProcess via Web Services

Wouldn't it be great if you could build your own web interfaces which integrate project process and scheduling information from the xProcess model?

We thought so too. That's why we've started an inititative in the open source project to provide access to xProcess via Web Services. This will be a different server to the current xProcess web server (see screen shot) which uses Tapestry and has facilities for supporting participants on xProcess projects (booking hours to tasks, closing tasks with gateways, access artifacts and assigning tasks). There will be a considerable amount of common code however as the authentication, integration with Subversion, scheduling and multi-threading will all follow the same design. However the web services interface is intended to give full flexible access to the complete underlying data model of xProcess. The first services being defined allow you to access and update project data, but the mechanisms of data exchange and update are generic so almost anything is possible!

If you think this project might be of interest to your needs, or if you're interested in participating in it, please contact us. Your input will be most welcome. There are further details on the Wiki about how to use the source code once downloaded: http://sourceforge.net/apps/mediawiki/xprocess/index.php?title=WebServices

Precision and accuracy in estimating

The difference between precision and accuracy has always been important to engineers and it is vital to understand too in the context of estimating for software projects. One approach to estimating a project is to create a detailed work breakdown structure and estimate each task, say in person-hours. A friend told me recently she had consulted on a project which had estimated its size as 135,213 person-hours! The precision in this case is 6 significant figures. As to the accuracy however, previous projects had errors or between 10 and 200%. The precision actually gets in the way of understanding the estimate, particularly as crucial business decisions involving significant proportions of a company's revenue may be based on such numbers. While three-point estimating (3pe) provides one way of capturing risk estimates along with the estimate themselves, the main contribution of agile methods in this area is the tight feedback loop from measured velocity timebox by timebox. Check out some of the links below to see how xProcess supports these concepts.

Naked Objects

I recently received and re-read Dan Haywood's excellent "Domain-Driven Design Using Naked Objects" (I'd read a review copy some months ago). It's definitely worth the effort and I'm hoping to get some of the exercises done too as this means I can get up to speed with where the Naked Objects framework currently is these days.One of the first pieces of prototyping we did with xProcess (back in 2003) was to create a domain model (using TogetherSoft's "Together") and then connect it to Naked Objects so we could dynamically interact with those early models of Project, Task and Person. Naked Object effectively animates a domain model so you have an instant user interface to manipulate the domain objects. I believe the approach helped us then and I would certainly recommend projects consider doing the same - particularly in early stages of a project - as getting good domain models early pays off many times over at the end of the day.

CSV Import in xProcess

We're currently working on new import mechanism in xProcess that is both more powerful and more flexible. It builds on the recently released facility to define custom columns in tables for tasks and other elements in the "rich-client" implementation. This allows you to display information about elements, say a calculated value based on standard or customised fields of a task. You can export values displayed in a table into a CSV (comma-separated values) file. The new facility will allow CSV files to be imported and the values in columns to be imported as properties of the elements imported. If you'd like to know more about this facility or be involved in beta testing the feature do let us know. You can contact the development team via the forum in the SourceForge project: http://sourceforge.net/projects/xprocess/develop.