Illustration by Brendan Davis

Using MetaFlow

Cooperation is the heart of any organization. Without it, no group of people can acomplish their common goals. Cooperation, though, comes at a pretty high cost. People are different, and they usualy have very different views about what cooperation means and what their role and the roles of others are in the group.

Any good business manager will tell you that facilitating cooperation is their most marketable skill. Why? Because the cost of cooperation is usually the most expensive cost for a company or oganization. Consider the common business term "group dynamics". What's dynamic about it? You have a wide range of people, usualy categorized somewhere between laize-fair and anal-retentive. Some may think their responsibilities light, others may take things very seriously. Just the time it takes to make sure everyone is on the same page costs a lot of money, and that doesn't take into acount the cost of time when arguments erupt, projects slide, and people come to you asking, "Was I supposed to do that?".

A big term thrown around now in the Internet industry (and especially in the vaguely-defined "content-managment" industry) is "Workflow". Don't belive necessarily what you read, because everyone defines it differently and often in cryptic technical terms. It's not a tool with button "X" and lever "Y". Workflow is really about *cooperation managment*. It's about using software to help you keep track of tasks; a job that gets exponentially more difficult the more tasks and people you add to the process.

MetaFlow is a tool that lets you describe how one or more people can carry out a sequence of steps to acomplish their goals. For example, your goal may be to publish articles in a newspaper. To acomplish this goal, you need a workflow system that lets your writers create articles, your editors review them, and finally allows you to publish reviewed articles in your newspaper. A tool that automates this process, from creation to publication, is a workflow tool. A good workflow tool means that newspapers get printed on time with good content, a bad workflow tool means you're out of business.

A newspaper is just one example of a system that can use workflow. Another example: your company may have a process that it goes through to approve contracts that are negotiated with outside contractors. In order for a contract to be approved, it must be "signed off" by a sales person (who negotiates the contract) the accounting department (to see if it costs too much) the research department (to see if it duplicates existing work) and finally the head boss. Using workflow, you can keep track of multiple contracts and where they are in the process; who has signed a contract and who needs to sign it next.

A good workflow system has many features, but greatest of all, it *must* be flexible. People are constantly evolving the way they do business and the processes they use to acomplish their goals. If your workflow does not evolve when you do, then it becomes an obstacle, slowing down your work instead of speeding it up. Flexibility means that the workflow can adapt by allowing you to easily add new steps to a process or remove old steps that no longer apply to your goals.

Recently, I encountered a number of projects that needed a workflow component. Some of these projects were Zope applications, but others required only a Python-based workflow component to be used by an existing Zope product or even just a stand-alone Python component.

To start on these projects, I looked at the two existing Zope workflow tools, the CMF and OpenFlow. Neither tool satisfied our needs for:

• A simple, standalone Zope component. In this regard, OpenFlow comes closest to being self-contained and having minimal dependencies. CMF Workflow is the opposite; no content can be moved through CMF Workflow unless it is CMF content and works through the CMF interface with CMF's tools. This is great for CMF, but we didn't need any of CMFs other features.
• An extensible object model. Both OpenFlow and CMF Workflow objects are monolithic Zope classes; there are no abstract base to extend upon. This ties the product to Zope only and locks out Python-only or Jython based projects.
• An clean, native Python API that is environment neutral. This means that no Zope assumptions are made at the most abstract level, and that the API is meant to be called from Python or Python Scripts, not just from the web. Both products have pretty good APIs in this regard.
• Documentation. Neither OpenFlow nor DCWorkflow is documented to any serious degree. Nor are there any example applications bundled with either products. Correction: OpenFlow does come bundled with an example
• Usability. DCWorkflow is pretty good in the usability department; most of the features are pretty obvious to anyone who knows a little bit about workflow. OpenFlow's usability was a little too opaque and confusing, and the OpenFlow designers seem to use a workflow terminology-set I'm unfamiliar with, so without documentation it made it even harder to understand without doing some research on the Internet.Ed. Note: OpenFlow uses an "activity"-based workflow terminology set instead of an entity-based one like MetaFlow and CMF.
• Tests. Neither OpenFlow or DCWorkflow come with any test code. This increases the risks and vendor-dependency associated with a important core component to our business. Well written tests can also make great developer level example documentation.

After I weighed all of the features of both systems I couldn't really come to a decision; neither package really made me feel like I was using something I could put my money on. In the end after a couple of prototypes, the engineers at beehive and I decided to write our own workflow tool that satisfied our goals. Here are some of its features:

• Environment-neutral. MetaFlow works in Python only, Zope, or Jython using the same API across all three environments.
• Standalone component. MetaFlow's Zope implementation is a standalone component and does not require any additional frameworks beyond a stock Zope system.
• Documentation. MetaFlow comes with both user and developer documentation, and also is bundled with an example "bulletin board" application built with page templates, python scripts and MetaFlow.
• Extensible object model. At the most abstract level MetaFlow makes no assumptions or dependencies on Zope. These abstract core classes are then extended to create the Zope component of MetaFlow. The future is wide open for us to extended the core classes in different ways.
• Unit Tests. There are some simple tests that flex the workflow core, which mitigates most of the risks. More Zope specific tests are also on the way.
• Simple. I only implemented the functionality I needed. While this may lack features that other workflows have, I (or you!) can add those when needed.

The rest of this series of articles is about using the MetaFlow product. I am obviously biased toward this product vs. other workflow products, so please don't take my word as impartial in this case. After reading these articles, I encourage you to take the time to look over the options available to you. It's quite likely that your needs differ from mine, and therefore you may find another package that satisfy you.

This is the first in a series of articles about using MetaFlow with Zope. The MetaFlow system allows you to create workflow objects to acomplish the kind of goals described above. This document is a high level overview of how MetaFlow works and how you can use it to automate common workflow processes. MetaFlow works with Zope 2.5 and can be installed like any other Zope product.

In the abstract, workflow consists of two kinds of objects, *states* and *transitions*. A state is a specific point in a workflow, for example in a newspaper, after an reported writes an article it is in the 'private' state. An article waiting for editorial review would be in the 'pending' state. An article that is published in the newspaper would be in the 'published' state.

Transitions are actions that take you from one state to another. In a newspaper, when a reporter wants to submit an article for review, the reporter takes the 'request review' transition and moves the article from the 'private' state to the 'pending' state. The editor then reads the article in the 'pending' state and wants to publish it in the paper, thus moving it to the 'published' state. The editor would do this by taking the 'publish' transition.

Alternatively, the editor may want to reject the article because it contains errors or it's just plain bad. In order to move the article from the 'pending' state back to the 'private' state, the 'reject' transition is taken. And finally, a published article may want to be retracted by the editor, in which case the 'retract' transition can be taken to take the article from the 'published' state to the 'private' state. A good way to think of states and transitions is with a simple graphic:

Here, you can see that states are boxes and transitions are arrows. There are three states, 'private', 'pending' and 'published' and four transitions, 'request', 'reject', 'publish' and 'retract'. This is the kind of workflow that is easy to create with MetaFlow.

Once the MetaFlow product is installed in your Zope system you can begin adding *Workflow* objects right away. A workflow object is a Zope object that contains a single *workflow defintion*. A workflow definition describes one workflow process. Many workflow objects can be created in the same folder, and each can hold a different workflow definition.

Workflow objects have X managment tabs across the top, including 'States' and 'Transitions'. These tabs are used to define workflows.

To add a new workflow object, select 'MetaFlow Definition' from the list of available objects and click 'Add'. The add form looks like this:

and asks you for an id and title. After you enter the id and title of the workflow object click 'Add'. For this example, use the id 'Workflow' and the title 'Newspaper Publication Workflow'.

The next step in designing a workflow is to think about what the workflow you need will look like. When building a workflow, you should consider the following points:

• Make it simple. MetaFlow can certainly handle very complex workflows with lots of states and transitions, but can *you* handle it? A complex workflow is an indication that maybe your process is broken and can be greatly simplified, for example, perhaps your workflow can be broken down into several simpler workflows.
• Think in terms of your users and their actions. A workflow is about people who want to acomplish goals. When designing a workflow, consider all of your users and all of the actions that wany to take to acomplish those goals.
• States are like labels and transitions are like actions, so name them that way. In order for your users to understand "what happens next" in your workflow you need to design and document your workflow using terms and concepts your users understand. Another good simile to remember: States are like adjectives and transitions are like verbs. If you "write the story" of your workflow, you can often pick the states and transitions right out of the sentences.
• Draw a diagram. Sketch what you want on paper first, and show it to all of the users who will be using the workflow. Listen to their input; maybe you can remove a state, or consolidate a bunch of related transitions. The more elements to your workflow you can simplify, the more time, money, and patience you'll save.

Now that you have thought about your workflow and talked about it with your peers, you can get on with building the workflow definition in Zope itself. The first step is to add new states.

States can be added by clicking on the 'States' tab. On this tab, you define the states in your workflow by filling out a form with your web browser. When you click on the states tab of a new workflow object, it says "There are no states defined in this workflow". To create a new state, type in the name of the state and click the "Create New State" button.

Create three new states named 'Private', 'Pending' and 'Pulished' by using the "New State" button on the States view. After creating these states, you should see three new states on the 'States' screen.

After adding your states, you must select and "initial" state. This is the state that this workflow starts on. In the case of our example, news items start off in the 'Private' state, so select the 'Private' state and click the "Set Initial State" button. We'll come back to this screen a bit later after we've added transitions to the workflow, which is the next step.

Transitions are added in much the same way that States are added. New transitions are created in the 'Transitions' tab and new transtions can be added by clicking on the "New Transition" button. Transitions require a *name* and a *destination* in order to be added to a workflow.

When you add a new transition, the form asks you for a "name" and a "destination". The destination of the transition is where the state that the transition ends up in when it is taken by a workflow. In the example above, four transitions need to be created:

• The 'request' transition goes to the destination 'Pending'
• The 'publish' transition goes to the destination 'Pulished'
• The 'reject' transition goes to the state 'Private'
• The 'retract' transition goes to the state 'Private'

Now you have three states and four transitions added to the workflow object. The last step is to edit each state so that there are transitions that can be taken from that state.

For example, from the 'Private' state, select the "request" transition and click "Save Changes". This makes the "request" transition available to be taken from the Private state. Do the same with the following states and transitions:

Pending -> Add Transition "reject", "publish"

Published -> Add Transition "retract".

Now, you have a fully functional workflow specification. You can verify this by looking at the 'States' view of the workflow object and verifying that it makes sense.

Now that you have a workflow definition, you need some objects that move through the workflow. The MetaFlow product comes with a "workflow-aware" file object that can be moved through a MetaFlow definition. In later articles in this series, we'll show you how to easily create your own kind of workflow aware objects.

To add a workflow file, select 'MetaFlow File' from the list of available objects to add. This form wants some information from you, an id, an optional title, an optional file to upload, and a method to find a workflow definition. You can specify either an id, an expression, or tell the workflow to automatically 'find' and use the first workflow definition it finds.

Since you know that you created a workflow definition called 'publication' in the beginning of this article, you select 'id' and type in the name 'publication' to make this file object refer to the publication workflow definition.

After adding the file, click on the file object and select the 'Workflow' tab, which looks like this:

Here you can see that the file object in indeed in the 'publication' workflow definition and that its current state is 'Private'. You are also given the option to take the 'request' transtition. If you take that transition, you can see that the file has moved from the 'Private' state to the 'Pending' state.

This is useful, but for your own workflow applications you probably don't want your users logging into the Zope managment interface and directly manipulating these objects. Instead, you would want to build a simple Zope application that uses page templates and python scripts to display and work with objects that are in a Zope workflow. In the next article in this series, I'll show you some simple page templates and python scripts that make a simple, but complete, newspaper publication system that uses the workflow definition described in this article.

Creating workflow with the MetaFlow product is easy. In the next article in this series, we'll take a look at building a simple newspaper application in Zope that moves various files through a publication workflow definition. In the next article in this series, I'll show you how to build a simple newspaper web application using MetaFlow, Zope Page Templates and a couple of python scripts to tie it all together.