Illustration by Lia Avant

Archetypes - Part II

In the previous article you learned what Archetypes is and how to create a minimal Archetypes-based product from scratch. In the present article you are going to learn:

• How to make small customizations to both your product views and widgets
• How to reuse parts of existing views and widgets in your own customized form
• How to add automagically created search forms to your existing product
• How to make the Product you created installable.

• You have read the previous article
• You have Archetypes (1.2.2+) installed and running
• You have CMFPlone (2.0RC3+) installed and running
• You have the product created in the first article installed and running (optionally you can download it here )

• an average Python version (basic modules and operations) and, of course, knowledge of Python
• installation of Zope Products
• have Archetypes (1.0.1+) installed and running
• have CMFPlone (1.0.5+) installed and running.

You might be wondering what those Automagical Search Forms are, and how they can be of use to you. First, a bit of history:

Automagical Search Forms is a feature developed for a customer by me and Lalo Martins back when Archetypes wasn't even in its 0.9 version. Since then, this feature has gone practically unnoticed and been used very little – indeed, it was so little used that it was non-functional for several releases and nobody even noticed it. Last month, Joel Burton rediscovered the feature, so we've put some work into getting it functional again and added a few improvements.

Basically, what this feature does is to allow rendering of a series of widgets based on the current Schema for an installed Archetypes product of your choice. This basically allows you to perform any kind of catalog-backed search without having to worry too much about changing your search form when your Schema changes: in general one doesn't have to worry about changing the search form, although it might eventually require changes.

The search form is created using the following criteria:

• the name of an Archetypes package
• (optionally) the name of an Archetypes class
• (optionally) a context parameter to provide the context of the object used to render the search form

Based on these criteria, Archetypes narrows its scope to a set of Schema which are then filtered so that only widgets for fields that are indexed in the “catalog_tool” are presented to the user.

After the set of widgets is computed, the template is rendered using the “search” macro of the widget, which most of the time is exactly equal to the “edit” macro. We will be also looking at how you can customize this widget at will.

In the previous article, we created a minimal Archetypes product, which provided an “Article” content type. Our “Article” had a very simple Schema, composed of the BaseSchema plus a “body” field, which was used to contain some text. Since we provided a “searchable” parameter for the “body” field, this field is actually searchable. If you enter into Plone's Search Box a word that is present in the body of an existing Article, it will be found. However, the contents of this field are indexed so we are searching for “an Article that has word xxxx in any of its searchable fields”. Such a search might not be optimal, and undoubtably you will want a more fine-grained control over the searchability of your content type, because as your site gets bigger and bigger the quality of search results may decrease: you will want to provide means for the user to make specific searches (such as “documents” under the subject “Zope”).

So, let's go back and make some changes to our content type to make use of another nice feature of Archetypes.

Archetypes is all about Schemas. Schemas are used to describe the various facets of your content type – the structure and type of your fields, where the values will be stored and also the widgets that will be used to present them. Schemas also hold some very low-level properties of your content type, which are mainly used as “system metadata” for bootstrapping and fine-tuning some underlying features.

One example of this system metadata is the “searchable” parameter we described above. When Archetypes renders the value to be indexed in the “SearchableText” index (default in CMF), it goes looking for fields with the searchable flag on and tries to concatenate all their values together to form a long string to be indexed.

Another example of this system metadata is the “index” parameter of fields. This parameter is used for specifying:

• that a field is to be indexed;
• what kind of index should be created for this field, if none exists at this point;
• whether this field will also get catalogued in the catalogue “metadata” (a.k.a. “brains”); and
• for filtering which widgets will be rendered on the Automagical Search Form.

Let's take a quick look at an example to see how you would provide this info.

In our “Article.py” of the previous installment, we defined the following Schema:

schema = BaseSchema + Schema(
             StringField('body',
             searchable=1,
             required=1,
             widget=TextAreaWidget(label='Body Content'),
  ))

Let's improve this a bit by adding a “group” field, which contains the information about the kind of Article we are dealing with. For this group field we will use a “vocabulary” which defines the values available for selection and can also be used to ensure that the values chosen come only from a restricted set. Using a vocabulary also allows us to employ a “SelectionWidget” that renders a drop-down box showing the values available for the user to choose from. The modified schema now looks like this:

from Products.Archetypes.public import DisplayList
ARTICLE_GROUPS = DisplayList((
    ('headline', 'Headline'),
    ('bulletin', 'Special Bulletin'),
    ('column', 'Weekly Column'),
    ('editorial', 'Editorial'),
    ('release', 'Press Release'),
    ))
schema = BaseSchema + Schema(
	       StringField('group',
             vocabulary=ARTICLE_GROUPS,
             widget=SelectionWidget(label='Group'),
             ),
             StringField('body',
             searchable=1,
             required=1,
             widget=TextAreaWidget(label='Body Content'),
  ))

As you can see, the vocabulary is created from a “DisplayList”, which takes a list of tuples where the first element of the tuple is the internal value, i.e., how it is stored, and the second element is the visible value, as it is rendered for presentation to the user.

Great, so now we have a group field to distinguish between various kinds of articles. What about making it indexable and searchable? Easy task. We just add an “index” parameter to the field. The modified field looks like this (the change is highlighted in bold type):

	       StringField('group',
		 index='FieldIndex:schema',
             vocabulary=ARTICLE_GROUPS,
             widget=SelectionWidget(label='Group'),
             ),

Here, we are telling Archetypes two things:

1. For the field “group”, create a “FieldIndex” (also called “group”)
2. Further, create a “group” column in the metadata of the catalog (this is done by the “:schema” here)

Here we have used a FieldIndex because it is the more effective type of index for searching within a restricted set of values.

You can also specify fallback indexes, in case one of them is not available, for example:

   index = ('TextIndexNG', 'TextIndex')

You should be now ready to try what we have achieved thus far:

1. Restart your Zope, if you had it running
2. Uninstall your Article product, using CMFQuickInstaller
3. And then re-install it, also using CMFQuickInstaller
4. Go to <yourplonesite>/portal_catalog
5. Click on the “Indexes” tab and make sure a “group” index was created, with an index type of “FieldIndex”
6. Click on the “Metadata” tab and make sure a “group” item was added there, as well

Now that you have your field indexed, let's take a look at how to create a search form and integrate it into the portal.

There are several different places where you could put the search form. One could put it at the top or bottom of a content type, or on some special page. Although it's not hard to do these, we will go with a more common approach: we will create a separate “Advanced Search” page, and have it appear as a “tab” on the Plone site.

For this, we need to create a Page Template with some common boilerplate code to apply the portal look to the template, in addition to the code to display the search widgets.

First, let's take a look at the complete template, and then we will explain its various parts and how they work together.

<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en-US"
      lang="en-US"
      metal:use-macro="here/main_template/macros/master"
      xmlns:tal="http://xml.zope.org/namespaces/tal"
      xmlns:metal="http://xml.zope.org/namespaces/metal"
      xmlns:i18n="http://xml.zope.org/namespaces/i18n"
      i18n:domain="plone">
  <head><title></title></head>
  <body>
    <metal:main fill-slot="main">
      <div tal:define="errors python: {}">                                          
        <tal:fields repeat="widget python: here.archetype_tool.getSearchWidgets()">
          <metal:fieldMacro use-macro="widget"/>                                    
        </tal:fields>                                                               
      </div>
    </metal:main>
  </body>
</html>

As you can see, there's not much new stuff here, except for the lines in bold type. The following line tells the template machinery that we will be using the “master” macro of the ”main_template”:

      metal:use-macro="here/main_template/macros/master"

And the following line tells the machinery that we will be overriding the “main” slot of the “master” macro with our own custom content:

    <metal:main fill-slot="main">

It's the lines in bold that we are now interested in. The first one declares an empty ”errors” variable which is required and used by the widgets later on. Then, the second line calls a method from “archetype_tool” to fetch a set of widgets to be rendered (which will be iterated over) and the third line renders each widget, one after another.

There was a more detailed explanation of the inner workings of the “getSearchWidgets” method at the end of the Part I, so we won't repeat it here .Instead we will present a more practical example, based on the “PublisherInventory” product which was released on the Collective and which makes use of this feature. (In fact, the feature was originally developed for a slightly modified version of PublisherInventory).

Screenshot 1 action box at the left.

Here we can see the results of using “getSearchWidgets('ArchPackage', 'Release')”, which builds a search form containing the 'Maturity' and 'Version' fields. As you can see, the 'Maturity' field presents several possibilities for searching ('any', 'Stable' and 'Development') which are the valid values for this field - thus, the valid values for searching.

Screenshot 2

In this screenshot, the call used was “getSearchWidgets('PublisherInventory', 'Series')”, which renders the widgets for fields that are indexed in the 'Series' content object of the 'Publisher Inventory' package. Note how different this is from the previous example, and how the fields that have a set of values ('Vocabulary') are rendered in the same way as in the 'edit' form.

Screenshot 3

Here we have an example of “getSearchWidgets('PublisherInventory')”, which renders the whole 'PublisherInventory' package. Compare this with the previous example, and note that there are some new fields, such as 'Color', 'Format' and 'Designer'. These new fields are from other content objects of the PublisherInventory package. Since they were not present in the 'Series' content object, this explains the difference.

Now that you have seen a few examples of how flexible the search form has become, let's see how you can add this to your product. We will use the same product we created in the previous article, so if you don't have the source code around, click here to get it.

To make the search really work, our autogenerated form needs to be changed just a little bit to point to the search script available in Plone, and also use Plone's 'search_results' template to display the results of our search. Basically, this means adding a 'form' tag around our generated form that has the 'action' attribute set to 'search'. Click here to download all code for this article

Now, to get this hooked up to your Plone site, we need to:

• Register a global action (so it appears on the top tabs).
• Create a 'skins/my_article' directory under our 'MyArticle' product
• Register the Filesystem Directory View with CMF
• Register our skin with Plone's 'portal_skins' tool

To do this, we will add some code to '__init__.py' of our existing 'MyArticle' product. The resulting file now looks like this:

from Products.Archetypes.public import process_types, listTypes
from Products.CMFCore import DirectoryView, utils

from Products.MyArticle.config import PROJECTNAME, \
     ADD_CONTENT_PERMISSION, GLOBALS

DirectoryView.registerDirectory('skins', GLOBALS)

def initialize(context):
    ##Import Types here to register them
    import Article

    content_types, constructors, ftis = process_types(
        listTypes(PROJECTNAME), PROJECTNAME)

    utils.ContentInit(
        PROJECTNAME + ' Content',
        content_types      = content_types,
        permission         = ADD_CONTENT_PERMISSION,
        extra_constructors = constructors,
        fti                = ftis,
        ).initialize(context)

And then add a few lines to 'Extensions/Install.py' to add the skin to Plone and register our global action:

from Products.MyArticle.config import PROJECTNAME, GLOBALS
from Products.Archetypes.public import listTypes
from Products.Archetypes.Extensions.utils import installTypes
from Products.CMFCore.utils import getToolByName
from Products.CMFCore.DirectoryView import addDirectoryViews
from StringIO import StringIO

def install(self):
    out = StringIO()

    installTypes(self, out, listTypes(PROJECTNAME), PROJECTNAME)
    skins_tool = getToolByName(self, 'portal_skins')
    addDirectoryViews(skins_tool, 'skins', GLOBALS)

    skins = skins_tool.getSkinPaths()

    for skin_name, skin_path in skins:
        path_elems = [p.strip() for p in skin_path.split(',')]
        for skin in ('my_article',):
            if skin not in path_elems:
                path_elems.insert(1, skin)
        new_path = ', '.join(path_elems)
        skins_tool.addSkinSelection(skin_name, new_path)

    at = getToolByName(self, 'portal_actions')

    at.addAction('my_search',
                 name='Advanced Search',
                 action='string:${portal_url}/my_search',
                 condition='',
                 permission='View',
                 category='portal_tabs')

    print >> out, "Successfully installed %s." % PROJECTNAME
    return out.getvalue()

This is a simple example, but it already shows the power of Archetypes and how easy it is to build new content objects. Let's look at it in detail and see what's going on.

This article has described how to modify an existing product in order to create indexes for certain fields automatically, and also how to create an automagical search form that is updated automatically according to the changes you make in your Schema.

This is a very powerful feature and, combined with user-modifiable Schemas, as on Andreas Jung's PloneCollectorNG , it can make a world of difference for the usability of your app.

It is planned to include user-modifiable Schemas in upcoming releases of Archetypes, including a schema-editor completely based on XML. To see an example (Note: Requires viewing using a Mozilla based browser) Click here.