Illustration by Brendan Davis

Open PageTemplates

Today, ZPT templates can be dynamically translated via the i18n: namespace, ported from the Zope3 implementation of ZPT, fetching the translations from Localizer or PlacelessTranslationService. However, a few weeks before ZPT supported the new namespace, some Plone users were already seeing their sites translated: they were using AltPT, a fork of ZPT based on an independent reimplementation of TAL. AltPT managed to get i18n first due to its extensible nature; some things in ZPT are so hardcoded that adding a new namespace to it took more time for the brave developer that did it, than it took me to write a completely new TAL implementation complete with i18n. Now, many months later, AltTAL and AltPT have been renamed to OpenTAL and OpenPT and have many users, interested in a more accessible and maintainable implementation of such a central tool. While ZPT now has i18n, its Unicode support is very problematic, when it exists; OpenPT, due to its approachable architecture, has a powerful framework for encoding negotiation, and other interesting features. This article is an overview of how and why OpenTAL/OpenPT are different from TAL and ZPT and how to benefit from these features.

OpenPT: Page templates are under *your* control

OpenPT is a fork of ZPT I made a few months ago. Instead of Zope Corporations (ZC)'s implementation of TAL, it is based on OpenTAL, which in turn is based on an XML-handling library named PAX.

OpenTAL is not a fork of TAL; is a completely new implementation written from scratch (except for its implementation of TALES, which is remotely based on ZC's).

But why reimplement it at all? Well, the details are in my interview :-) (see this issue of ZopeMag) but in short, it's a matter of who is in control. ZPT is very difficult to extend, due to some design decisions aimed at optimization. When I wanted to add some features to it (i18n and SHOW [see below]), it turned out to be easier to reimplement it than to extend the existing code. So I did just that.

Additionally, OpenTAL was designed from the beginning to be usable/useful without Zope. Recently the TALES engine was made Zope-independent, so you can now have full page-template functionality in your own Python application. There are attempts underway to put OpenTAL in two other Python-based Web toolkits - Twisted and Webware.

On the Zope front, OpenPT is a drop-in replacement for ZPT; it implements exactly the same language, except that you can install (or write) syntax add-ons (for example SHOW, described below), and you can disable handlers (MeTAL, i18n, TAL, or add-ons) on a per-template basis.

Figure 1 - Adding a Open PT

The ZMI is a bit different, focused on this extra flexibility and the improved Unicode support, [insert screenshot] but don't let that fool you - the syntax is the same. In fact, if you install it alongside CMF, it "takes over" handling of PageTemplateFiles, giving you the full power of OpenPT for your CMF or Plone site.

Figure 2 - Editing a Open PT

Currently, the major drawback of OpenPT when compared with ZPT is that it is much slower. On the sites we have running this in production, we mitigate this problem by optimizing the transform engine with "psyco" but that's not a final solution; we won't dare dub this "1.0" before it goes through some major optimization work. Of course, speed is a secondary benefit for us compared to extensibility and readability, so we will only optimize as long as the resulting code doesn't give anyone any nightmares.

The OpenTAL project homepage (including OpenPT, PAX and PlacelessTranslationService) is at:

http://savannah.nongnu.org/projects/opental

and as of this writing the latest release is 0.5. The 0.6 release is targeted primarily at optimization.

1. OpenTAL is just an XML transform driven by namespaces You should be able to easily add your own namespace handlers You should be able to evaluate only TAL, turning off MeTAL, for example - or evaluate only your custom namespace, turning off TAL.

Figure 3 - File System - Open Page Template

One of the nice things about DTML was that you could add your own application-specific tags; we are in no way omniscient, and therefore there *must* be applications we don't cover.

The i18n namespace demonstrates this: to make my point, I implemented it for OpenPT in a separate package, the OpenPTi18n Zope product. You can remove this product and OpenPT will still work (without i18n, of course). You can switch i18n off on a per-template basis.

Another thing I have been wanting to do for a while is an application-specific handler to speed up common HTML-generating tasks (all DTML programmers love the way you can insert an '<img>.' tag pointing to an image object by doing '<dtml-var myimgobj>.'; in PageTemplates you have to use the uglier '<img src="dummy" tal:replace="structure myimgobj/tag" />.'). As soon as OpenPT was usable, I implemented a prototype of this and called it SHOW; you can download it from the same location as the rest of the OpenTAL/OpenPT/pax family.

Let's look at Show/__init__.py and see how hard it is to add a new namespace handler:

from show_handler import show_handler, show_compiler

def initialize(context):

# plug into Zope OpenPT
	      
  from Products.OpenPT.OpenPTBase import register_handlers
	     
  register_handlers(show_handler, show_compiler)

That's it. (Of course the real magic is in show_handler.py, but if you go into the source to see how it's done you'll find it pretty readable.)

2. The template is only compiled when you edit it (by changing the actual text, or any property that affects how it is interpreted, such as the list of active handlers). We should do as much as possible of the expensive computing in the compiling stage, to make rendering faster.

3. You are in control, not I. The template should solve problems of the people implementing sites, and the best way to do that is make it so flexible that you can easily bend it to your will. Currently, besides the namespace handler flexibility mentioned above, OpenPT gives you fine-grained control over the encodings used for processing data in Zope and for outputting the final result of rendering; if you are using OpenPTi18n and getting your translations from PlacelessTranslationService, these tools will negotiate encodings with OpenPT so that you're guaranteed to get an encoding that can correctly represent both your content and your (possibly translated) UI.

It also gives you control over the result content type independent of parser, which means, for example, you can dynamically generate SVG or any XML-based format (well, you can generate these with ZPT too, but it's not as convenient). More interestingly, you can also generate non-XML data: I'm using it to generate i18n-enhanced CSS and dynamic email.

(By the way, just as you can plug in new namespace handlers, you can also easily plug in new parsers; anything that returns a PAX tree is fine. I'm toying with the idea of adding a parser based on:

PyRXP

if it's as fast as they claim, and one for

if I can find the time.

And finally, if that is not enough, you are more than welcome to join me at the mailing list and CVS and help me shape OpenPT into an even better tool.

A few months ago, when AltTAL was in its infancy, one of the cool people from Twisted came to me asking if they could use it outside Zope. Well, I used to run my tests from Python on in a Zopeless environment, so I was pretty sure you could. However, having TAL and MeTAL is not good enough; for any real work you need TALES (the expression language, which interprets the values of the attributes - for example, "here/title" or "python:here.title" are TALES). My implementation of TALES was an almost straight copy of the one found in ZPT, which is deeply hooked into Zope. So I couldn't help him, but the thought kept bugging me.

Later, when I made the migration to OpenTAL, I made the decision to move TALES from OpenPT to OpenTAL. That was no easy accomplishment; in the end, I ended up with something completely different (there is no TALES.py in OpenTAL anymore; you will find the implementation in three modules: Context, Expressions and PythonExpr). This means you now can *really* use page templates without Zope:as mentioned above, there are people interested in using it in Webware and Twisted.

To provide non-Zope OpenTAL applications with a useful TALES context, I wrote a (yet incomplete) subpackage named OpenTAL.Static. Making use of the excellent:

Path module

written by Jason Orendorff, it allows you to traverse the directory tree, treating directories, Python modules and OpenTAL templates as objects:

        from OpenTAL.Static.Directory import Directory, Path
        from OpenTAL.Static.TAL_File import Defaults

        def gen_file(template, **kwargs):
            base = template.path.splitext()[0]
            template.gen = template.directory / (base + '.html')
            print template.name, '->', template.gen
            gen = file(template.gen, 'w')
            gen.write(template(**kwargs).encode('utf-8'))
            gen.close()

        d = Directory()
        Defaults['bindings']['cwd'] = lambda self: d

        for template in d._ls('*.tal'):
            gen_file(template)

The reason for the name "Static" is that I originally planned to use this package to generate static HTML from OpenTAL templates; as of this writing the package is already stable enough so that I can use it in the Website of my pet project:

You can see how the pages are generated by browsing

(the code snippet above is adapted from the make.py file).

Sincerely, I don't really care how many people use this. I'm more interested in how well it helps the people who do use it; in how many people are using it for purposes I didn't foresee; and how many people are developing it. Personally, I wrote it to "scratch an itch", as Eric Raymond says, and that it did better than I expected. What started as an experiment is now an exciting, useful tool that not only is incredibly fun to work with, but also helps keep food on my table (and that is something that cannot be overlooked when you have three kids to share it with). Heck, it is even interesting enough to write quite a long article about it.:-)