Illustration by Lia Avant

Zope sites and ZEO

Slow or unresponsive Web sites turn users off. Therefore it is imperative that site owners take necessary measures to scale their sites. With Zope, this is easy. Use Zope Enterprise Object, or ZEO for short – with an unresponsive site just add a Unix box, install ZEO Client and the rest is taken care of by ZEO. ZEO is also useful as a development tool and as a means of accessing Zope sites from the command line, which is very convenient.

What is ZEO? When we run stock Zope, only one Zope instance can access the Zope Object Database (ZODB) which is usually a FileStorage instance named Data.fs.

ZEO changes this behavior. With ZEO, many Zope instances (ZEO Clients, ZC) can access the database (ZEO Storage Server). Thus, we may have many different servers each running a Zope instance that acts as a ZEO Client, even with each box running more than one ZEO Client. These ZEO Clients then access the ZEO Storage Server for data. See Diagram 1.

ZEO is a client and server system, with ZEO Storage Server (ZSS) serving a consistent database to all ZEO Clients (usually Zope instances). In addition, ZEO implements a temporary or persistent on-disk cache that caches recently used objects, speeding up requests. The caches can also be used if the ZEO Storage Server dies. Changes in a ZEO Client are propagated to the ZSS. The ZSS invalidates outdated data in caches in other ZCs, thus synchronizing the data between ZCs.

A ZEO Storage Server needs to serve the database. Zeo Client will check its cache (if persistent cache is configured – if persistent cache is not selected the default setting is temporary cache) and verify it with the ZSS when it connects. Every client disconnect or crash results in validation when one restarts. Zope checks the index of file 0 and file 1 (the two cache files ZEO sets up – see the next paragraph), verifies the data with the server, invalidating data that is outdated.

Each ZC that is configured to use persistent cache will keep two cache files in the instance's home var directory. The files use the following convention c--digit.zec; where storage and client is from zope.conf. With the default cache size, the cache will first be written to file 0 (c1-clientname-0.zec). When the cache-size reaches (cache-size)/2 (or 20MB/2) (20 MB is the default cache size), the client starts writing to file 1 (c1-clientname-1.zec). When this file reaches 10MB, file 0 is recreated, and the new cache data is written to file 0. This continues forever.

If a ZSS serves out data using a public IP address, any Tom, Dick or Harry can connect and serve the data too (a ZC is a server to the end user, a client to the ZSS), unless an authentication scheme (based on HTTP digest authentication) is used. A more secure way is to run ZSS and ZCs in a private network, or run through a Virtual Private Network for servers that run in different geographical locations..

To install ZEO successfully, one needs to have a running Zope instance. This article describes installation for Zope 2.7 on a Unix box. Binary installation may differ in the path to the file, but the theory of operation is still the same.

Get the Zope tarball from http://www.zope.org/Products. To make things easier, we will use the following terminology:

• “zope source tree” for the path we untar the Zope tar ball to (e.g - /home/kdie/Zope-2.7.0-b3).
• Zope top-level binary directory – the directory where we installed Zope (the default is /opt/Zope-2.7, and can be specified with ./configure –prefix=/usr/local/zope2.7)
• Zope instance – where files for our Zope home will be installed (e.g /usr/local/zope)

Installation steps: For a successful installation, it is advised that the installation be done by the user who will run the Zope instance. In this example, we'll be using the username zopeuser.

1. untar the tar ball using your favorite method. tar -zxf /path/to/Zope-2.7.0-b3.tgz
2. change directory to the zope source tree
   cd /path/to/Zope-2.7.0-b3
3. run ./configure;make;make install
   ./configure –prefix=/usr/local/zope2.7;make;make install
4. now, change directory to the zope top level directory
   cd /usr/local/zope27
5. run bin/mkzopeinstance.py The directory specified here will be our Zope instance. Also, fill in the initial administrator/manager username and password. The default Zope instance directory contains configuration files and data, and will look something like this:

   |-- Extensions		#where External methods are stored
   |   `-- README.txt
   |-- Products		#where 3rd party products are stored
   |   `-- README.txt
   |-- README.txt
   |-- bin		#directory where the binaries/scripts are stored
   |   |-- runzope
   |   |-- runzope.bat
   |   |-- zopectl
   |   `-- zopeservice.py
   |-- etc		#directory that holds configuration data 
   |   `-- zope.conf
   |-- import		#directory where zexp are stored to be imported
   |   `-- README.txt
   |-- inituser
   |-- log		#logs for Zope 
   |   `-- README.txt
   `-- var		#directory for Data.fs and exported zexp
       `-- README.txt
   

6. It is advisable to run Zope in debug mode to clear all errors.
   /usr/local/zope$/bin/runzope

If everything is in order, we are now ready to install and run ZEO.

In previous Zope versions, ZEO was not included in the Zope tar ball. To run ZEO, we had to get the necessary tar ball (initially, ZEO was distributed as a tar ball). Then, to use ZEO it was necessary to extract it from the ZODB; which sometimes confused users.

As of Zope2.7, users no longer need to worry since ZEO is bundled with core Zope.

ZEO installation is not complex. Change to the zope top-level binary directory and run mkzeoinstance.py

	/usr/local/zope27$bin/mkzeoinstance /path/to/zope/instance 8800

The necessary files will be copied to the Zope instance directory. Note that the zeo.conf file is for ZSS configuration. Client configuration is done in zope.conf

The default zeo.conf is as follows:

# ZEO configuration file

%define INSTANCE /usr/local/zope

</folder>

<zeo>
  address localhost:8800
  read-only false
  invalidation-queue-size 100
  #monitor-address PORT
  # transaction-timeout SECONDS
</zeo>

<filestorage 1>
  path $INSTANCE/var/Data.fs
</filestorage>

<eventlog>
  level info
  <logfile>
    path $INSTANCE/log/zeo.log
  </logfile>
</eventlog>

<runner>
  program $INSTANCE/bin/runzeo
  socket-name $INSTANCE/etc/zeo.zdsock
  daemon true
  forever false
  backoff-limit 10
  exit-codes 0, 2
  directory $INSTANCE
  default-to-interactive true
  # user zope
  python /usr/bin/python
  zdrun /home/kdie/zope27/lib/python/zdaemon/zdrun.py

  # This logfile should match the one in the zeo.conf file.
  # It is used by zdctl's logtail command, zdrun/zdctl doesn't write it.
  logfile $INSTANCE/log/zeo.log
</runner>

Some of the storage options that can be changed:

• address
  The address the server listens to. It can be of “host:port” or “/path/to/file” notation. The former signifies TCP-IP setup, while the later means Unix socket.
• read-only
  signifies whether the server will run in read-only mode or not. The default is false.
• invalidation-queue-size
  The default is 100: this is the number of recent invalidation changes that are stored in memory. Higher size consumes more memory.
• monitor-address
  The address at which the monitor server listens for any monitor client connect. If specified, a monitor server is started. The monitor server provides server statistics in a simple text format.
• Transaction-timeout
  Since only one transaction can commit at any given time, the server needs to time out a long transaction to give other clients the opportunity to commit.

Once everything is set, we can test the ZSS by running

	/path/to/zope/instance/bin/runzeo

runzeo runs in the foreground and only stops with a Ctrl-Break.

To have a working ZEO Client, edit zope.conf and add the following:

# Temporary storage:
<zodb_db temporary>
    <temporarystorage>
      name sessions
    </temporarystorage>
    mount-point /temp_folder
    container-class Products.TemporaryFolder.TemporaryContainer
 </zodb_db>
#
# ZEO client storage:
 <zodb_db main>
   mount-point /
   <zeoclient>
     server localhost:8800
     storage 1
     name zeostorage
     client local
     cache-size 102400
     var $INSTANCE/var
   </zeoclient>
 </zodb_db>

With this setup, we will have a ZEO Client that uses a persistent cache of 100MB connecting to storage 1 of the ZSS.

The ZEO Client comes with some configurable parameters. Some of these parameters are:

• storage
  The name of the storage that the client wants to use. The ZSS may serve more than one storage. The default storage name is 1.
• cache-size
  The maximum size of the client cache, in bytes. The default is 20 MB. Setting this too small may affect performance.. Setting it too large will slow client connect since a client needs to validate its data.
• name
  The storage name. If unspecified, the address of the server will be used as the name.
• client
  Enables persistent cache files. The string passed here is used to construct the cache filenames. If not specified, only temporary cache files will be created. The temporary cache files will be deleted once the client stops.
• var
  The directory where persistent cache files are stored. By default, cache files, if they are persistent, are stored in the current directory.
• Min-disconnect-poll
  The minimum delay in seconds between attempts to connect to the server. The default value is 5 seconds.
• max-disconnect-poll
  The maximum delay in seconds between attempts to connect to the server. The default is 300 seconds.
• wait
  A boolean indicating whether the constructor should wait for the client to connect to the server and verify the cache before returning. The default is true.

To add more ZEO clients, install Zope as required, test the Zope instance and repeat the “Installing ZEO client” steps.

To test the zeo client, run

	/usr/local/zope$bin/runzope

Some debugging notice will fly by, and everything should be ok once we get:

2003-12-14T23:29:08 INFO(0) Zope Ready to handle requests

There are a few ways to verify that ZEO and Zope are running. The easiest way is to check the log files. By default, zeo.log is used to record all ZSS messages. Checking the contents of zeo.log will determine the success or failure of starting ZSS..

$tail /usr/local/zope/log/zeo.log
------
2003-12-15T14:26:34 INFO(0) RUNSVR opening storage '1' using FileStorage
------
2003-12-15T14:26:35 INFO(0) ZSS:2109 StorageServer created RW with storages: 1:RW:/usr/local/zope/var/Data.fs
------
2003-12-15T14:26:35 INFO(0) zrpc:2109 listening on ('', 8800)

To confirm that our ZEO client is working, check out the contents of the event.log:

/usr/local/zope$tail var/event.log
------

2003-12-15T14:27:00 INFO(0) ZEC:/usr/local/zope/var/c1-local-1.zec flipping cache files.  new current = 1

------
2003-12-15T14:27:00 INFO(0) Zope Ready to handle requests

Now that everything is running as expected, we should use zopectl and zeoctl to start and stop the ZSS and the ZEO client.

Starting ZSS with zeoctl

/usr/local/zope$bin/zeoctl start 

Starting the ZEO client with zopectl

/usr/local/zope$bin/zopectl start 

Now we can check the status using zeoctl and zopectl

/usr/local/zope$bin/zeoctl status
program running; pid=2349
/usr/local/zope$ bin/zopectl status
program running; pid=2353

There are many ways we can balance the requests to the Web sites once the cluster comprising the ZSS and ZEO clients is operational.

Some of the many ways to load balance the cluster:

• Use a commercial load balancer product, from commercial vendors such as Cisco Systems, F5 Labs, Zeus and many others
• balance , available at http://balance.sf.net
• DNS roundrobin
• Linux Virtual Server, at http://www.linuxvirtualserver.org

One great benefit of running Zope with ZEO is the ability to debug and access Zope interactively. For example, we may resolve Zope Management Interface lockup using such a method. Accessing Zope interactively also makes certain automatic operations, such as packing or backing up the database, much easier.

1. Go to the Zope instance home
   

          cd /usr/local/zope
   

2. Run bin/zopectl debug

   /usr/local/zope$bin/zopectl debug 
   

3. Do what is necessary, for example pack the database

   /usr/local/zope$bin/zopectl debug   
   Starting debugger (the name "app" is bound to the top-level Zope object)
   >>> app.Control_Panel.Database.main.manage_pack(days=1)
   

   We can introspect all objects using Python “dir”, use __doc__ to get the doc strings, and do anything that we can do with normal Python objects. This is where the ZEO coolness comes in.

4. We need to perform one more bit of Zope magic to commit the changes.

   >>> get_transaction().commit()
   >>> app._p_jar.close()
   

Zope 2.7 comes with batteries included. With ZEO, we can scale sites easily by chucking in more processing power. ZEO also makes debugging and automating Zope easier.

This basic example was kept very simple, covering the basic goals:

• Model your content using XML
• Produce a view of the content using XSLT

The details to accomplish these goals:

Zope Book - Scalability and ZEO