Illustration by Lia Avant

Zope and SOAP

SOAP enables developers using different programming languages to write programs that interact with each other, despite different data types and structures. It is an XML-based protocol that defines the standards for encoding the messages that are sent between a Web service and a client.

This article describes how to create your own Web service and how to make use of existing Web services, i.e. how to send data to and receive data from a Web service. This is shown using the Zolera SOAP Infrastructure (ZSI), a Python package that provides an implementation of SOAP messaging.

Note: If the use of "Web service" seems inconsistent in this article it is because in the literature the expression is used both for the method called and for the interface to the method called, which serializes and deserializes the SOAP message.

Today's Website visitors demand more than simple HTML pages with pictures. Therefore, Websites have, over the years, developed into Web applications, offering more and more services, such as online shopping, online banking or personal member areas, where visitors to a Website can choose from among several modules (e.g. News, Weather, etc.). These services are wide-ranging, and although Python is a powerful language, sometimes a service is best programmed using a language other than Python, for instance Java. In order to interact with this other language, there needs to be an interface that Python can access. These interfaces are called Web services.

In this article, I will describe the results of my attempts at creating a Web service on one Zope server and calling the Web service from a different Zope server. This will hopefully help you to create your own Web services using Zope, or at least enable you to make use of existing Web services.

After some research, I decided to use the Zolera SOAP Infrastructure (ZSI) since most people on the different mailing lists seemed to think it was the most potent implementation for Python and I had some sample code from a colleague to work with to get me started. The ZSI comes with some documentation and a range of unit tests, which also helped. For the Web service part, I adapted a Zope patch by guylux and the Web service example from the ZSI documentation. This Web service receives a SOAP document containing a Player object with a name and a list of scores. It calculates the average of the scores and sends back that number.

SOAP and corresponding Web services allow clients to access methods on a different server, in particular independent of the client and server languages. It is able to receive data packages through various standard Internet transportation protocols, such as HTTP, and transform them into the programming language that the Web service is written in. This transformation is called "deserialization".

In order for the Web service to correctly understand the data packages, they must have been encoded, i.e. serialized, according to a standard packaging protocol. SOAP is such a packaging protocol. To the Web service, it doesn't matter what programming language the client application is written in that sends it a SOAP-encoded package. As long as the package is serialized correctly and the data encoded in it conforms to what the Web service offers, everything will be fine.

In addition to receiving data packages, the Web service is also able to send back serialized packages to the client application. That way, a client application is able to call functions or methods written in a different language (and on a different server) and, if necessary, receive their results.

Figure 1: SOAP Messaging via HTTP

SOAP is an application of the XML specification and therefore uses this standard for its definitions and functions. If you are unfamiliar with XML, you may want to learn a bit about it before starting on Web services.

Listing 1: Sample SOAP Message

As you can see, SOAP messages are enclosed in envelope tags (lines 1-12). The envelope-tag contains information about the standards used for encoding (lines 1-6). Within the envelope there may be a header part (not included in this code example) and there must be a body part (lines 7 – 11). Because of the XML standard, every tag in a SOAP message must have a start and an end.

Within the body part, you define as many child nodes as you need. In this example, there is only one child node: AnInteger.

With this section I just wanted to give you an idea of what is being sent back and forth between a Web service and a client. See the various SOAP books or the SOAP specification for detailed information.

The Zolera SOAP Infrastructure (ZSI) is one of several Python implementations for SOAP messaging. ZSI requires Python 2.0 or later and PyXML version 0.6.6 or later.

ZSI converts between native Python datatypes and SOAP syntax and allows you to create complex structures using the different typecodes available. Typecodes are the datatypes which define how a Python object is converted to SOAP and vice versa. Besides the normal datatypes such as strings and integers, which are defined in the main ZSI typecode module TC, there are various others available in the modules TCapache, TCcompound, TCnumbers and TCtimes.

To create your own datatypes, you create a class with constructor method (__init__) and assign it a typecode. I will explain this in more detail in the sections below.

For a detailed description of how ZSI does the things it does, see the documentation that comes with it.

In this section, I will go through the code for the client which calls the Web service. There is also a Zope product for SOAP messaging called SOAP Method (see Section "What is there besides ZSI?" below) but in cases where you need to serialize your own data types (more complex types than just a string or an integer, for example) or for some Java data types (for example, Java HashMaps: see below under "Example for Sending Dictionaries to a Java Web Service"), you may need to call the Web service from Python using ZSI (or some other SOAP implementation) since ZSI is more powerful than SOAPMethod.

Remember, throughout this article I will only explain how to access a Web service using ZSI. Other Python implementations for SOAP do things differently. Moreover, I will explain things using External Methods, not a Zope product or Python scripts.

There are a couple of steps needed to call a Web service with ZSI:

1. Create a connection to the server which the Web service is hosted on.
2. Create a request class for each method (or at least for the methods that have different parameters).
3. Create a typecode for each request class.
4. Create methods that use the connection to call the Web service's methods and return the Web service's response, if necessary.

First, let me explain what my client does. As I said above, I adapted one of the examples from the ZSI documentation. I have a form in Zope in which you can enter a player name and three scores for the player. The form data is sent to an External Method called getPlayerAverageMethod which connects the method getPlayerAverage to Zope. This method (see Listing 3) converts the form data into a SOAP message, connects to the Web service on another Zope server and returns the Web service's response, which is the calculation of the average of the three scores (see Fig. 2).

Figure 2: Workflow on the client side

Listing 2 shows the form for entering the data. Sorry, I'm still kind of a DTML guy so this is a DTML Document. However, creating this form using ZPTs shouldn't be too difficult.

The score data is sent to the External Method as a list of integers. Note that there must be a hidden input field named HTTP_SOAPACTION which needs to have a true value, in this case 1. I will explain why later in the section "Patching the ZPublisher".

Listing 2: Form for entering the request data

The first step is creating the connection to the Web service using the Binding class provided in ZSI's module client. The class' constructor needs to know at least the server (e.g., '192.168.0.44'), port (e.g., '8080') and URL (e.g., '/Webservice/showPlayerAverageMethod') to the Web service (lines 5-7, Listing 3). There are more parameters that can be used to create a Binding object, but this is all we require here. They are described in the ZSI client module.

For reasons that I haven't figured out yet, the URL must contain the method that is being called, in this example the method showPlayerAverageMethod, which is an External Method on the Zope server that provides the Web service. I assume this is because the Web service I created works slightly differently than a "normal" Web service where you register your Web service methods with the server so that they are known to the server as long as the Web service is running.

The next part is to create the request class (lines 8-12, Listing 3) and assign it a typecode (lines 13-18, Listing 3). The request class and typecode are used to serialize data. In this example, the data consists of a player's name and three scores. Therefore, the class constructor must initiate and store these values for each class instance. In this case, the scores are stored as a list.

Since the data, i.e. the class, is a complex datatype (because it combines a string and an array with integers) I use the Struct class defined in the module TCcompound to create the typecode. The arguments for the Struct class are: the class the data is based on (here: Player class), a list of the typecodes representing the data stored in the class (lines 15-17, Listing 3), and the name for the element in the SOAP message (line 18, Listing 3). More parameters are possible – see TCcompound.py. Note that the list of scores is defined here as an array (line 16, Listing 3).

Listing 3: The Web service client

Next comes the method that creates a Player instance from the form data (line 26, Listing 3) and calls the remote procedure, i.e. the Web service, via the Binding object (line 30). The parameters for the remote procedure call are the URL to the Web service, the name of the Web service method (in this example, this name is already contained in the URL), the object that is to be serialized and sent out, and the type the response will have. Again, there are various other parameters that the RPC method can use, e.g. the trace_file parameter, but these are not necessary for the function to work.

The result returned by the RPC method is a list containing the values of the child elements contained in the SOAP message returned by the Web service. Since I know that in my example the response will contain only one child element – the average of the scores – I take the first (and sole) element of this list (line 33, Listing 3) and return it.

The following is an example of a serialized SOAP message created from sample form data:

Figure A: Example of SOAP message serialized from form data

During a project a colleague of mine needed to call functions on a Web service written in Java. Most of the calls were made using the SOAPMethod product; however, there were some problems sending dictionaries where Java HashMaps were expected by the Web service. Therefore, he needed to look into other ways to call Web services and found a way to do it with ZSI.

Here is the code for the request class and the assigned typecode:

Figure B: Code for the request class

The dictionary is converted to SOAP using the Apache Map class defined in the module TCapache.py.

For the Web service, I used the Web service example from the ZSI documentation and put it in a method called showPlayerAverage in a module called webservice.py. This method is integrated into Zope via an External Method object called showPlayerAverageMethod.

Figure 3: Workflow on the Web service side

In the Web service method, I read out the variable soap_data (lines 10-11, Listing 4a) which contains the incoming SOAP message (see the section below "Patching the ZPublisher" for more information on soap_data). The SOAP message is then used to initiate an instance of the class ParsedSoap which is defined in the ZSI module parse.py (line 16, Listing 4a). If there is an error instantiating ParsedSoap an exception is thrown. The output of this exception is stored in the buffer created in line 14. This buffer is used to convert the output into a SOAP message (line 18, Listing 4a), i.e. it is serialized. Using this serialized output the response object is changed to contain the output as body (line 19), and the content type of the response object is changed to 'text/xml' (line 20). The response object is then sent back to the client (line 21). This method of creating a response object is used a number of times throughout the showPlayerAverage method, not just for exceptions but also for the desired end result – the SOAP message containing the average of the scores given in the incoming request..

Note that when setting the content type in the header of the request (e.g., in line 20), the third argument must be zero so that the current response header for the content type is replaced by the new one.

Listing 4a: First part of the Web service

After some more parsing and exception catching, we come to the part where the actual Web service, i.e. calculation of the average score, is carried out. The part up to here (line 43) would actually be better located in the ZPublisher module HTTPRequest.py so that it does not have to be repeated for each Web service method. But for this single Web service example, I left it here.

Next comes something you partially know already from the client code: a request class (Player) with its typecode and a response class (Average) with its typecode. The request class is needed again because the incoming SOAP message has to be parsed (line 60, Listing 4b) so that the elements Name and Scores are extracted and can subsequently be used. Note that the constructor of the Player class does not have to be complete, as it had to be in the client code, since I don't instantiate the class.

Lines 66 and 67 show a support method which divides the sum of the Player's scores by the number of scores given. The sum is calculated using the add method of the operator module (lines 69 and 70, Listing 4b). The reduce method takes a function (here: operator.add), a sequence (here: player.Scores) and an optional starting value (here: 0). The function is then iteratively applied to the sequence elements; first, the first two values are added together, then the result is added to the third value, and so on.

In line 71, an instance of the class Average is created using the average calculated by the support method foo. The Average instance is then converted into a SOAP message, i.e. serialized. Finally, the SOAP message is put into the body of the response object, the content type is set to 'text/xml' and the response is sent back to the client (lines 78-80).

Listing 4b: The second part of the Web Service

Here is a sample SOAP response message:

Figure C: Example of a SOAP message serialized by the Web Service

The value 207 is the answer to a request for the average of 3 values (not the values shown above in "Example of SOAP message serialized from the form data").

Guylux's patch for the ZPublisher consists of three files: a patched version of the file HTTPRequest.py, a SOAP support module (soap.py) and a SOAP client library (soaplib.py). The patch in the HTTPRequest.py uses the files soap.py and soaplib.py. This didn't seem to work with the SOAP message created by ZSI so I changed two lines in the following snippet from the patch:

Figure 5: Snippet of the patched module HTTPRequest.py required to work with the ZSI Web Service

In these patched versions you can see why the form earlier (on the client side) needed the hidden input field HTTP_SOAPACTION. This variable is used to determine that the incoming request is a SOAP message and to deal with it accordingly.

If the Zope server receives a request and it contains the environ variable HTTP_SOAPACTION, the body part of the request (which contains the SOAP message) is put into the request variable soap_data. Remember, this variable is used in the method showPlayerAverage() to get to the actual SOAP message.

I guess it's possible to do the deserializing in HTTPRequest.py and not in the Web service method (showPlayerAverage()), as is done in guylux's code (line 357), but this was the easiest way for now.

Besides ZSI there are several other Python implementations of SOAP, such as SOAPpy and soaplib, and there is a library of classes concerning WSDL called wstools. As mentioned above, soaplib was used by guylux for his ZPublisher patch, so you may want to look into that for your Web service.

A while ago, there were some efforts by Brian Lloyd to program a WebServices package for Zope. Unfortunately, this package was abandoned due to lack of demand and supporters.

I mentioned earlier the Zope product SOAPMethod. This product can be downloaded from Zope.org; it was created by EIONET.

Figure 4: The form for adding a SOAPMethod object in Zope

This product might be enough for your (client side) SOAP messaging, so you may want to try it before delving into the deeper concepts.

The Web service described in this article is not really fully developed. As I said above, doing the serializing within the module HTTPRequest.py might be better than doing it in every Web service method. But I believe the Web service described above is a good start and will hopefully help you to create your own Web services or at least help you understand how Web services work. Hopefully someday SOAP messaging will be fully integrated into Zope and patches will no longer be necessary.

Simple Object Access Protocol (SOAP) 1.1 Specification:
http://www.w3.org/TR/SOAP/

Zope patch for Web services by guylux
http://www.zope.org/Members/guylux/Zope-soap

The SOAP Method Product:
http://www.zope.org/Members/guylux/Zope-soap

Overview of Python Web services, SOAPpy, wstools, ZSI:
http://pywebsvcs.sourceforge.net/

Webservice package for Zope, CVS: abandoned for now:
http://http://cvs.zope.org/Packages/WebService/

Wiki Frontpage of the WebServices package
At dev.zope.org

SOAP for Python by Secret Labs, soaplib:
http://http://www.pythonware.com/products/soap/