In order to handle a large site, Zope needs lots of disk space, memory, and CPU resources to do its job. Usually, Zope does this as quickly and efficiently as it can with the resources you give it. When you begin to customize your site in Python, however, you may end up noticing that your code is rather slow. What has happened?

First, you should look at the obvious: Do you have enough resources? Are you accidentally running Zope in debug mode? Are you using the ZCatalog to do your searches quickly, or are you slowly searching lots of objects by hand? Is your server big enough to handle your load? Is your network bandwidth fast enough? Before taking the expensive time to analyze and modify your software to speed it up, it might be worth it to upgrade your hardware first. You have to weigh the numbers.

If it looks like you have enough resources and it's your programs that are slowing things down, it's quite possible you have poorly optimized code. Lots of common programming mistakes can slow down your Zope site. There are a number of tricks in the code optimizer's bag to cure these problems, but before you can do this, of course, you have to know what the problem is. And to know what the problem is, you need a tool that can analyze your program while it is running and tell you exactly at which points in your code things are slowing down.

Such a tool is called a *profiler*. When your Zope site is running, it is possible for the underlying Virtual Machine to keep track of statistics about your code, such as how many times your methods were called, who called them, and how much time the calls took. This information is very valuable when you are trying to determine the resource consumption of your program.

Profiling is not free: when you run Zope under the profiler your Zope site will run *much* slower. Collecting and storing the statistical data for each and every method and function call in your Zope site takes a lot of CPU time and memory, so profiling should only be applied on a development system, not a "production" system. In the second article I'll show you how to set up a separate ZEO client for a site that can profile a system while other ZEO clients run normally.

It is important to know that a profile is not an analysis of your site design or code; it only provides *statistical observations* about your site while it is running. As when the police profile a suspect in a crime, Python's profiler lets you profile your own "suspect" code, but like the police you'll only catch the right guy if you look in the right place and interpret the information correctly.

Profiling can often be tricky because your code may cause *other* code to be called without its being obvious. For example, sloppy page templates and Python scripts can cause the security system to be called too frequently. When you profile this behavior, it may look to you like the security system is the bottleneck, when in reality it's your code. So you have to be careful. The example below shows how to detect this situation.

Python's HotShot profiler is used in this article to show how to analyze a Zope system overall, but there are two other products worth mentioning that profile specific parts of Zope.

1. The 'Page Template Profiler', as the name suggests, profiles page templates at the Tag Attribute Language (TAL) level. This tool is especially useful if your application uses many, many templates. But keep in mind that if your TAL code is so complex that you need to profile it, you might want to think about moving some of that complexity into Python.
2. The 'Call Profiler' is a more unique tool because it traces individual requests through Zope, measuring the time it spends on each object to satisfy a request.

Using these specific profilers is beyond the scope of this article, but reading this article will help you understand the concepts of using any profiler, for almost any language. After you are comfortable with the concepts presented here, I encourage you to go out and explore more of the science and art of profiling.

Python's new HotShot profiler was developed by Zope Corporation to address many deficiencies in the older Python profiler. In particular, the HotShot profiler is written in C, thus removing one of the biggest problems with the old profiler which was written in Python: this caused such a large drag on the interpreter that it often greatly skewed the results. Many other improvements have also been made in the rewrite.

Zope Corporation has made it very easy to profile Zope. To turn on profiling, you must carry out a simple editing operation on your Zope configuration file. Where this file is depends on how your Zope is configured and installed, but in a "standalone" installation, you can find it in the 'etc/' directory of your Zope source tree or, for multi-host installations, in the 'etc/' directory of each host's "instance home". Once you have determined the right file, simply search for the string "profile". You will find the following configuration option:

# Description:
# Names a file on the filesystem which causes Zope's Python
# profiling capabilities to be enabled. For more information, see
# the Debug Information - > Profiling tab of Zope's Control_Panel
# via the Zope Management Interface. IMPORTANT: setting this
# filename will cause Zope code to be executed much more slowly
# than normal. This should not be enabled in production.
#
# Default: unset
#
# Example:
#
# publisher-profile-file $INSTANCE/var/profile.dat


Note that the last line is commented out. To turn on profiling, just uncomment this line. The value of this configuration option points to a file where the statistical information will be dumped. In this article we'll be looking at the profile information in real-time, so we won't need this file, but it will be there in case you want to analyze profile data "off-line" or send your profile data to another person.

So, now that you have correctly set the above configuration option, start up your Zope server and log into the Control Panel (you may notice that your site now runs two or three times slower, this is the cost of profiling!). From the Control Panel, go to the 'Debug Information' link. This takes you to a page with two tabs along the top. Click on the one to the right called 'Profiling' to get the following screen:

Zope Profiler Screenshot

On the full screen there is *a lot* of information. Only a small amount of it is shown above for formatting reasons. Before going into the details of what the screen means – which will be done in Part 2 of this article – I want to talk a bit more about the theory of profiling. Profiling is much more than just turning on a variable and looking at this screen. The question now is What are you going to profile? What does this information really mean? Let's step back and try to get a hold on what really needs to be done.

Profiling is statistical analysis. As mentioned above, you collect lots and lots of data about how often your methods are called, how long they take, who called them, and who they call. What you're looking for is whether you can discern in all this information any patterns of excessive resource consumption. In order to make the best analysis you need the best possible data, and *lots* of it. You can't just run your Zope system for a couple of requests and expect the profiler to show you anything of significance - you have to run many requests.

Further, if you have just a general feeling that your site is slower than it should be overall, then you might try to simulate lots of real-world requests on your Zope server while it is being profiled. This method, however, will end up profiling a lot of Zope that you probably have no interest in, and this mass of "extra" useless data will drown out the important data that you are looking for. If you're a site developer, not a core Zope developer, you want to profile *your* code, not the core code of Zope.

If you want to know whether a specific chunk of Python is performing poorly, you will get the best information out of your profile data if you carefully craft your requests so that they only "exercise" your particular bits of code, over and over again, till the patterns in the data you are collecting become statistically significant and the "noise" (profiled data you're not interested in) washes out. This method will allow you to concentrate on and analyze more relevant data.

Profiling can be compared to a doctor making a diagnosis. Doctors generally do not begin examining your entire body as soon as you enter their office. First, they ask questions, they try to "target" exactly where they should examine you. Doing a full body examination up-front is almost always a waste of time and resources. Profiling is analogous: in order to find the problem, you must first know what part of Zope's "body" you are examining and focus your profiling efforts on that one part.

In the next article in this two-part series, I will show you not only how to interpret the above information, but how you can benchmark and profile your Zope application to improve its run-time performance. Come back in the next issue of Zope Magazine to find out how this is done.