To start a CherryPy application begins with a startup script for assigning all your controllers and generally telling it what to do. How your application runs is greatly dependent upon how this script operates. By default the CherryPy tutorial doesn’t allow for RESTFul routing or allowing the controllers access to the configuration files as they are imported.

I’m required to use different environments for building an app. Working locally to develop it I need development settings. Once done I send it off to QA which needs those settings. Finally in production it needs settings for that as well additional requirements as not to show tracebacks. This is easily done in the deploy script.

One of the much touted announcements that came from the release of CherryPy 3.1 stable was something called the “cherryd” script in which you could daemonize your application instead of running it under mod_python or as a background job. I looked at the script and was horrified to realize that it was so completely different from the deploy scripts I had written.

Then I looked at it even closer, following each line and suddenly realized that the cherryd was not a be all deploy script, it was merely a working script which was basic in it’s use. Having realized that only a few parts of it were needed to fully daemonize the deploy scripts I’ll show below while maintaining the RESTFul resources.

The CherryPy Example 2 code files for you to follow along. I’ve commented heavily in them for what each part does.

Before we begin with the actually deploy script let’s look at the configuration files themselves. I’ve intentionally left them very small but distinct enough so you can see how they work when running your app under the different environments.

The first configuration is the config/site_config.py. This holds all the settings common to all your running environments. Whether running in development or production the site_config will be the same for both.

site_config.py

[global]
server.thread_pool = 10
server.socket_host = "0.0.0.0"

The server.thread_pool is how many threads should your cherrypy app handle during operation. Having more threads allows your app to handle more requests and perform faster under heavy loads. I’m running applications that have 400 threads. On some machines I’ve had to lower that as they could handle so many of them. Play with it for the best performance of your application.

The server.socket_host set to 0.0.0.0 simply allows your app to run on any IP without you needing to assign it. Certainly this can be changed to 127.0.0.1 or even any other IP address. Leaving it at 0.0.0.0 means I don’t have to worry about it.

The deploy script first loads the site_config. After which it loads the environment config based upon what you specified in the startup command.

development.py

[global]
server.environment = "development"
server.socket_port = 7180

qa.py

[global]
server.environment = "production"
server.socket_port = 7181

production.py

[global]
server.environment = "production"
server.socket_port = 7182

As you can see each file holds the same settings. Yet these setting are different. Looking at the server.socket_port port assignment you can see that I can be developing the app while running the production app on the same machine since the running ports are different.

So now that we have the configuration files out of the way let’s look at the deploy script. The example deploy.py is complete, I’ll only discuss the relevant parts to load the configuration files and starting the app here.

deploy.py (partial)

if __name__ == '__main__':

    p = OptionParser()
    p.add_option('-d', action="store_true",
                 dest='daemonize', help="run the server as a daemon")
    p.add_option('-e', '--environment',
                 dest='environment', default=None,
                 help="apply the given config environment (development, staging, qa, production)")
    p.add_option('-p', '--pidfile',
                 dest='pidfile', default=None,
                 help="store the process id in the given file")

    options, args = p.parse_args()

    if not options.environment in ["development", "staging", "qa","production"]:
        print "A running environment is required"
        exit(1)

    """Load up the configuration files first before loading any
    controllers."""
    load_config(options.environment)

    """Since we've now loaded the configuration files we can load
    the controllers so they have access to the cherrypy.config.get(..)
    No need to use any PyYAML. :)"""
    server_start()

    """When we are running in production mode we don not want tracebacks
    Additionally we want a redirect to the corresponding error page."""
    if options.environment == "production":
        cherrypy.config.update({'error_page.default': \
                                "%s/public/index.html" % (os.path.abspath("."))}
                              )
        cherrypy.config.update({'error_page.401': \
                                "%s/public/errors/401.html" % (os.path.abspath("."))}
                              )
        cherrypy.config.update({'error_page.404': \
                                "%s/public/errors/404.html" % (os.path.abspath("."))}
                              )
        cherrypy.config.update({'error_page.500': \
                                "%s/public/errors/500.html" % (os.path.abspath("."))}
                              )

    cherrypy.config.update({'log.error_file': \
                            "%s/logs/%s_error.log" % (os.path.abspath("."), options.environment)}
                          )
    cherrypy.config.update({'log.access_file': \
                            "%s/logs%s_access.log" % (os.path.abspath("."), options.environment)
                           }
                          )

    engine = cherrypy.engine

    """The plugins.Daemonizer handles all the daemon process setup for us.
    Additional processes and configs setup here when running under daemon"""
    if options.daemonize:
        cherrypy.config.update({'log.screen': False})
        plugins.Daemonizer(engine).subscribe()

    """Writes a pid file to the directory specified in the:
    -p /var/run/my_app.pid CLI option"""
    if options.pidfile:
        plugins.PIDFile(engine, options.pidfile).subscribe()

    """Setup the signal handler to stop the application while running"""
    if hasattr(engine, "signal_handler"):
        engine.signal_handler.subscribe()
    if hasattr(engine, "console_control_handler"):
        engine.console_control_handler.subscribe()

    """Finally start the app with engine.start(). Any kill signals
    are handled in the except. So if you need any other cleanup
    processes that need handled on shutdown put them there."""
    try:
        engine.start()
    except:
        sys.exit(1)
    else:
        engine.block()

So first off notice that we are going to run this from the command line “python ./deploy.py” This will never be imported for use elsewhere. To help out it also uses the OptionParser module to make it easier for command line assignment:

python ./deploy.py -e development  (Run the development configuration)
python ./deploy.py -e qa -d -p /var/log/qa.pid (Run the qa configuration as a daemon with pid file)

So with the above commands we’ve instructed the application to load the specific config file we need. That the job of function: load_config(options.environment).

I’m skipping the server_start function for now to revisit it when we discuss controllers.

Notice:

"""When we are running in production mode we do not want tracebacks
    Additionally we want a redirect to the corresponding error page."""
    if options.environment == "production":
        cherrypy.config.update({'error_page.default': \
                                "%s/public/index.html" % (os.path.abspath("."))}
                              )
        cherrypy.config.update({'error_page.401': \
                                "%s/public/errors/401.html" % (os.path.abspath("."))}
                              )
        cherrypy.config.update({'error_page.404': \
                                "%s/public/errors/404.html" % (os.path.abspath("."))}
                              )
        cherrypy.config.update({'error_page.500': \
                                "%s/public/errors/500.html" % (os.path.abspath("."))}
                              )

    cherrypy.config.update({'log.error_file': \
                            "%s/logs/%s_error.log" % (os.path.abspath("."), options.environment)}
                          )
    cherrypy.config.update({'log.access_file': \
                            "%s/logs%s_access.log" % (os.path.abspath("."), options.environment)
                           }
                          )

This section is special. If we are running the production environment then we want to setup the error handling pages for our application. Under all environments we also need to setup the corresponding log files for access or error. Why didn’t this get included in the configuration files?

Notice the os.path.abspath. In the configuration files we can not import other modules. Only use python functions that don’t need importing. Since other scripts can start the deploy script it’s difficult to tell what the file path would be in the config files unless you specifically say “/PATH_TO_DIRECTORY/logs/production_access.log”. That however, is not very portable is it.

This rest of the file has the comments on what each part means.

Give it a try. Run two commands:
python ./deploy.py -e development
python ./deploy.py -e production

view them at http://localhost:7180/ and http://localhost:7182/ . Notice that both give you the index.html page and that both are running. Now go to http://localhost:7180/nothing and http://localhost:7182/nothing.

Now do you see how the different environment config files come into play. The error for the development app shows you the error (404) tracebacks, whereas the production shows you the 404 html page instead.

Probably the most important file a CherryPy app requires is the deploy script. As developers we need a way to build in one environment and run production in another. This can be incorporated in the deploy script fairly easily and once done we can forget about it. Being able to daemonize the CherryPy app leads to huge performance increases. The cherryd script is merely a basic working script which we can incorporate into our own deploy scripts.

In the next post I’ll show you how to incorporate RESTFul controllers and custom named routes into the deploy script for a well rounded application.

Tags: cherrypy, cherrypyd, script