To start a CherryPy application begins with a startup script for assigning all your controllers and generally telling it what to do. How your application runs is greatly dependent upon how this script operates. By default the CherryPy tutorial doesn’t allow for RESTFul routing or allowing the controllers access to the configuration files as they are imported.

I’m required to use different environments for building an app. Working locally to develop it I need development settings. Once done I send it off to QA which needs those settings. Finally in production it needs settings for that as well additional requirements as not to show tracebacks. This is easily done in the deploy script.

One of the much touted announcements that came from the release of CherryPy 3.1 stable was something called the “cherryd” script in which you could daemonize your application instead of running it under mod_python or as a background job. I looked at the script and was horrified to realize that it was so completely different from the deploy scripts I had written.

Then I looked at it even closer, following each line and suddenly realized that the cherryd was not a be all deploy script, it was merely a working script which was basic in it’s use. Having realized that only a few parts of it were needed to fully daemonize the deploy scripts I’ll show below while maintaining the RESTFul resources.

The CherryPy Example 2 code files for you to follow along. I’ve commented heavily in them for what each part does.

Before we begin with the actually deploy script let’s look at the configuration files themselves. I’ve intentionally left them very small but distinct enough so you can see how they work when running your app under the different environments.

The first configuration is the config/site_config.py. This holds all the settings common to all your running environments. Whether running in development or production the site_config will be the same for both.

site_config.py

[global]
server.thread_pool = 10
server.socket_host = "0.0.0.0"

The server.thread_pool is how many threads should your cherrypy app handle during operation. Having more threads allows your app to handle more requests and perform faster under heavy loads. I’m running applications that have 400 threads. On some machines I’ve had to lower that as they could handle so many of them. Play with it for the best performance of your application.

The server.socket_host set to 0.0.0.0 simply allows your app to run on any IP without you needing to assign it. Certainly this can be changed to 127.0.0.1 or even any other IP address. Leaving it at 0.0.0.0 means I don’t have to worry about it.

The deploy script first loads the site_config. After which it loads the environment config based upon what you specified in the startup command.

development.py

[global]
server.environment = "development"
server.socket_port = 7180

qa.py

[global]
server.environment = "production"
server.socket_port = 7181

production.py

[global]
server.environment = "production"
server.socket_port = 7182

As you can see each file holds the same settings. Yet these setting are different. Looking at the server.socket_port port assignment you can see that I can be developing the app while running the production app on the same machine since the running ports are different.

So now that we have the configuration files out of the way let’s look at the deploy script. The example deploy.py is complete, I’ll only discuss the relevant parts to load the configuration files and starting the app here.

deploy.py (partial)

if __name__ == '__main__':

    p = OptionParser()
    p.add_option('-d', action="store_true",
                 dest='daemonize', help="run the server as a daemon")
    p.add_option('-e', '--environment',
                 dest='environment', default=None,
                 help="apply the given config environment (development, staging, qa, production)")
    p.add_option('-p', '--pidfile',
                 dest='pidfile', default=None,
                 help="store the process id in the given file")

    options, args = p.parse_args()

    if not options.environment in ["development", "staging", "qa","production"]:
        print "A running environment is required"
        exit(1)

    """Load up the configuration files first before loading any
    controllers."""
    load_config(options.environment)

    """Since we've now loaded the configuration files we can load
    the controllers so they have access to the cherrypy.config.get(..)
    No need to use any PyYAML.  """
    server_start()

    """When we are running in production mode we don not want tracebacks
    Additionally we want a redirect to the corresponding error page."""
    if options.environment == "production":
        cherrypy.config.update({'error_page.default': \
                                "%s/public/index.html" % (os.path.abspath("."))}
                              )
        cherrypy.config.update({'error_page.401': \
                                "%s/public/errors/401.html" % (os.path.abspath("."))}
                              )
        cherrypy.config.update({'error_page.404': \
                                "%s/public/errors/404.html" % (os.path.abspath("."))}
                              )
        cherrypy.config.update({'error_page.500': \
                                "%s/public/errors/500.html" % (os.path.abspath("."))}
                              )

    cherrypy.config.update({'log.error_file': \
                            "%s/logs/%s_error.log" % (os.path.abspath("."), options.environment)}
                          )
    cherrypy.config.update({'log.access_file': \
                            "%s/logs%s_access.log" % (os.path.abspath("."), options.environment)
                           }
                          )

    engine = cherrypy.engine

    """The plugins.Daemonizer handles all the daemon process setup for us.
    Additional processes and configs setup here when running under daemon"""
    if options.daemonize:
        cherrypy.config.update({'log.screen': False})
        plugins.Daemonizer(engine).subscribe()

    """Writes a pid file to the directory specified in the:
    -p /var/run/my_app.pid CLI option"""
    if options.pidfile:
        plugins.PIDFile(engine, options.pidfile).subscribe()

    """Setup the signal handler to stop the application while running"""
    if hasattr(engine, "signal_handler"):
        engine.signal_handler.subscribe()
    if hasattr(engine, "console_control_handler"):
        engine.console_control_handler.subscribe()

    """Finally start the app with engine.start(). Any kill signals
    are handled in the except. So if you need any other cleanup
    processes that need handled on shutdown put them there."""
    try:
        engine.start()
    except:
        sys.exit(1)
    else:
        engine.block()

So first off notice that we are going to run this from the command line “python ./deploy.py” This will never be imported for use elsewhere. To help out it also uses the OptionParser module to make it easier for command line assignment:

python ./deploy.py -e development  (Run the development configuration)
python ./deploy.py -e qa -d -p /var/log/qa.pid (Run the qa configuration as a daemon with pid file)

So with the above commands we’ve instructed the application to load the specific config file we need. That the job of function: load_config(options.environment).

I’m skipping the server_start function for now to revisit it when we discuss controllers.

Notice:

"""When we are running in production mode we do not want tracebacks
    Additionally we want a redirect to the corresponding error page."""
    if options.environment == "production":
        cherrypy.config.update({'error_page.default': \
                                "%s/public/index.html" % (os.path.abspath("."))}
                              )
        cherrypy.config.update({'error_page.401': \
                                "%s/public/errors/401.html" % (os.path.abspath("."))}
                              )
        cherrypy.config.update({'error_page.404': \
                                "%s/public/errors/404.html" % (os.path.abspath("."))}
                              )
        cherrypy.config.update({'error_page.500': \
                                "%s/public/errors/500.html" % (os.path.abspath("."))}
                              )

    cherrypy.config.update({'log.error_file': \
                            "%s/logs/%s_error.log" % (os.path.abspath("."), options.environment)}
                          )
    cherrypy.config.update({'log.access_file': \
                            "%s/logs%s_access.log" % (os.path.abspath("."), options.environment)
                           }
                          )

This section is special. If we are running the production environment then we want to setup the error handling pages for our application. Under all environments we also need to setup the corresponding log files for access or error. Why didn’t this get included in the configuration files?

Notice the os.path.abspath. In the configuration files we can not import other modules. Only use python functions that don’t need importing. Since other scripts can start the deploy script it’s difficult to tell what the file path would be in the config files unless you specifically say “/PATH_TO_DIRECTORY/logs/production_access.log”. That however, is not very portable is it.

This rest of the file has the comments on what each part means.

Give it a try. Run two commands:
python ./deploy.py -e development
python ./deploy.py -e production

view them at http://localhost:7180/ and http://localhost:7182/ . Notice that both give you the index.html page and that both are running. Now go to http://localhost:7180/nothing and http://localhost:7182/nothing.

Now do you see how the different environment config files come into play. The error for the development app shows you the error (404) tracebacks, whereas the production shows you the 404 html page instead.

Probably the most important file a CherryPy app requires is the deploy script. As developers we need a way to build in one environment and run production in another. This can be incorporated in the deploy script fairly easily and once done we can forget about it. Being able to daemonize the CherryPy app leads to huge performance increases. The cherryd script is merely a basic working script which we can incorporate into our own deploy scripts.

In the next post I’ll show you how to incorporate RESTFul controllers and custom named routes into the deploy script for a well rounded application.

Tags: cherrypy, cherrypyd, script

When the requirement was given to create a web application that was merely more than an API service CherryPy was chosen. CherryPy is an HTTP Framework without all the bells and whistles of database libraries, and templating libraries. As a web service provider it’s turned out to be very fast because of the lack of the other systems.

However what is the use of a web service that doesn’t have a connected database and some minor templating views? Since Ruby on Rails is the predominate framework in our shop and only a few of use work with Python I wanted to use CherryPy in the style of Ruby On Rails so that others transitioning from RoR would be familiar.

While I’m certainly not an evangelist for RESTFul resources I do find that it has it’s place and certainly helps me contemplate what is going to happen when a request is given for certain call methods, (POST, GET, PUT) etc. The CherryPy request dispatcher uses the routes library, which fully supports RESTFul controllers.

Another consideration is how to set up your environments; development, qa, production, etc. This was kind of tricky as a few others I talked to used the PyYAML library to inject global variables throughout their libraries. I found that this doesn’t have to be the case. Why bloat your application with another library with knowing when to import your controllers will allow you to have all that you need within standard CherryPy config.

When setting up your CherryPy application in a manner of the MVC framework you need to have the folder structure. I’ve included a downloadable zip that includes the folders neccessary for this post and subsequent posts. You’ll clearly see that it arranges every system rationally for easy look up and organisation. The key however is in the deploy.py file as it is the start and stop script for your application.

Feel free to download the CherryPy Example 1 file and get a feel for where I am going with the subsequent posts. Subsequent posts are going to cover everything from A-Z on setting up a RESTFul CherryPy application that you can daemonize for a really fast API server that is light on the templating side.

Tags: application, cherrypy, Programming, Python, web site, website

It’s no secret that Google Rank Spanked me, tkaing me from a PR 3 to a PR 0 when they went after paid posts. This site had none of those because: A) I work for IZEA / PayPerPost, B) I work for them so I can’t tkae those opps nor any competitors. Yet because of A I got nailed in the first round and never got any Page Rank back.

No I’m not going to crawl on me hands and kness to Google to get it back either. I’ll entertain the other Search Engines and Web Crawlers out there.

However I do use WordPress, and while I didn’t upgrade in a while until a few weeks ago I did have problems with the WordPress versions I was using being suspectible to hack attempts and injecting malicous code into my posts.

While I never figured out how the attacks were being done I did notice that Google noticed. Thanks to Mouseclone for first alerting me to this I set about sanitizing my posts. All was wel and good.

I recieved word from Cassknits that she couldn’t really enjoy the site because Google had slapped a malicous site label on me. Har Har I thought; Google just making a mockery of me, again. Yes in fact a lookup of techraving on google search did show such a branding.

OK, fine. I do have an account with Google and checked the webmaster tools for the site maps.

Sure enough Google said they found malicous code that triggered their flags but also which pages they found it on. Wouldn’t you know it was exactly the kind of scum I had been battling. One post I either didn’t catch or got re-injected.

I reluctantly, but tired of dealing with the security flaws of WordPress, upgraded. So far no new injections have occured. Only a week after the upgrade though I I wasn’t getting hit on evry post . I did however ask Google to re-evealuate my site to remove the malicious site label. Only took a few days and sure enough I am in the clear from bad things, and that was a week again; no fair thinking I’m writing this apease them.

I don’t wwigh a whole lot of worth on Google in Page Rank or indexing. When I target my keywords carefully and use those to my niche adventagously I have performed very well. Irregardless of my Page Rank. Google is a search engine, just like all the others.

However Google has seemed to take one extra step in that they have the technology and the know-how to protect us from spreading the crap the sticks to the bottom of the intrenet pond. Certainly we can by “some” programs to do this; Google does it for free. We just have to look.

I do have to give them credit for that.

Tags: google, security