I’ve devouted a great deal of time working with MVx frameworks in several languages: ColdFusion; PHP; Python; ASP .Net MVC; and of course Ruby on Rails. Clearly each has it’s own excellencies for what the app needs to accomplish, but when it comes to a well rounded application framework for building an application Ruby on Rails does the job. Albeit there are things I don’t really like about Ruby on Rails as well.

My “language” of choice is Python, yet I find the frameworks sub-ordinate in many areas for managing the apps. Especially in the area of database management. Clearly the ability to run db:migrate in Ruby on Rails to update your database to the current version via file naming is powerful, yet in case of trouble being able to rollback those changes is immensely so. The Python frameworks lack this ability and in managing several Python applications just to make the changes during a deploy is time consuming. God forbit that something goes wrong and I need to revert the changes (hasn’t happened yet, now I don’t have to worry).

So for those not familiar with the Ruby on Rails migration system let me explain. As you need changes to your database you place those changes in the “self.up” section of the file. Typically what you changed on the up in those files you also revert in the “self.down” section of the same file. Another crucial benefit is that you don’t need to know your database syntax, Ruby on Rails does it for you in their formatting, being that Ruby on Rails has database adapters.

db:migrate does this in one command, up/down, as well. If you have 20 migration files it’s one command versus 20 MySQL cli commands, and twenty password entries. It’s extremely powerful, I’ll admit, and I so need it for managing a few Python apps using CherryPy.

None of the Python apps I’ve used does anything close to this so I created one. Keeping in mind that this is totally independant from design from frameworks and it can be used in any situation or application. It fully supports updating you database schema to a new structure as well as reverting the schema via a record in a database table.

A sample migration looks like this:

[up]
add_column: users, registered_at, not_null : True, type : datetime
add_column: users, deleted_at, type : datetime
raw_sql: select * from users

add_index: users, my_stuff, unique : True, using : btree, columns : [account_key ASC, first_name DESC]
remove_index: users, my_stuff

change_column: users, account_key, type : varchar, length : 100, default : NULL

create_table: users_test, force : True, engine : InnoDB, char_set : latin1
    primary_key: id, account_key
    column: id, type : int, length : 11, not_null : True, auto : True
    column: account_key, type : varchar, length : 100, default : NULL
    column: first_name, type : varchar, length : 100, default : NULL
    column: updated_at, type : DATETIME, default : NULL

drop_table: users_test

[down]
remove_column: users, registered_at
remove_column: users, deleted_at

So if my database version was 002 then upgrading would run the up section. Creating, changing, dropping, etc. If I needed to go back down to version 001 then the down section would run, removing the registered_at and deleted_at columns.

I’m not releasing the code just yet. I need to DRY up the code and make sure a few other particulars are in order. I do have all those you see working correctly and I want to open the floor to a name. My particular choice is PyMigrate; but since I intend to let everyone use it I see reason to let everyone choose a name as wellr.

Until the initial release candidate it only supports MySQLdb as that is the db I am most familiar with. The database adapters are plugins so it will be really easy to include other database types before and after initial release. All I need  from you Python peeps is any ideas and to help me name this by Febuary 9th!

I’ve got some small refactoring to do plus lot’s of documentation to write to help get everyone started so check in here often! Leave your naming choices though in the comments as I’ll pick one by Feb. 9th.

I’m still plucking away at this project as time allows using CherryPy. This weekend I just started to use it and finally got a working environmental configuration system and then next step was to figure out if RESTful controllers could be used. As it turns out you can not only use RESTful controllers but have the additional benefit of format types as well (html, xml, json, etc).

This is due to the CherryPy RoutesDispatcher using the routes library. I’ve successfully setup dispatcher with RESTful routes and have tests to show that they work as planned.

I’ll do a follow up post with working examples this weekend.

Tags: cherrypy, pylons, Python, routes

Here’s my dilemma, I’m a developer working on code on my machine. Once I’m done I send the code to QA who then says yay or nay. Once I get a yay then the code goes into production on a live server. CherryPy has a site config for all apps as well as an app specific config, which pertains to the code I writing. Under the CherryPy documentation I would have to modify 3 different files in order to add 1 new model! That’s insane.

Additionally, I’m working on my computer (localhost) so I need the SQLObject connections to point to my machine. When I send my work to QA the db connections are completely different. Production environment, again different. Traces, ip address, etc, all different from the various environments. CherryPy 3 does have a “global” config for all apps as well as a “site” config pertaining to your app, but no where does it suggest setting up these environments. Outside of the “environment = whatever” in the config.

Do I really have to write my code on my machine, send it to QA, log into shell and make the modifications, get a yay and send to production, log into shell there and make modifications? Do this across twenty developers as well?

No…

I hope you see the catastrophe that could ensue because an out of the box framework doesn’t allow environments to be set up within an app. Likewise I do judge a framework by the lack of code duplication (DRY). Searching the documentation for CherryPy as well as many other sites I found no information concerning my requirements. So it was time to start “playing”

CherryPy does allow you to remain DRY and set up environments!
Takes a little work around though, but with a deploy script it’s easy.

First let’s create the scripts:

# Site Config (site_config.py)

[global]
server.protocol_version = "HTTP/1.1"
tools.proxy.on = True
tools.proxy.local = "X-Forwarded-Host"
tools.proxy.remote =  "X-Forwarded-For"
log.error_file = "logs/error.log"
log.access_file = "logs/access.log"

#Development (development.py)

[global]
server.socket_host = "127.0.0.1"
server.socket_port = 8082
log.screen = True
tools.log_tracebacks.on = True

#QA (qa.py)

[global]
server.socket_host = "127.0.0.1"
server.socket_port = 8083
log.screen = True
tools.log_tracebacks.on = False

#routes.py (routes.py)

[/]

#whatever

#deploy script (deploy.py)

import sys
import cherrypy

import models
from models.base import Base
#add your additional models as well from models folder

if __name__ == '__main__':
    cherrypy.config.update(sys.argv[1] + ".py")
    cherrypy.config.update(sys.argv[2] + ".py")
    cherrypy.tree.mount(Base(), script_name="/", config="routes.py")
    cherrypy.engine.start()

So in the command line when you call:
>>python deploy.py site_config.py development
what happens is:

1. Loads the models as classes.
2. cherrypy loads the site wide config variables (site_config.py). These pertain to all the environments.
3. chrrypy loads, adds or overwrites, all the environment settings. Specific to development (locally), qa (hope it works), production (umm better work)
4. Mounts each model to the correct path as found in the routes.py file which contains settings for each route.

Of course it’s not completely DRY. As each new model is made you do have to update the deploy script as well as the routes.py, but it’s a lot better than doing all your environment files.

I do feel CherryPy would be much more powerful if out of the box we could set up situations like this as well as a plain routes file (Dispatcher) so hacks like this aren’t needed. However this hack at least kept me considering using CherryPy.