Posts Tagged ‘Programming’

CherryPy, The Setup

September 27th, 2008 by ScottK | No Comments | Filed in CherryPy

When the requirement was given to create a web application that was merely more than an API service CherryPy was chosen. CherryPy is an HTTP Framework without all the bells and whistles of database libraries, and templating libraries. As a web service provider it’s turned out to be very fast because of the lack of the other systems.

However what is the use of a web service that doesn’t have a connected database and some minor templating views? Since Ruby on Rails is the predominate framework in our shop and only a few of use work with Python I wanted to use CherryPy in the style of Ruby On Rails so that others transitioning from RoR would be familiar.

While I’m certainly not an evangelist for RESTFul resources I do find that it has it’s place and certainly helps me contemplate what is going to happen when a request is given for certain call methods, (POST, GET, PUT) etc. The CherryPy request dispatcher uses the routes library, which fully supports RESTFul controllers.

Another consideration is how to set up your environments; development, qa, production, etc. This was kind of tricky as a few others I talked to used the PyYAML library to inject global variables throughout their libraries. I found that this doesn’t have to be the case. Why bloat your application with another library with knowing when to import your controllers will allow you to have all that you need within standard CherryPy config.

When setting up your CherryPy application in a manner of the MVC framework you need to have the folder structure. I’ve included a downloadable zip that includes the folders neccessary for this post and subsequent posts. You’ll clearly see that it arranges every system rationally for easy look up and organisation. The key however is in the deploy.py file as it is the start and stop script for your application.

Feel free to download the CherryPy Example 1 file and get a feel for where I am going with the subsequent posts. Subsequent posts are going to cover everything from A-Z on setting up a RESTFul CherryPy application that you can daemonize for a really fast API server that is light on the templating side.

Tags: , , , , ,

Celebrating the birth of my first “WebBot”

April 13th, 2008 by ScottK | No Comments | Filed in Python

Today is a special day for me as I have given birth to my first fully functional and interactive webbot. This momentous occasions has lifted me from creating spiders written in PHP and even JavaScript to a whole new level of software automation that I have been tinkering with for a while. As the added bonus I have learned a bit more python and actually accomplished early on the true functionality of what I wanted to achieve.

 Given that I am the primary developer for Zookoda I know that the feed retrieval system can be a bit awkward at times. The library written in ColdFusion isn’t as adaptive as it could be and I wanted to find another way to retrieve feeds. Add to the fact that some members feeds are only retrieved via a login only makes things more difficult. Enter Python and the mechanize and ClientForm libraries.

Using both these libraries I was easily able to detect whether a I had a log in form based upon the system I was hitting and entering the credentials for the login form. Easy enough as form.click() submitted the form along with the cookie to retain the session to retrieve the authentication for getting the feed. So why didn’t I stop here?

The login problem and subsequent retrieval was quite easy, but I wanted to go up against something that was way more difficult. Being that as developers for IZEA we use campfirenow as a means of collaborating and as it turns out I found my challenge.

First the login is a POST method on the form. Second, sending a message is done via the post but the message box is done through an XMMLHttpRequest. So clicking the “Send Message” means you have to have JavaScript enabled. Coincidentaly logging in with the mechanize and ClientForm was no problem, but the sending of a message was quite difficult.

So how does this relate? My baby ScooterBot is able to enter the configured room and if required, log in. It can then parse the XHTML and read the message id, who the person was that made the message, and the message itself. If you are a user of campfirenow then you know the XHTML changes, i.e. the rows you see. So this proves my original intent that my program can be adaptive.

The second challenge I asked of myself was to response to the entries (message) entered by others. Here is where the program took a hair pulling turn. The textarea you enter text and subsequent submit button is unobtrusively watched by JavaScript, and as Python scrapers don’t have the capability I was out of luck. Until I discovered that the mechanize did support it as “mechanize.urlopen(<complete_url> + ”/speak”, urllib.urlencode({”message” : message, “t” : time.time()})

At which point the correct POST was made, with urllib.urlencode as the second arg, and that mechanize still held my session cookie. Trust me I tried for hours with out it and only came up against the log in page. 

I won’t bore you with the “interactivity” as it’s far from final as of yet. ScooterBot does look at commands given to it and acts based upon those. It even re-acts to such things as “@ScooterBot: I Love you” at which point response with “<person>: I am kinda partial to you too”. The beginning of sentiment is there as well as it looks for keywords with users conversations and interjects a pre-programmed phrase.

It is still very rough though as it’s only one day old ;) However, years ago I was one day old and I have learned so much :)

Tags: , , ,

Web 2.0, Namespacing

April 3rd, 2008 by ScottK | No Comments | Filed in JavaScript

With plugins, widgets, and what-not’s every JavaScript author is vying for space on a web page, whether their own or someone elses. How then to keep the funciton literal names from colliding with other authors, or even previously written code? Writing JavaScript strictly with global functions really places you in a situation that someone may use the same name and make your function useless, and ultimately your entire program.

This is where namespacing your JavaScript reduces the risk of anyone colliding with your code. Quite simply namespacing in JavaScript is just as simple creating an object and attaching the functions to that object, the JavaScript Prototype scope, gotta love it. Yes, you are building a JavaScript object that sets all the methods apart from anything else someone would create. So this object can, and should, contain any needed functions that it needs to operate.

The actual naming convention of the object is really up you. I personally like to namespace of the object intent, Browser. (Browser detection object), TR. (functions relating to the general javascript of Tech Raving). The real intent is to use a name that someone else may not use. Take Browser for instance, what are the chances that someone else could use that for an object or function name? Probably pretty good. BrowserSJK as a name doesn’t really look good but now what are the chances that someone will use that!

To actually create a namespace object is relativly simple:

var TR = {
    method1 : function() {
        //Do stuff here
    },
    method2 : function() {
        //Do stuff here
    }
}

So now you can use TR.method1().

That’s good for a start but what about variables you would normally need. Easy enough to set up and use and gain access to:

var TR = {
    attribute1 : “hello”,
    attribute2 : “world”,

    setAttribute1 : function(temp) {
        this.attribute1 = temp;
        this.method1();
    },
    method1 : function() {
        //Do stuff here
    },
    method2 : function() {
        //Do stuff here
    }
}

TR.setAttribute1(”test”);

The difference in namespacing this way as opposed to just writing the variables and functions is the attachment to the TR object and the use of the this keyword to make the object reference itself in the executable scope. Here there be dragons when global scoping varibals and functions because of naming collisions, although it’s easy enough to access them if no collision occurs. Namespacing protects the variables and methods by wrapping them in an object, yet some care is needed by accessing them through the Namespace name, and using the “this” keyword for access outside of the method, yet referencing within the object namespace. Sounds complicated I know, it’s a prototype thing, but until you attach events to dynamically created html elements it’s really straight forward.

So what’s a developer to do if they have existing code that has either conflicted or just needs a little love and refactoring? That’s simple as well just create an object and then prepend all the methods and varibles with the Namespace.

old code:

var myVar1 = “hello”;
var myVar2 = “world”;

function method1() {
    //do Somtehing;
}

new code:

var myObj = new Object();

myObj.myVar1 = “hello”;
myObj.myVar2 = “world”;

myObj.method1 = function(temp) {
    this.myVar1 = temp;
}

myObj.method1(”test”);

Now the new code is protected from generalization of naming. Unless you have hundreds of lines of code the refactor should go quickly. If you have hundreds of lines of the old code I would highly suggest breaking into related objects using the same namespace approach. ;)

JavaScript used to be authors writing code for their own sites in order to help acheive some dynamic capabilities. Maybe specialized libraries, such as menu’s or link generators where “given” out, but really only one was used on a publishers site. Now a days with the many widgets given out it’s quite common for publishers to have many third party JavaScripts and not from the same author. Namespacing is a way to protect your code as an author from the other authors.

As web 2.0 grows in popularity, it’s important for us developers to ensure that our code is tested and works even in the event of failure on another party.

Tags: , ,