Psychic Origami

Python Framework to Python Framework

The good thing about moving from Turbogears to Python was that it's all Python code. This meant that things like the core chess code didn't need touching at all. It also meant that a lot of fairly standard code could be almost directly ported. Methods on controller objects in Turbogears became view functions in Django. Methods on model objects stayed methods and so on. A lot of the porting was fairly mechanistic. I moved all of the Turbogears code into a folder for easy referral and then built the Django version from nothing back up. Initially most of the work was done at the model/db level where I could copy over the code, convert it to Django style and then copy over and update the automated tests. I used Django Coverage to make sure the code was still all actually getting tested.

I could have opted to exactly match the database from the Turbogears version, but opted instead to make it a more Django like. This meant using the regular Django user models and so on. As Turbogears 1.0 involved a custom user model I had to create a separate user profile model in the Django port. There were a few other changes along these lines, but most of the porting was not so structural.

A lot of the hard work during porting came from missing bits of functionality that had far reaching effects. Testing was very awkward until a lot of the code had been ported already.

Cheetah templates to Django templates

Chrss used Cheetah for templates. Cheetah is not as restrictive with it's templates as Django. It's very easy to add lots of logic and code in there. Some pages in chrss had quite a bit of logic - in particular the main chess board page. This made porting rather tricky with Django. I had to put a lot of code into template tags and filters and carefully re-organise things. Porting the templates was probably the hardest part. Automated tests helped a bit with this, but a lot of the issues needed visual verification to ensure things really were working as they should.

One advantage of going from Cheetah to Django's tempate language was the incompatible syntax. This meant I could simply leave bit's of Cheetah code in a template and it would serve as quite a good visual reminder of something that was yet to be ported.

The second 90%

A good portion of the site was ported before my son's birth. It seemed like maybe it wouldn't take much longer, as it felt like 90% of the work was done. Of course it turned out there was another 90% yet to finish.

Beyond the usual tweaking and finishing of various odds and ends, the remaining work consisted of:

• Completing the openid integration
• Migrating the database

For the open id integration I opted to use Simon Willison's Django OpenID app - hoping to be able to have a nice drop-in app. Possibly due to the slightly unfinished nature of the app and mostly due to my desire to customise the urls and general flow of the login/register process this took a fair while. It might have been quicker directly porting the original OpenID code I used with Turbogears, but it did work out ok in the end.

Of course it turns out that after all that hard work, that hardly anyone seems to use OpenID to login to sites anymore. I might have been better off integrating Django Social Auth instead, for Twitter and/or Facebook logings. However I decided that this would have been too much like feature creep and stuck with the original plan.

The chrss database isn't very large. The table recording the moves for each game is currently over 70,000 rows, but each row is very small. So the database dump when tar gzipped is less than 3Mb in size. This gave me plenty of options for database migration. I'd roughly matched the schema used for the Turbogears site, but did take the opportunity to break from the past slightly. I created a Python script that used talked to the MySQL database and generated an sql dump in the new format. By using Python I was free to do any slightly more complex database manipulation I needed. The only real tricky part was converting to using Django's user models.

One wrinkle with the database migrating was a bit disturbing. When I setup the app on webfaction I found some very odd behaviour. Logging in seemed to work, but after a couple of page requests you'd be logged out. The guys at webfaction were very helpful, but we were unable to get to the bottom of the problem. During testing I found that this problem did not happen with SQLite or Postgres, so was something to do with MySQL. This was one of those times when using an ORM paid off massively. Apart from the database migration script no code needed to be touched. If I'd had more time I might have persevered with MySQL, but Postgres has been working very well for me.

Conclusion

Chrss has been running using Django and Postgres for nearly eleven months now and has been working very well. I've had to fix a few things now and then, but doing this in Django has been very easy. I was also able to automate deployment using Fabric, so new code could be put live with the minimum of fuss. When you've only got a limited time to work with, automated deployment makes a huge difference.

Hopefully now that sleep routines are better established and my own sleep is less interrupted I'll have a chance to add new features to chrss soon.