Everything I know about Python...

Learn to Write Pythonic Code!

Check out the book Writing Idiomatic Python!

Discuss Posts With Other Readers at discourse.jeffknupp.com!

Unit Testing in Django

As a follow-up to my post Starting a Django Project the Right Way, I wanted to talk about the importance of writing tests for Django applications. I previously mentioned that my first site IllestRhyme, has no app specific tests for it. This is both embarrassing and true. I've lost countless hours to fixing problems caused by new changes. I wasn't going to make the same mistake with linkrdr. Having a set of unit tests that I can run in an automated fashion has made a world of difference.

The Django unittest framework (really the Python unittest framework) is both simple and powerful. Along with the test client (django.test.client.Client), there's a lot you can do with Django right out of the box.

Setup

To start, we'll want to create a dump of our database data to use during testing.

$ ./manage.py dumpdata --format=json > my/app/directory/initial_data.json

This will give us a json fixture that mimics the current state of our production database. Note that since this is a fixture for all of the apps installed, we've put it in a non-standard directory. To let the test runner find our fixture, we'll need to set FIXTURE_DIRS to the directory we just dumped our data to.

Now that we have our data copied, let's run whatever tests our installed apps have already:

$  python manage.py test

This hopefully gives us output like:

.....................................................................................................................................................................................................................................................................................................................................................................
----------------------------------------------------------------------
Ran 357 tests in 30.025s
OK

This is also a good check of the integrity of your database, as Django will try to load a fixture representing all of your data. If you've been screwing around with the admin interface or the shell adding and deleting records, you may have integrity errors. If you do (like I did), you'll have to fix them manually and re-dump your data.

Read on →


Introducing linkrdr

I started work on a new site on Monday: linkrdr. It's the next generation feed reader for people who subscribe to tens or hundreds of feeds. linkrdr aggregates your feeds but more importantly, your links. It ranks links according to a relevance formula. Purely chronological based readers are just terrible at managing a mountain of links.

The idea for linkrdr came from a Hacker News post describing exactly the problem linkrdr solves. I realized I had exactly the same problem as the submitter: too many links, too little time. I don't want to miss out on a quality post on a blog that doesn't publish very often. At the same time, if there's a link that's showing up in a number of my feeds, it's a good bet that it's worth reading.

Read on →


Starting a Django Project the Right Way

Using Django 1.6?

Check out the new, updated version of this post with Django 1.6 specific changes and updates.

One of the things I wish I had known when starting my Django project for IllestRhyme was "How do I start a real Django project". As in, one that's actually going to be used and developed more, not the toy project from the (admittedly excellent) Django documentation.

Having just gone through this process again for my new site, I wanted to share the knowledge I've gained about how to properly start a project in Django. By the end of this post, you will have

1. A fully functional Django project
2. All resources under source control (with git)
3. An environment independent install of your project (using virtualenv)
4. Automated deployment and testing (using Fabric)
5. Automatic database migrations (using South)
6. A solid start to your new site

None of these steps, except for perhaps the first, are covered in the official tutorial. They should be. If you're looking to start a new, production ready Django project, look no further.

Read on →


Building Your Second Django Site

I started work on my second Django powered site today. It's not ready to be unveiled, but I've realized that, while your first site will always be your baby, your second is where you start to hit your stride.

So what have I done differently? For one, testing. For IllestRhyme I never wrote any tests, which has caused me no end of troubles. Now that the site is stable and actively used, it's difficult to make myself go back and write tests, though I know I need to. The new site has unit tests for everything, and though it's somewhat of a pain to do, it will most certainly pay dividends in the long run.

In the production environment, everything is running under virtualenv, Python's unofficial answer to the problem of running multiple Python environments. It effectively does a clean Python install to a local directory and hijacks pip and easy_install commands to install packages locally. This guarantees packages for the new site don't clash with packages for the old site.

Also, I'm using Fabric for automating deployment tasks. While before I relied on a set of git hooks, this became a bit cumbersome. Fabric is fantastic for deployment. My fabfile backs up the site, downloads packages from git, creates symlinks, run South migrations, runs the tests, and reloads Apache. Deployment should really be a one-button-press activity. Fabric makes it easy. Thanks to a number of commenters (Alexis Bellido on this site, joelhaasnoot, spleeyah, jsvaughan, and marcofucci on Hacker News) for pointing this out on a previous post.

Read on →


Analytics for Django Sites

This is the first in a series of posts I plan to do over the next month about Analytics and Django. In this post I'll walk through how I began to use a number of analytics tools to drive decisions about my site, IllestRhyme.

One of the first things I did when writing IllestRhyme was to sign up for Google Analytics. I had never run a web site before, but I was aware of Google's analytics offering. I admit, for the first few weeks after the site went live I would start at the Google Analytics page and hope to see users in the Live View. I wasn't really using the data for anything. I was using Google Analytics as a virtual scoreboard.

It wasn't until I signed up for Clicky that I started to take analytics seriously. In fact, more specifically, it was when I combined Clicky and django-analytical that I really took my first deep-dive into using analytics data for decision making. Since django-analytical already integrates with a number of analytics services, including Clicky, setup was a breeze. Sure, I could have inserted the raw code into my Django templates, but django-analytical gave me a single point at which to configure all my analytics services.

More importantly, and I can not stress this enough, django-analytical's Clicky integration let me see my users in Clicky via their contrib.auth usernames. This was huge. Now, instead of staring at IP addresses, I could follow users on their visit to the site (using Clicky's awesome Spy feature) in real-time by user name. Believe me, nothing will teach you more about you users than recognizing usernames and their associated behavior patterns. I could tell which users were hitting the site to check quickly for updates, which users hung around for a while, and which users used the site like it was crack. It was this last group that I was initially interested in.

Read on →