Python Web Frameworks

Chapter 4. Developing Your Own Framework

As the final step in our tour of Python web frameworks, we’ll look briefly at how to develop your very own framework. Even if you don’t plan to build one, knowing a bit about how the internals work might be a good exercise.

Why Create a Framework?

The framework list in this report has 30 frameworks, and that’s just the ones that have a significant(ish) number of downloads. There are many more. Still, people keep creating frameworks, so you might some day want to do that too. In fact, one of the reasons that there are so many Python web frameworks is that it’s not very hard to create one. Python has very good building blocks for this, starting with the WSGI standard and libraries like Werkzeug and WebOb that already implement all of the plumbing, which can be used as a starting point.

Sometimes, instead of a general framework, you might need one that is more tailored to your problem domain. It’s much easier to start from a solid foundation, so you could consider basing your special domain framework on an existing one. Frameworks that have flexible configuration and extension mechanisms, like Pyramid or CherryPy, lend themselves very well to this approach.

Parts of a Basic WSGI Framework

If you decide to build a framework, using Werkzeug or WebOb is a good recommendation. You want to focus on the user-configurable parts, not the internals.

Routing

You will need some way to match a URL to a view. The most common way of doing this is using regular expression routes, like Django or Flask. If you want to go for this approach, you should consider using Werkzeug, which includes a routing system right out of the box. There are also some routing libraries available on PyPI that can be used with WebOb, or by themselves.

There are other ways to map URLs to views. Pyramid offers traversal, which treats your site’s content as a tree and maps views to resources or types of resources. CherryPy uses exposed methods from regular Python classes. Maybe you can think about other approaches. Just use something that feels natural to you.

Templates

There are a couple of very popular template systems for Python, and it’s highly recommended to use one of them. Jinja2 and Mako are two very good options.

Some frameworks are highly tailored to work with a chosen template system, while others leave that decision completely in the hands of the developer. Whatever your choice, you can’t go wrong, as most of these template systems are proven and very stable.

Other Features

Most frameworks offer other features as part of the package. Authentication is a common need and thus is usually provided for in some way. There are many authentication systems and libraries out there. The best advice here is to try to be flexible. You never know how a user will need to authenticate their users.

Many frameworks make the assumption (perhaps less of a certainty these days) that a relational database backend will be part of a web application, and offer at least some degree of database support. If that’s your decision, simply go for SQLAlchemy integration. It’s the most complete (and popular) solution for Python.

Consider offering some support for form handling and input validation, but be aware that there are many libraries that do this very well. Research your alternatives a bit before deciding to do it yourself.

If you intend for your framework to be used by other people, or at least plan to use it more than once, you need to offer a configuration system that allows your users to easily set up and deploy their applications.

Documentation

Even if your framework is only for internal use at your company, do your best to document it. In fact, if you can’t document it, it is better to use one of the existing frameworks. Future maintainers of your applications will thank you.

Good documentation consists of much more than class names and methods. Some narrative documentation is very useful, and many people find that code examples speak volumes. Beginners like tutorials and less technical explanations. Use a documentation tool, like sphinx, to make documentation generation easier.

Framework Building Blocks

You don’t have to start your framework from scratch. There are a couple of WSGI toolkits that take care of the basics, allowing you to concentrate on the unique features provided by your framework.

The two most popular WSGI toolkits are WebOb and Werkzeug. Many web frameworks use one of these libraries. They are time-tested and fully featured, so they provide an excellent base for developing web applications without worrying about the details of HTTP and WSGI.

WebOb (http://webob.org)

WebOb provides objects that map much of the specified behavior of HTTP, including header parsing, content negotiation, and correct handling of conditional and range requests. Pyramid is one well-known framework that uses WebOb.

Werkzeug (http://werkzeug.pocoo.org)

Werkzeug is one of the most advanced WSGI utility modules. It includes a powerful debugger, fully featured request and response objects, HTTP utilities to handle entity tags, cache control headers, HTTP dates, cookie handling, file uploads, and a powerful URL routing system. Flask, one of the most popular Python web frameworks, uses Werkzeug.

Some Useful Resources

If you do decide to write your own framework, or just want to know a bit more about how to do it, there are some excellent web resources that can show you how. Here are three of the best.

You can start with A Do-It-Yourself Framework, in which Ian Bicking explains what WSGI is by building a simple framework. This is a very good tutorial and a great way to find out what’s inside a real framework.

Ian Bicking has a second tutorial, this time using the WebOb library. Another Do-It-Yourself Framework has more details about a framework’s parts, and even goes line by line to explain the code.

Some people learn better using screencasts instead of written documentation. If that’s your case, you’ll find Chris McDonough’s series of screencasts about using repoze.bfg to build a micro framework useful. Remember, repoze.bfg is the framework that became Pyramid. Even if you don’t use either of these frameworks, you should still find this series very instructive. The videos are at http://bfg.repoze.org/videos.