Introduction to Tornado

A Simple Persistent Web Service

Now we know enough to write a web service that can access data in a MongoDB database. First, we’re going to write a web service that just reads data from MongoDB. Then, we’ll write one that reads and writes data.

A Read-Only Dictionary

The application we’re going to build is a simple web-based dictionary. You should be able to make requests for a particular word, and get back the definition for that word. Here’s what a typical interaction might look like:

$ curl http://localhost:8000/oarlock
{definition: "A device attached to a rowboat to hold the oars in place",
"word": "oarlock"}

This web service will be drawing its data from a MongoDB database. Specifically, we’ll be looking up documents by their word attributes. Before we actually look at the source code for the web application itself, let’s add some words to the database in the interactive interpreter.

>>> import pymongo
>>> conn = pymongo.Connection("localhost", 27017)
>>> db = conn.example
>>> db.words.insert({"word": "oarlock", "definition":»
    "A device attached to a rowboat to hold the oars in place"})
ObjectId('4eb1d1f8136fc4be90000000')
>>> db.words.insert({"word": "seminomadic", "definition": "Only partially nomadic"})
ObjectId('4eb1d356136fc4be90000001')
>>> db.words.insert({"word": "perturb", "definition": "Bother, unsettle, modify"})
ObjectId('4eb1d39d136fc4be90000002')

See Example 4-1 for the source code for our dictionary web service, which will look up the words we just added and then respond with the definition.

Example 4-1. A dictionary web service: definitions_readonly.py

import tornado.httpserver
import tornado.ioloop
import tornado.options
import tornado.web

import pymongo

from tornado.options import define, options
define("port", default=8000, help="run on the given port", type=int)

class Application(tornado.web.Application):
    def __init__(self):
        handlers = [(r"/(\w+)", WordHandler)]
        conn = pymongo.Connection("localhost", 27017)
        self.db = conn["example"]
        tornado.web.Application.__init__(self, handlers, debug=True)

class WordHandler(tornado.web.RequestHandler):
    def get(self, word):
        coll = self.application.db.words
        word_doc = coll.find_one({"word": word})
        if word_doc:
            del word_doc["_id"]
            self.write(word_doc)
        else:
            self.set_status(404)
            self.write({"error": "word not found"})

if __name__ == "__main__":
    tornado.options.parse_command_line()
    http_server = tornado.httpserver.HTTPServer(Application())
    http_server.listen(options.port)
    tornado.ioloop.IOLoop.instance().start()

Run this program on the command line like so:

$ python definitions_readonly.py

Now use curl or your web browser to make a request to the application.

$ curl http://localhost:8000/perturb
{"definition": "Bother, unsettle, modify", "word": "perturb"}

If we request a word that we haven’t added to the database, we get a 404 response, along with an error message:

$ curl http://localhost:8000/snorkle
{"error": "word not found"}

So how does this program work? Let’s discuss a few key lines from the code. To begin, we include import pymongo at the top of our program. We then instantiate a pymongo Connection object in the __init__ method of our Tornado Application object. We create a db attribute on our Application object, which refers to the example database in MongoDB. Here’s the relevant code:

conn = pymongo.Connection("localhost", 27017)
self.db = conn["example"]

Once we’ve added the db attribute to our Application object, we can access it as self.application.db in any RequestHandler object. This is, in fact, exactly what we do in the get method of WordHandler in order to retrieve a pymongo collection object for the words collection. The following is the code for the get method:

def get(self, word):
    coll = self.application.db.words
    word_doc = coll.find_one({"word": word})
    if word_doc:
        del word_doc["_id"]
        self.write(word_doc)
    else:
        self.set_status(404)
        self.write({"error": "word not found"})

After we’ve assigned the collection object to the variable coll, we call the find_one method with the word that the user specified in the path of the HTTP request. If we found a word, we delete the _id key from the dictionary (so that Python’s json library can serialize it), then pass it to the RequestHandler’s write method. The write method will automatically serialize the dictionary as JSON.

If the find_one method doesn’t find a matching object, it returns None. In this case, we set the response’s status to 404 and write a small bit of JSON to inform the user that the word they specified wasn’t found in the database.

Writing the Dictionary

Looking words up in the dictionary is lots of fun, but it’s a hassle to have to add words beforehand in the interactive interpreter. The next step in our example is to make it possible to create and modify words by making HTTP requests to the web service.

Here’s how it will work: issuing a POST request for a particular word will modify the existing definition with the definition given in the body of the request. If the word doesn’t already exist, it will be created. For example, to create a new word:

$ curl -d definition=a+leg+shirt http://localhost:8000/pants
{"definition": "a leg shirt", "word": "pants"}

Having created the word, we can request it with a GET request:

$ curl http://localhost:8000/pants
{"definition": "a leg shirt", "word": "pants"}

We can modify an existing word by issuing a POST request with a definition field to a word (the same arguments we use when creating a new word):

$ curl -d definition=a+boat+wizard http://localhost:8000/oarlock
{"definition": "a boat wizard", "word": "oarlock"}

See Example 4-2 for the source code for the read/write version of our dictionary web service.

Example 4-2. A read/write dictionary service: definitions_readwrite.py

import tornado.httpserver
import tornado.ioloop
import tornado.options
import tornado.web

import pymongo

from tornado.options import define, options
define("port", default=8000, help="run on the given port", type=int)

class Application(tornado.web.Application):
    def __init__(self):
        handlers = [(r"/(\w+)", WordHandler)]
        conn = pymongo.Connection("localhost", 27017)
        self.db = conn["definitions"]
        tornado.web.Application.__init__(self, handlers, debug=True)

class WordHandler(tornado.web.RequestHandler):
    def get(self, word):
        coll = self.application.db.words
        word_doc = coll.find_one({"word": word})
        if word_doc:
            del word_doc["_id"]
            self.write(word_doc)
        else:
            self.set_status(404)
    def post(self, word):
        definition = self.get_argument("definition")
        coll = self.application.db.words
        word_doc = coll.find_one({"word": word})
        if word_doc:
            word_doc['definition'] = definition
            coll.save(word_doc)
        else:
            word_doc = {'word': word, 'definition': definition}
            coll.insert(word_doc)
        del word_doc["_id"]
        self.write(word_doc)

if __name__ == "__main__":
    tornado.options.parse_command_line()
    http_server = tornado.httpserver.HTTPServer(Application())
    http_server.listen(options.port)
    tornado.ioloop.IOLoop.instance().start()

The source code is exactly the same as the read-only service, except for the addition of the post method in WordHandler. Let’s look at that method in more detail:

    def post(self, word):
        definition = self.get_argument("definition")
        coll = self.application.db.words
        word_doc = coll.find_one({"word": word})
        if word_doc:
            word_doc['definition'] = definition
            coll.save(word_doc)
        else:
            word_doc = {'word': word, 'definition': definition}
            coll.insert(word_doc)
        del word_doc["_id"]
        self.write(word_doc)

The first thing we do is use the get_argument method to fetch the definition passed in to our request from the POST. Then, just as in the get method, we attempt to load the document with the given word from the database using the find_one method. If such a document was found, we set its definition entry to the value we got from the POST arguments, then call the collection object’s save method to write the changes to the database. If no document was found, we create a new one and use the insert method to save it to the database. In either case, after the database operation has taken place, we write the document out in the response (taking care to delete the _id attribute first).