You are previewing RESTful Web Services.

RESTful Web Services

Cover of RESTful Web Services by Leonard Richardson... Published by O'Reilly Media, Inc.
  1. RESTful Web Services
    1. SPECIAL OFFER: Upgrade this ebook with O’Reilly
    2. A Note Regarding Supplemental Files
    3. Foreword
    4. Preface
      1. The Web Is Simple
      2. Big Web Services Are Not Simple
      3. The Story of the REST
      4. Reuniting the Webs
      5. What’s in This Book?
      6. Administrative Notes
      7. Conventions Used in This Book
      8. Using Code Examples
      9. Safari® Enabled
      10. How to Contact Us
      11. Acknowledgments
    5. 1. The Programmable Web and Its Inhabitants
      1. Kinds of Things on the Programmable Web
      2. HTTP: Documents in Envelopes
      3. Method Information
      4. Scoping Information
      5. The Competing Architectures
      6. Technologies on the Programmable Web
      7. Leftover Terminology
    6. 2. Writing Web Service Clients
      1. Web Services Are Web Sites
      2. del.icio.us: The Sample Application
      3. Making the Request: HTTP Libraries
      4. Processing the Response: XML Parsers
      5. JSON Parsers: Handling Serialized Data
      6. Clients Made Easy with WADL
    7. 3. What Makes RESTful Services Different?
      1. Introducing the Simple Storage Service
      2. Object-Oriented Design of S3
      3. Resources
      4. HTTP Response Codes
      5. An S3 Client
      6. Request Signing and Access Control
      7. Using the S3 Client Library
      8. Clients Made Transparent with ActiveResource
      9. Parting Words
    8. 4. The Resource-Oriented Architecture
      1. Resource-Oriented What Now?
      2. What’s a Resource?
      3. URIs
      4. Addressability
      5. Statelessness
      6. Representations
      7. Links and Connectedness
      8. The Uniform Interface
      9. That’s It!
    9. 5. Designing Read-Only Resource-Oriented Services
      1. Resource Design
      2. Turning Requirements Into Read-Only Resources
      3. Figure Out the Data Set
      4. Split the Data Set into Resources
      5. Name the Resources
      6. Design Your Representations
      7. Link the Resources to Each Other
      8. The HTTP Response
      9. Conclusion
    10. 6. Designing Read/Write Resource-Oriented Services
      1. User Accounts as Resources
      2. Custom Places
      3. A Look Back at the Map Service
    11. 7. A Service Implementation
      1. A Social Bookmarking Web Service
      2. Figuring Out the Data Set
      3. Resource Design
      4. Design the Representation(s) Accepted from the Client
      5. Design the Representation(s) Served to the Client
      6. Connect Resources to Each Other
      7. What’s Supposed to Happen?
      8. What Might Go Wrong?
      9. Controller Code
      10. Model Code
      11. What Does the Client Need to Know?
    12. 8. REST and ROA Best Practices
      1. Resource-Oriented Basics
      2. The Generic ROA Procedure
      3. Addressability
      4. State and Statelessness
      5. Connectedness
      6. The Uniform Interface
      7. This Stuff Matters
      8. Resource Design
      9. URI Design
      10. Outgoing Representations
      11. Incoming Representations
      12. Service Versioning
      13. Permanent URIs Versus Readable URIs
      14. Standard Features of HTTP
      15. Faking PUT and DELETE
      16. The Trouble with Cookies
      17. Why Should a User Trust the HTTP Client?
    13. 9. The Building Blocks of Services
      1. Representation Formats
      2. Prepackaged Control Flows
      3. Hypermedia Technologies
    14. 10. The Resource-Oriented Architecture Versus Big Web Services
      1. What Problems Are Big Web Services Trying to Solve?
      2. SOAP
      3. WSDL
      4. UDDI
      5. Security
      6. Reliable Messaging
      7. Transactions
      8. BPEL, ESB, and SOA
      9. Conclusion
    15. 11. Ajax Applications as REST Clients
      1. From AJAX to Ajax
      2. The Ajax Architecture
      3. A del.icio.us Example
      4. The Advantages of Ajax
      5. The Disadvantages of Ajax
      6. REST Goes Better
      7. Making the Request
      8. Handling the Response
      9. JSON
      10. Don’t Bogart the Benefits of REST
      11. Cross-Browser Issues and Ajax Libraries
      12. Subverting the Browser Security Model
    16. 12. Frameworks for RESTful Services
      1. Ruby on Rails
      2. Restlet
      3. Django
    17. A. Some Resources for REST and Some RESTful Resources
      1. Standards and Guides
      2. Services You Can Use
    18. B. The HTTP Response Code Top 42
      1. Three to Seven Status Codes: The Bare Minimum
      2. 1xx: Meta
      3. 2xx: Success
      4. 3xx: Redirection
      5. 4xx: Client-Side Error
      6. 5xx: Server-Side Error
    19. C. The HTTP Header Top Infinity
      1. Standard Headers
      2. Nonstandard Headers
    20. Index
    21. About the Authors
    22. Colophon
    23. SPECIAL OFFER: Upgrade this ebook with O’Reilly
O'Reilly logo

Django

by Jacob Kaplan-Moss

Django is a framework that makes it easy to develop web applications and web services in Python. Its design is very similar to Rails, though it makes fewer simplifying assumptions. You can apply the generic ROA design procedure to turn a dataset into a set of RESTful resources and implement those resources directly in Django.

I’ll show you how I implemented a social bookmarking service in Django, along the lines of the Rails implementation in Chapter 7. Since this book isn’t intended to be a Django tutorial, I’m leaving out most of the intermediary steps of Django development so I can focus on the parts that specifically apply to RESTful web services. If you’re interested in learning more about Django, you should check out the free online Django Book and the official Django documentation.

Create the Data Model

Most Django developers start by designing the data model. This corresponds to the first step of the generic ROA procedure, “Figure out the data set.” The model is usually stored in a relational database using Django’s object-relational mapping (ORM) tools. It’s certainly possible to write RESTful services that don’t use a database, but for the social bookmarking application a database makes the most sense. It’s fairly straightforward to translate the Rails migration from Example 7-1 into a Django model, as seen in Example 12-10.

Example 12-10. The Django model (models.py)

from datetime import datetime
from django.db import models
from django.contrib.auth.models import User

class Tag(models.Model):
    name = models.SlugField(maxlength=100, primary_key=True)

class Bookmark(models.Model):
    user                = models.ForeignKey(User)
    url                 = models.URLField(db_index=True)
    short_description   = models.CharField(maxlength=255)
    long_description    = models.TextField(blank=True)
    timestamp           = models.DateTimeField(default=datetime.now)
    public              = models.BooleanField()
    tags                = models.ManyToManyField(Tag)

There’s a few subtleties and a lot of power squeezed into these few lines of code:

  • I chose to use the built-in Django User model, rather than create my own users table as the Rails example does. This has a few advantages, the biggest being that the built-in User model will handle much of the authentication and authorization. For more information on this, see Chapter 12 of the Django Book.

  • Django has no direct analog to the Rails acts_as_taggable plugin, so in the last line of the Bookmark definition I just define a many-to-many relation between Bookmark and Tag.

  • I’m defining the tag’s name as a SlugField rather than a string. This is a Django class that automatically restricts tag names to those that can show up in a URI. This makes it easy to prohibit tags that contain spaces or other non-alphanumeric characters.

  • Most of the database indexes created explicitly in the Rails schema are automatically added by Django. In particular, slug fields and foreign keys automatically are given indexes. Notice, however, that I’ve had to explicitly specify db_index=True on the url field. That field won’t get an index by default, but I want to search it.

Define Resources and Give Them URIs

The Rails implementation of the social bookmarking application exposes 11 resources. To keep the size of this section down, I’m only going to implement 4 of the 11:

  • A single bookmark

  • The list of a user’s bookmarks

  • The list of bookmarks a user has tagged with a particular tag

  • The list of tags a user has used

In particular, notice that I’m not exposing user accounts as resources. To use this service you’ll need to pre-create some sample user accounts in the database.

Ruby on Rails imposes simplifying assumptions that affect your URI design. Instead of defining resources, you define Rails controllers that expose resources at certain URIs. Django makes you design your URIs from scratch. Django’s philosophy is that the URI is an important part of a web application’s user interface, and should not be automatically generated. This fits in with the ROA philosophy, since a resource’s only interface elements are its URI and the uniform interface of HTTP.

Since Django forces you to design URLs explicitly, there’s no “path of least resistance” as there is in Rails, so I’m able to make my Django URIs a little more compact and readable than the Rails URIs were. I’ll modify the URI structure given for the Rails application in three ways:

  • The Django “house style” (as it were) is to always end URIs with a trailing slash. It’s possible to go either way, of course, but to fit more with what Django developers expect I’ll make all URIs include the trailing slash. That is, I’ll use URLs like /users/{username}/ instead of /users/{username}.

  • Rails’s controller-based architecture make it convenient to expose bookmarks as /users/{username}/bookmarks/{URL}/. In Django it’s just as convenient to use the more compact /users/{username}/{URL}/, so that’s what I’ll use.

  • Since I’m not exposing user accounts as resources, I can use URIs of the form /users/{username}/ for a different purpose. I’m going to expose my “bookmark list” resources there.

  • The Rails implementation uses POST to create a new bookmark as a subordinate resource of a bookmark list. I’ll create new bookmarks using the other technique: by sending a PUT to /users/{username}/{URI}/, bypassing the bookmark list altogether. Rails had a problem with embedding a URI in another URI, so back then I exposed URIs like /users/{username}/bookmarks/{URI-MD5}. Here I can use the actual URIs themselves.

I can easily use a Django URI configuration file to map these URIs to resources (Example 12-11). This is the equivalent of the routes.rb file in Example 7-3. It’s a lot simpler, though, because Django doesn’t try to make decisions about URI format for me.

Example 12-11. A Django URI configuration: urls.py

from django.conf.urls.defaults import *
from bookmarks.views import *

urlpatterns = patterns('',
    (r'^users/([\w-]+)/$',              bookmark_list),
    (r'^users/([\w-]+)/tags/$',         tag_list),
    (r'^users/([\w-]+)/tags/([\w-]+)/', tag_detail),
    (r'^users/([\w-]+)/(.*)',           BookmarkDetail()),
)

The urls.py file is a small Python module that maps incoming URIs (represented as regular expressions) to the functions that handle the requests. Any groups in the regular expression are passed as arguments to the function. So if a request comes in for users/jacobian/tags/python, Django will match it against the third regular expression, and call the tag_detail function with two arguments: “jacobian” and “python”.

Since Django evaluates the URI patterns in order, I have to put the tag URIs before the bookmark URLs: otherwise, Django would interpret /users/jacobian/tags/ as a request for a bookmark of the (invalid) URI tags.

Of course, now I’m committed to writing four functions in a module called bookmarks.views. I won’t leave you hanging, so let’s move onto those functions.

Implement Resources as Django Views

Django interprets the Model-View-Controller pattern differently than Rails does. In Rails, to implement a resource’s behavior under the uniform interface, you put code in a controller class. In Django, that code goes into the view. The Django FAQ has more on this distinction, but I’ll show you the implementation of two views: the read-only view function for the “bookmark list” resource and the read/write view class for the “bookmark” resource.

The bookmark list view

The bookmark list view function is a nice simple one to start with, because the “bookmark list” resource only responds to GET (see Example 12-12). Remember, I’m exposing bookmark creation through PUT, not through POST on the bookmark list the way the Rails implementation does.

Example 12-12. First try at the bookmark list view

from bookmarks.models import Bookmark
from django.contrib.auth.models import User
from django.core import serializers
from django.http import HttpResponse
from django.shortcuts import get_object_or_404

def bookmark_list(request, username):
    u = get_object_or_404(User, username=username)
    marks = Bookmark.objects.filter(user=u, public=True)
    json = serializers.serialize("json", marks)
    return HttpResponse(json, mimetype="application/json")

The first step is to turn the argument username into a Django User object. The username variable comes from the capture group in the regular expression from Example 12-11. It’s everything between the parentheses in ^users/([\w-]+)/$. Since a request for a non-existent user’s bookmarks should return an HTTP response code of 404 (“Not Found”), I look up a user with Django’s get_object_or_404() shortcut function. This will automatically raise a Http404 exception if the user doesn’t exist, which Django will turn into an HTTP response code of 404. This serves the same purpose as the if_found method defined in the Rails application, way back in Example 7-9.

In Chapter 7 I kept the implementation short by using ActiveRecord’s to_xml function to convert objects from the database (such as user accounts) into XML representations. I’ve used a similar trick here. Rather than represent lists of bookmarks as Atom feeds, I use Django’s serialization API to turn database rows into JSON data structures (see http://www.djangoproject.com/documentation/serialization/).

Django can also serialize database rows into a JSON data structure or an ActiveRecord-like XML representation: switching to the XML representation would be as easy as changing the serializer type in the third line of the view, and the mimetype in the last line. Django’s default JSON output is relatively straightforward. Example 12-13 shows what it does to a row from my bookmarks table.

Example 12-13. A JSON representation of a bookmark

[{
    "pk": "1",
    "model": "bookmarks.bookmark",
    "fields": {
        "tags": ["example"],
        "url": "http:\/\/example.com\/",
        "timestamp": "2007-01-30 21:35:23",
        "long_description": "",
        "user": 1,
        "short_description": "Example",
        "public": true
    }
}]

The bookmark_list implementation from Example 12-12 will work, but it’s a bit naive. It returns all of a user’s bookmarks every time it’s called, and that will chew up both database resources and bandwidth. Example 12-14 shows an implementation that adds support for conditional GET (see Conditional GET” in Chapter 7). Adding handling for Last-Modified and If-Modified-Since does make this view more complex, but the bandwidth savings will make it worth the effort.

Example 12-14. Second try at the bookmark list view

import datetime
from bookmarks.models import Bookmark
from django.contrib.auth.models import User
from django.core import serializers
from django.http import *
from django.shortcuts import get_object_or_404

# Use the excellent python-dateutil module to simplify date handling.
# See http://labix.org/python-dateutil
import dateutil.parser
from dateutil.tz import tzlocal, tzutc

def bookmark_list(request, username):
    u = get_object_or_404(User, username=username)

    # If the If-Modified-Since header was provided, 
    # build a lookup table that filters out bookmarks
    # modified before the date in If-Modified-Since.
    lookup = dict(user=u, public=True)
    lm = request.META.get("HTTP_IF_MODIFIED_SINCE", None)
    if lm:
        try:
            lm = dateutil.parser.parse(lm)
        except ValueError:
            lm = None # Ignore invalid dates
        else:
            lookup['timestamp__gt'] = lm.astimezone(tzlocal())

    # Apply the filter to the list of bookmarks.
    marks = Bookmark.objects.filter(**lookup)

    # If we got If-Modified-Since but there aren't any bookmarks,
    # return a 304 ("Not Modified") response.
    if lm and marks.count() == 0:
        return HttpResponseNotModified()

    # Otherwise return the serialized data...
    json = serializers.serialize("json", marks)
    response = HttpResponse(json, mimetype="application/json")

    # ... with the appropriate Last-Modified header.
    now = datetime.datetime.now(tzutc())
    response["Last-Modified"] = now.strftime("%a, %d %b %Y %H:%M:%S GMT")
    return response

There’s a number of other improvements that could be made here—most notably the ability to show private bookmarks to authenticated users—but you’ve already seen these features in Chapter 7. I’ll leave porting them to Django as exercises, and forge ahead.

The bookmark detail view

The second view I’ll show you has more moving parts. A “bookmark list” resource only responds to GET, but a “bookmark” resource must handle three HTTP methods. GET on a bookmark retrieves a representation of the bookmark, PUT creates or updates a bookmark, and DELETE removes a bookmark. Since we don’t want users modifying each others’ bookmarks, the bookmark resource needs to take authentication into account.

The most obvious way to sketch out the bookmark_detail view is with a series of if clauses:

def bookmark_detail(request, username, bookmark_url):
    if request.method == "GET":
        # Handle GET
    elif request.method == "POST":
        # Handle POST
    elif request.method == "PUT":
        # Handle PUT
    elif request.method == "DELETE":
        # Handle DELETE

However, this is unelegant and will quickly lead to an unwieldy view. Instead, I’ll take advantage of Python’s “duck typing” and implement the bookmark detail view as a callable object. In Python, functions are first-class objects, and the syntax of a function call (object(argument)) is transformed into a method call on the object (object.__call__(argument)). This means that any object can be called like a function, if it defines the __call__ method. I’m going to take advantage of this trick by implementing the bookmark detail as a class with a __call__ method.

This is why the last line of Example 12-11 looks different from the other three. The first three tie regular expressions to Python function objects: bookmark_list and the like. The last one ties a regular expression to a custom object that happens to implement __call__. The __call__ implementation will do some preliminary work, check the HTTP method of the incoming request, and dispatch to an appropriate action function (see Example 12-15).

Example 12-15. The bookmark detail view, part 1: dispatch code

class BookmarkDetail:

    def __call__(self, request, username, bookmark_url):
        self.request = request
        self.bookmark_url = bookmark_url

        # Look up the user and throw a 404 if it doesn't exist
        self.user = get_object_or_404(User, username=username)

        # Try to locate a handler method.
        try:
            callback = getattr(self, "do_%s" % request.method)
        except AttributeError:
            # This class doesn't implement this HTTP method, so return
            # a 405 ("Method Not Allowed") response and list the
	    #allowed methods.
            allowed_methods = [m.lstrip("do_") for m in dir(self)
                                               if m.startswith("do_")]
            return HttpResponseNotAllowed(allowed_methods)

        # Check and store HTTP basic authentication, even for methods that
        # don't require authorization.
        self.authenticate()

        # Call the looked-up method
        return callback()

The BookmarkDetail.__call__ method checks the HTTP method of the incoming requests, and dispatches each one to an appropriate method of the form do_<METHOD>. For instance, a GET request is dispatched to do_GET. Rails does something similar behind the scenes, turning a GET request into a call to MyController#show.

The BookmarkDetail class also needs to handle HTTP basic authentication, so let’s take a look at that now. In a real application, these functions would go into a superclass to be used by every view that required authentication. Think back to Chapter 7, and the way I put the must_authenticate Rails filter into the base ApplicationController class (see Example 12-16).

Example 12-16. The bookmark detail view, part 2: authentication code

from django.contrib.auth import authenticate

class BookmarkDetail:

    # ...

    def authenticate(self):
        # Pull the auth info out of the Authorization header
        auth_info = self.request.META.get("HTTP_AUTHORIZATION", None)
        if auth_info and auth_info.startswith("Basic "):
            basic_info = auth_info[len("Basic "):]
            u, p = auth_info.decode("base64").split(":")

            # Authenticate against the User database. This will set
            # authenticated_user to None if authentication fails.
            self.authenticated_user = authenticate(username=u, password=p)
        else:
            self.authenticated_user = None

    def forbidden(self):
        response = HttpResponseForbidden()
        response["WWW-Authenticate"] = 'Basic realm="Bookmarks"'
        return response

Now we can check self.authenticated_user within the individual do_{METHOD} methods. I’ve also written a forbidden() helper that sends an HTTP 401 (Forbidden) response with the correct WWW-Authenticate header.

Now I’ll implement the “bookmark” resource’s response to each HTTP method it exposes. GET is the simplest, so let’s start there. Example 12-17 shows the implementation of do_GET. It illustrates the same concepts as the bookmark list’s response to GET, back in Example 12-14. The only major difference is that we enforce the privacy of private bookmarks.

Example 12-17. The bookmark detail view, part 3: GET

def do_GET(self):
    # Look up the bookmark (possibly throwing a 404)
    bookmark = get_object_or_404(Bookmark,
        user=self.user,
        url=self.bookmark_url
    )

    # Check privacy
    if bookmark.public == False and self.user != self.authenticated_user:
        return self.forbidden()

    json = serializers.serialize("json", [bookmark])
    return HttpResponse(json, mimetype="application/json")

Next up is PUT (see Example 12-18). This method needs to take an incoming representation of a bookmark’s state, and use it to either create a new bookmark or update an existing one. The incoming representation is available as self.request.raw_post_data, and I use the Django serialization library to turn it from a JSON data structure to a Django database object.

Example 12-18. The bookmark detail view, part 4: PUT

def do_PUT(self):
    # Check that the user whose bookmark it is matches the authorization
    if self.user != self.authenticated_user:
        return self.forbidden()

    # Deserialize the representation from the request. Serializers
    # work the lists, but we're only expecting one here. Any errors
    # and we send 400 ("Bad Request").
    try:
        deserialized = serializers.deserialize("json",
                                               self.request.raw_post_data)
        put_bookmark = list(deserialized)[0].object
    except (ValueError, TypeError, IndexError):
        response = HttpResponse()
        response.status_code = 400
        return response

    # Look up or create a bookmark, then update it
    bookmark, created = Bookmark.objects.get_or_create(
        user = self.user,
        url = self.bookmark_url,
    )
    for field in ["short_description", "long_description",
                  "public", "timestamp"]:
        new_val = getattr(put_bookmark, field, None)
        if new_val:
            setattr(bookmark, field, new_val)
    bookmark.save()

    # Return the serialized object, with either a 200 ("OK") or a 201
    # ("Created") status code.
    json = serializers.serialize("json", [bookmark])
    response = HttpResponse(json, mimetype="application/json")
    if created:
        response.status_code = 201
        response["Location"] = "/users/%s/%s" % \
                                    (self.user.username, bookmark.url)
    return response

After all that, DELETE (Example 12-19) looks very simple.

Example 12-19. The bookmark detail view, part 5: DELETE

def do_DELETE(self):
    # Check authorization
    if self.user != self.authenticated_user:
        return self.forbidden()

    # Look up the bookmark...
    bookmark = get_object_or_404(Bookmark,
        user=self.user,
        url=self.bookmark_url
    )

    # ... and delete it.
    bookmark.delete()

    # Return a 200 ("OK")
    response = HttpResponse()
    response.status_code = 200
    return response

Further directions

The tag views (and all the other interesting features like bundles, etc.) will follow a similar pattern. In fact, with a little work, this BookmarkDetail class could be refactored into a more general purpose resource class for handling many different types of objects.

Conclusion

Django isn’t just a framework for handling HTTP requests. Like Rails, it contains a lot of sub-libraries that handle common problems in web application and web service design. You’ve seen the object-relational mapper that works like Rails’s ActiveRecord, the built-in User model, and the serialization of model objects into JSON representations. Django has many other libraries, including a comment model and a tool for generating syndication feeds. Though it’s mostly used for web applications, Django makes an excellent base for Python implementations of RESTful web services.

The best content for your career. Discover unlimited learning on demand for around $1/day.