Chapter 6. Designing Read/Write Resource-Oriented Services
In Chapter 5 I designed a fantasy web service that serves map images of various planets,[22]navigation information for moving around the map, and information about places on the planets: restaurants, meteor craters, and so on. That’s a huge amount of data to serve, but it can all be contained in a premade data set. There’s nothing a user can do to put his own data on the server.
Clients for the map service in the previous chapter can do all sorts of interesting things with maps and places, but they can’t rely on the server to track anything except the preset data. In this chapter I expand the scope of the map service. It becomes less like a search engine’s web service and more like Amazon S3 and the Flickr and del.icio.us APIs. It not only serves data, it stores data on its clients’ behalf.
How open should I make the new service? A totally open service would allow users to provide their own versions of everything in the standard data set. Clients could create their own planets, and upload custom maps and databases of places. If I was too lazy to find map data myself (I am), I could even start with an empty database and allow the community to populate my entire data set. That’s what del.icio.us and Flickr did.
Is this a good idea? When designing a web service, which levers of state should you expose, and which should you keep to yourself? That depends on what your users want to do, and how much of their applications you’re willing to write for them.
A client uses a web service because the service has something it wants: some data, a place to store data, or a secret algorithm. A web service is an abstraction layer, like an operating system API or a programming language library. If you wrote a math library for working with infinite series, and all your users started using it to estimate the value of π, you’d probably add that feature as a higher-level library function. That way all your users could use the same well-tested π-estimation code instead of each person writing his or her own implementation. Similarly, if all your users implement the same features on top of your web service, you might help them out by moving those features into the service. If all your users want to add certain kinds of custom data to the data set, you can start supporting a new kind of resource, so they don’t have to define their own local structures.
My goal here is fairly modest: to illustrate the concepts of resource-oriented service design. It’s certainly possible to design a mapping service that starts off with an empty data set and gets everything through user contributions, but such a service would have more moving parts than I’ve got concepts to explain. If I decided to show you that service, this chapter would start out well, but once I’d explained all the concepts I’d still have a lot of domain-specific design work to do, and it would get boring.
I want the map service to have about as many moving parts as I have new concepts to explain. I’m going to expand the previous chapter’s service just enough so that clients can annotate the map with custom places. Every custom place is associated with a user account, and may be public or private to that account.
User Accounts as Resources
If I’m going to let anyone with a web service client annotate our worlds, I need some way of distinguishing his custom places from the standard places in my database. I’ll also need a way to distinguish one user’s places from everyone else’s places. Basically, I need user accounts.
When a client annotates Earth or Mars with a custom place, the place he has created is associated with his user account. This way a client can find his place later. If the client chooses to expose that place publicly, other clients will see links to it in the representations they fetch.
Most existing web services have some kind of system for letting people sign up for user accounts or “API keys.” Even services that only give read-only access often make you sign up for an account, so they can track and ration your usage. If you’ve followed along with all of the examples in the book, by this time you have an Amazon Web Services account, a del.icio.us account, and a Flickr account.
Yahoo! Web Services does things a little differently. Instead of tying the key to you personally, you can sign up for any number of application keys. You can distribute the application key with your application, and anyone can use it. Yahoo! tracks application usage, not individual usage. I registered the key “restbook” for a particular “application”: this book. You and anyone else can use that key to run the sample Yahoo! Web Services code in this book.
The procedure for signing up for these web accounts doesn’t vary much. You use your web browser to go to a web site and fill out some HTML forms. You usually have to click through a legal agreement, and maybe respond to a verification email. Sometimes your web service account is tied to your preexisting account on the corresponding web site.
The user account system I’m about to design works a little differently. In my map service, user accounts are resources, just like the maps themselves. In fact, they’re my first read/write resources. My clients won’t have to use their web browsers to sign up for a user account: they can create one with a generic web service client.
Why Should User Accounts Be Resources?
Why have I decided to design my user accounts differently from those of nearly every existing web service? I have two reasons. First: most web services make you sign up for an account through a web application. Web application design is a well-understood topic and it’s not the topic of this book. Web services are indeed very similar to web applications, but resource creation is one of the places where they differ. The main difference here is that HTML forms currently support only GET and POST. This means web applications must use overloaded POST to convey any unsafe operation. If I tried to cover the typical method of getting a user account, I’d end up skimming the details as not relevant to web services. Treating user accounts as read/write resources means I can demonstrate the new resource-oriented design procedure on a data structure you’re probably familiar with.
Second, I want to show that new possibilities open up when you treat everyday data structures as resources, subject to the uniform interface. Consider an Internet-connected GPS device that ties into my map service. Every hour or so, it annotates Earth (as exposed through my web service) with its current position, creating a record of where the GPS device is over time.
There will be thousands of these devices, and each one should only be able to see its own annotations. The person in charge of programming the device should not be limited to creating a single user account for personal use. Nor should everyone who buys the device have to go to my web site and fill out a form before they can use the device they bought.
Since user accounts are resources, every one of these devices can have its own account on my web service (possibly with a username based on the serial number), and these accounts can be created automatically. They might be created in batches as the devices are manufactured, or each one may create an account for itself when its owner first turns it on.
The end users may never know that they’re using a web service, and they’ll never have to sign up for a key. The device programmer does need to know how our web service works, and needs to write software that can create user accounts. If user accounts are resources, it’s obvious how the device programmer can do this. HTTP’s uniform interface gives most of the answers ahead of time.
Authentication, Authorization, Privacy, and Trust
Once I start exposing user accounts, I need some way of determining which user, if any, is responsible for a given HTTP request. Authentication is the problem of tying a request to a user. If you want to name a new place on Mars, I need some way of knowing that the new place should be associated with your user account instead of someone else’s. Authorization is the problem of determining which requests to let through for a given user. There are some HTTP requests I’d accept from user A but reject from user B: requests like “DELETE user A” or “GET all of user A’s private places.” In my service, if you authenticate as user A, you’re allowed to manipulate user A’s account, but not anyone else’s.
I’ll have more to say about RESTful modes of authentication and
authorization in Chapter 8, but here are the
basics. When a web service client makes an HTTP request, it may
include some credentials in the HTTP header Authorization. The service examines the
credentials, and decides whether they correctly identify the client as
a particular user (authentication), and whether that user is actually
allowed to do what the client is trying to do (authorization). If both
conditions are met, the server carries out the request. If the
credentials are missing, invalid, or not good enough to provide
authorization, then the server sends a response code of 401
(“Unauthorized”). It sets the WWW-Authenticate response header with
instructions about how to send correct credentials in the
future.
There are several standard kinds of authentication. The most common are HTTP Basic, HTTP Digest, and WSSE. Some web services implement custom forms of authentication: in Chapter 3 I showed how Amazon S3 implements authentication with a sophisticated request signing mechanism. It doesn’t really matter which authentication mechanism I choose since I’m not actually implementing this service, but let’s say I go with the simplest choice: HTTP Basic authentication.
There’s also the notion of privacy. Given that user A’s list of
private annotations can’t be accessed by any other user, the
representation of that list still needs to be transmitted over the
Internet. The data’s going to go through a lot of computers before it
gets to the client. What’s to stop one of those computers from
examining the supposedly private list? To solve this problem I’m going
to encrypt each HTTP transaction over SSL. In the previous chapter I
presented URIs that started with
http://maps.example.com/. In this chapter my URIs
all start with https://maps.example.com/.
Using HTTPS instead of HTTP prevents other computers from eavesdropping on the conversation between client and server. This is especially important when using HTTP Basic authentication, since that authentication mechanism involves the client sending its credentials in plain text.
Now I’ve got a secure, trusted means of communication between the client and the server. But there’s one more relationship to consider: the relationship between the client software and the human end user. Why should the end user trust the client software with its authentication credentials? Let me ask you a question to clarify the problem. Whenever you log in to a web site, you’re trusting your web browser to send your username and password to that web site, and nowhere else. Why do you trust your browser with that information? How do you know your browser doesn’t have a secret backdoor that broadcasts everything you type to some seedy IRC channel?
There are several possible answers. You might be using an open source browser like Firefox, which has good source control and a lot of people looking at the source code. You might say there’s safety in numbers: that millions of people use your brand of browser and there haven’t been any problems traceable to the browser itself. You might monitor your network traffic to make sure your browser is only sending the data you tell it to send. But most people just take it on faith that their web browser is trustworthy.
That’s the human web. Now imagine I send you a cool new web service client for managing your del.icio.us bookmarks. Do you trust that client with your del.icio.us username and password? Do you trust it as much as you trust your web browser with the same information? Hopefully not! No web service client is as popular as a web browser, and no web service client has as many eyes on the source code. On the human web, we usually ignore the problem by taking a leap of faith and trusting our web browsers. On the programmable web the problem is more obvious. We don’t necessarily trust our own clients with our authentication credentials.
There’s nothing in the HTTP standard to deal with this problem, because it’s a problem between the end user and the client: HTTP lives between the client and the server. Solving this problem requires forgoing all the standard ways of sending authentication information: Basic, Digest, and WSSE don’t work because they require the client to know the credentials. (You can solve it with Digest or WSSE by having a tiny, trusted account manager send encrypted authentication strings to the actual, untrusted client. I don’t know of any web service clients that use this architecture.)
Big names in web services like Google, Amazon, eBay, and Flickr have come up with ways for a client to make web service requests without knowing the actual authentication credentials. You saw a hint of this in Chapter 3: I showed how to sign an Amazon S3 request and give a special URI to someone else, which they could use without knowing your password. I’ll have more to say about this in Chapter 8. For now I just want you to know that there’s a complication on the programmable web you might never have considered. Because there’s not yet any standard way of solving this problem, I’m going to punt on it for now and use HTTP Basic authentication for my services. My users will have to trust their clients as much as they trust their web browsers.
Turning Requirements into Read/Write Resources
Now that I’ve identified a new data set (user accounts), I’m going to go through the same design procedure I did for the data set I developed in the previous chapter (planets, places on the planets, maps of the planets, and points on the maps). But the procedure from the previous chapter only suffices for read-only resources. This chapter makes it possible for clients to create, modify, and delete resources. So I’ve added two steps to the procedure (steps 4 and 5).
Figure out the data set
Split the data set into resources
For each kind of resource:
Name the resources with URIs
Expose a subset of the uniform interface
Design the representation(s) accepted from the client
Design the representation(s) served to the client
Integrate this resource into existing resources, using hypermedia links and forms
Consider the typical course of events: what’s supposed to happen?
Consider error conditions: what might go wrong?
Figure Out the Data Set
Most sites with user accounts try to associate personal information with your account, like your name or email address. I don’t care about any of that. In my map service, there are only two pieces of information associated with a user account:
The name of the account
A password used to access the account
Each user account also has some subordinate resources (custom places on planets) associated with it, but I’ll figure that part out later. All I need for now is a way of identifying specific user accounts (a username), and a way for a client to present credentials that tie them to a certain user account (a password).
Since I don’t track any personal information, there’s no reason apart from tradition to even call this a “user account.” I could call it a “password-protected set of annotations.” But I’ll stick to the traditional terminology. This makes it easier to visualize the service, and easier for you to come up with your own enhancements to the user account system.
Split the Data Set into Resources
This was a fairly large step back in Chapter 5, when my data set was large and vague: “planets, places, and maps.” Here the data set is fairly constrained: “user accounts.” I’ll expose each user account as a resource. In terms of the Chapter 5 terminology, these new resources are resources of the second type. They’re the portals through which my service exposes its underlying user objects. Another site might also expose the list of user accounts itself as a one-off resource, or expose algorithmic resources that let a client search the list of users. I won’t bother.
Name the Resources with URIs
This part is also easy, since I only have one kind of resource. I’ll
expose a user account with a URI of the following form: https://maps.example.com/user/.{user-name}
Expose a Subset of the Uniform Interface
This is the first new step. I skipped it when designing read-only resources, because there was nothing to decide. By definition, read-only resources are the ones that expose no more than the HTTP methods GET, HEAD, and OPTIONS. Now that I’re designing resources that can be created and modified at runtime, I also have PUT, POST, and DELETE to consider.
Even so, this step is pretty simple because the uniform interface is always the same. If you find yourself wishing there were more HTTP methods, the first thing to do is go back to step two, and try to split up your data set so you have more kinds of resources. Only if this fails should you consider introducing an element of the RPC style by making a particular resource support overloaded POST.
To reiterate the example from Chapter 5: if you have resources for “readers,” and resources for “published columns,” and you start thinking “it sure would be nice if there was a SUBSCRIBE method in HTTP,” the best thing to do is to create a new kind of resource: the “subscription.” As HTTP resources, subscriptions are subject to HTTP’s uniform interface. If you decide to forgo the uniform interface and handle subscriptions through overloaded POST on your “reader” resources, defining the interface for those resources becomes much more difficult.
I can decide which bits of the uniform interface to expose by asking questions about intended usage:
Will clients be creating new resources of this type? Of course they will. There’s no other way for users to get on the system.
When the client creates a new resource of this type, who’s in charge of determining the new resource’s URI? Is it the client or the server? The client is in charge, since the URI is made up entirely of constant strings (
https://maps.example.com/user/) and variables under the client’s control ({user-name}).
From those two questions I get my first result. To create a user account, a client will send a PUT request to the account’s URI. If the answer to the second question was “the server’s in charge of the final URI,” I’d expect my clients to create a user by sending a POST request to some “factory” or “parent” URI. See the Custom Places” section later in this chapter for a case where the answer to the second question is “the server’s in charge.”
Will clients be modifying resources of this type? Yes. It’s questionable whether or not a user should be allowed to change his username (I’m not going to allow it, for simplicity’s sake), but a user should always be allowed to change his password.
Will clients be deleting resources of this type? Sure. You can delete an account when you’re done with it.
Will clients be fetching representations of resources of this type? This is up for debate. Right now there’s not much information associated with a user account: only the username, which is part of the URI, and the password, which I won’t be giving out.
I’m going to say yes, which means I will be exposing GET and HEAD on user account resources. If nothing else, clients will want to see whether or not their desired username already exists. And once I allow users to define custom places, clients will want to look at the public places defined by specific users.
Design the Representation(s) Accepted from the Client
My data set comes with no built-in user accounts: every one is created by some client. The obvious next step in this design is to specify how the client is supposed to create a user account.
Let’s go back to Chapter 3 and Amazon S3 for a minute. A client creates an S3 bucket by sending an empty PUT request to the URI of the bucket. The client doesn’t need to send an entity-body in the request, because the bucket has no state other than its name.
To create an S3 object inside a bucket takes a little more work. An S3 object has two bits of state: name and value. The name goes into the URI, the destination of the PUT request. But the value needs to go into the entity-body of the PUT request. S3 will accept any data at all in this entity-body, because the whole point is that the value of an S3 object can be anything, but there needs to be something there: you can’t have an empty object.
Most web services are a little pickier about what goes into the entity-body: it has to be in a certain format and convey certain bits of resource state. My user accounts have two elements of resource state: the username and the password. If a PUT request is going to succeed in creating a user account, it needs to convey both pieces of state. The username is included in the scoping information: any PUT request that creates an account will have that account’s username in the URI. What about the password?
The client will send the new user’s password in an entity-body, as part of a representation. In Chapter 5, I introduced representations as documents the server sends the client: a way for the server to convey the state of a resource. Representations flow the other way, too. They’re how a client suggests changes to the state of a resource. When you PUT an S3 object, the entity-body you send is a representation of the object. The representation you send with a PUT request is an assertion about the new state of a resource.
In Representing the List of Planets” in Chapter 5 I considered several possible representation formats. I looked at plain text, JSON, XML using a made-up vocabulary, and Atom (XML again, but using a preexisting vocabulary). I decided on XHTML, a preexisting XML vocabulary oriented around marking up human-readable documents. In that chapter the question was what format would be most useful when served to the client. Now, the question is how the client should format its proposed state changes. What format makes it easiest for the client to convey a password to the server?
When the state is complex, it’s helpful for the server to accept the same representation format it sends. The client can request a representation with GET, modify the representation, and then PUT it back, committing its changes to the underlying resource state. As we’ll see in Chapter 9, the Atom Publishing Protocol uses this technique effectively. And, of course, S3 serves the representation of an object byte for byte the way it was when the client first PUT it into the system. S3 doesn’t even pretend to know anything about the meaning of the representations it serves.
Here, I’ve only got one item of state (the password), and it’s not one that the server will ever send to the client. Now’s a good time to introduce a representation format for simple cases like these.
My map service accepts a form-encoded representation when a
client tries to create or edit a user. The only pieces of state I’ve
associated with a user are its name and password. The name goes into the URI and
I’ve decided it can’t change, so my user representations just look
like “password={the-password}”. Example 6-2 is hypothetical Ruby code for creating a user
account with the map service.
require 'rubygems'
require 'rest-open-uri'
require 'cgi'
require 'uri'
def make_user(username, password)
open("https://maps.example.com/user/#{URI.escape(username)}",
:data => CGI::escape("password=#{password}"), :method => :put)
endNote
A couple things to note here. First, I’ve started transmitting
sensitive data (passwords) over the network, so I’m now using HTTPS.
Second, I’m actually using two different kinds of encoding in this
code sample. The username, which goes into the URI, is URI-encoded
using URI.escape. The password,
which goes into the representation, is form-encoded with CGI::escape. URI-encoding is similar to
form-encoding, but it’s not the same, and confusing them is a common
source of subtle bugs.
Changing an account’s password is the same as creating the
account in the first place. The client sends a PUT request to the
account URI, with a new representation of the account (that is, the
new password). Of course, no one can change an account’s password
without authorization. To modify a user account, a client must also
provide an Authorization header
that convinces my service it has the right to modify that account. In
short, changing a user’s password requires knowing the current
password. As I said earlier, my service expects incoming Authorization headers to conform to the HTTP
Basic authentication standard.
A DELETE request never requires a representation, but deleting a
user from my service will require a proper Authorization header. That is: to delete a
user account you must know that user’s password.
Design the Representation(s) to Be Served to the Client
A client will GET a user account’s URI to retrieve a representation of a user account, just as a client GETs the URI of a map or a place to retrieve a representation of that map or place. What should the representation of a user account look like?
Right now it won’t look like much, since I’ve only got two pieces of state to convey, and one of them (the password) I don’t want to be sending out. Indeed, in a well-designed system I won’t even have the password to send out. I’ll only have an encrypted version of it, for use in authentication. Once I integrate custom places into this representation, it’ll look better. For now, Example 6-3 is a fairly sparse XHTML document.
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
<head>
<title>User homepage for leonardr</title>
</head>
<body>
<p class="authenticated">
You are currently logged in as
<a class="user" href="/user/leonardr">leonardr</a>.
</p>
<p>User homepage for
<a class="user" href="/user/leonardr">leonardr</a></p>
<form id="modifyUser" method="put" action="">
<p>Change your password:
<input class="password" name="password" /><br />
<input class="submit" /></p>
</form>
</body>
</html>Once again I’m using the representation to convey the current resource state, and to help the client drive to other states. I used an HTML form to describe a future PUT request the client might make if it wants to change the user’s password (an item of resource state). Note that there’s no form telling the client how to get a representation, or how to delete this user. It’s taken for granted that you use HTTP GET and DELETE for that. I only need hypermedia for complicated things: links to other resources (so the client knows which URI to GET or DELETE), and descriptions of representations.
Note
You may have noticed a problem in Example 6-3. Its form specifies an HTTP method of PUT, but HTML forms only allow GET and POST. As with the “repeat” syntax in Example 5-11, I’m using the as-yet-unreleased XHTML 5 to get around the shortcomings of the current version of HTML. Another way to handle this is to send a WADL snippet instead of an HTML form, or use the trick described in Chapter 8 to run PUT requests over overloaded POST.
If you GET someone else’s user account, you’ll be served a different representation, similar to the one in Example 6-4.
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
<head>
<title>User homepage for samruby</title>
</head>
<body>
<p class="authenticated">
You are currently logged in as
<a class="user" href="/user/leonardr">leonardr</a>.
</p>
<p>User homepage for <a class="user" href="/user/samruby">samruby</a></p>
</body>
</html>This representation has no controls for altering the state of
the resource, because the client isn’t authorized to do that: the
client authenticated as leonardr
and this is samruby’s page. Right
now the representation does nothing but confirm to leonardr that a user named samruby exists. If there was no such user, a
GET request to /user/samruby would
give a status code of 404 (“Not Found”), and the client would be free
to create samruby with a PUT
request.
Link This Resource to Existing Resources
In the previous chapter I defined several classes of resource: the list of maps, individual maps, places, and lists of places (that is, lists of search results). None of these are directly relevant to user accounts, but there are a couple of nice features I can add at this point.
One nice feature is to add the “authenticated” message (seen in the two sample representations above) to the representation of every resource. It’ll be displayed whenever the client submits a request with valid credentials. The “authenticated” message is a piece of hypermedia that shows an authenticated client how to retrieve data about its user account. Every resource is now connected to the user account of the user who requested it.
Another nice piece of hypermedia would be one that shows an unauthenticated client how to create a user account. The best place for this bit of hypermedia would be the representation of the list of planets: after all, that’s the service’s “home page.” It already contains links to the other main parts of the service, so it should contain a link to this new part.
Once again, HTML hypermedia isn’t quite up to the job. And once again, I’m going to use XHTML 5, which makes minor changes to HTML, rather than introduce a totally new technology like WADL in the middle of a chapter. Example 6-5 is an XHTML 5 snippet that tells a client how to create a user.
<form id="createUser" method="PUT" template="/user/{username}">
<p>Username: <input type="text" name="username" /><br />
<p>Password: <input type="password" name="password" /><br />
<input class="submit" />
</form>Note
The two deviations from the HTML you’re familiar with are in
the method attribute (like Example 6-3, it specifies PUT
where HTML 4 allows only GET and POST), and the brand-new template attribute, which inserts a form
variable (“username”) into the URI using the URI
Templating standard.
As of the time of writing, URI Templating was a proposed addition to HTML 5, but it hadn’t been approved. It’s possible that it will be rejected, and that Example 6-5 won’t be valid HTML 5 any more than it is valid HTML 4. In that case you can use URI Templating unofficially (forcing your users to write custom clients), or switch to WADL.
The hypermedia form talks about the syntax of the PUT request, but it can’t say much about the semantics. A web service client can read the HTML form in Example 6-5, but its understanding is limited. It knows that the form is labelled “createUser” but it doesn’t know what “createUser” means. It knows that if it PUTs a certain representation to a certain URI, the server will probably accept it. It knows what PUT means, because PUT always means the same thing. It knows that the representation should include a “username,” but it doesn’t know a username from an ostrich. It takes a human being—a programmer—to understand what a user is, that “createUser” means “create a user,” what a username is, and all the rest. A programmer needs to set the rules about when and how user accounts are created. This piece of hypermedia does nothing but tell the client how to structure the PUT request when it comes time to “createUser,” whatever that means. It’s a promise from the web service to the client.
Many web services put all of this data up front, in a single WSDL or WADL file, for the ease of the client programmer. This is somewhat contrary to the REST design philosophy because it violates, or at the very least subverts, the principle of connectedness. But in web services, where the client must be programmed in advance, it’s an understandable impulse, and often it doesn’t cause any problems.
What’s Supposed to Happen?
Let’s consider what might happen when a client sends a PUT request to /user/leonardr. As is usual with HTTP, the server reads this request, takes some action behind the scenes, and serves a response. I need to decide which numeric response code the response will have, and what HTTP headers and/or entity-body will be provided. I also need to decide how the request will affect resource state: that is, what real-world effects it will have.
It’s not hard to see what happens if all goes well with a PUT
request. If there’s no user called “leonardr,” the service creates one
with the specified password. The response code is 201 (“Created”), and
the Location header contains the
URI of the newly created user.
If the user account already exists, the resource state is modified to bring it in line with the client’s proposed new representation. That is, the account’s password is modified. In this case the response code may be 200 (“OK”), and the response entity-body may contain a representation of the user account. Or, since the password change never affects the representation, the response code may be 205 (“Reset Content”) and the response entity-body may be omitted altogether.
PUT requests are the only complicated ones, because they’re the only ones that include a representation. GET and DELETE requests work exactly according to the uniform interface. A successful GET request has a response code of 200 (“OK”) and a representation in the entity-body. A successful DELETE request also has a response code of 200 (“OK”). The server can send an entity-body in response to a successful DELETE, but it would probably contain just a status message: there’s no longer a resource to send a representation of.
What Might Go Wrong?
A request that creates, modifies, or deletes a resource has more failure conditions than one that just retrieves a representation. Here are a few of the error conditions for this new resource.
The most obvious problem is that the client’s representation might be unintelligible to the server. My server expects a representation in form-encoded format; the client might send an XML document instead. The status code in this case is 415 (“Unsupported Media Type”).
Alternatively, the client might not have provided a representation at all. Or it might have provided a form-encoded representation that’s ill-formed or full of nonsense data. The status code in this case is 400 (“Bad Request”).
Maybe the representation makes sense but it tells the server to put the resource into an inconsistent or impossible state. Perhaps the representation is “password=”, and I don’t allow accounts with empty passwords. The exact status code depends on the error; in the case of the empty password it would probably be 400 (“Bad Request”). In another situation it might be 409 (“Conflict”).
Maybe the client sends the wrong credentials, sends
authorization credentials for a totally different user account, or
doesn’t send the Authorization
header at all. A client can only modify or delete a user if it
provides that user’s credentials. The response code in this
case is 401 (“Unauthorized”), and I’ll set the WWW-Authenticate
header with instructions to the client, giving a clue about how to
format the Authorization header
according to the rules of HTTP Basic authentication.
If the client tries to create a user that already exists, one possible response code is 409 (“Conflict”). This is appropriate because carrying out the PUT request would put the service’s resources into an inconsistent state: there’d be two user resources with the same username. Another possibility is to treat the PUT request as an attempt to change an existing user’s password without providing any authentication, and send a response code of 401 (“Unauthorized”).
As in the previous chapter, there might be an unspecified problem on the server side: response code 500 (“Internal Server Error”) or 503 (“Service Unavailable”).
Custom Places
Now I’m ready to go through the resource design procedure all over again. This time I’m designing the custom places clients can create: places that will show up on maps alongside the built-in places. Hopefully you’re getting the hang of the procedure by now (if not, take heart: I’ll do it some more in the next chapter), so this trip through it will be somewhat abbreviated. This time I want to focus on what makes custom places different from user accounts.
Figure Out the Data Set
A web service client can create any number of places on any of the planets for which I have maps. Custom places will show up in lists of search results, just like the built-in places from the previous chapter. Custom places can have the same data as built-in places: a type (“city”), a name (“Springfield”), coordinates of latitude and longitude (“39.81E 89.64W”), and a textual description (“The capital of Illinois”). Many custom places may share the same coordinates (“My house” and “My current location”), and a custom place may share a location with a built-in place.
Every custom place is associated with some user account. Custom places may be public or private. A private place is visible and modifiable only to someone who provides the credentials for the user account that “owns” the place.
Split the Data Set into Resources
Each custom place will be a resource, just as every built-in place is. I also want to let clients get a list of their custom places. In my design, a user account is just a password-protected list of places, so I won’t be exposing the place list as a separate resource. Instead I’ll expand the “user account” resource so it encompasses a user’s list of places. This is analogous to the way a bucket in Amazon S3 is represented as nothing but a list of objects.
Name the Resources with URIs
A custom place is clearly a subordinate resource, but subordinate to what? I could reasonably associate it with a user account, a geographic point on some planet, or an enclosing place like a city, country, or planet. Which of these relationships should I capture with my URIs?
I’ve chosen to name custom places much the same way I name
built-in places. Each place is associated with a geographic point, and
can be accessed with a URI of the form /user/. The new element is
{username}/{planet}/{latitude},{longitude}/{place
name}{username}, intended to distinguish between
different people’s views of the same place: for instance, Sam’s review
of Joe’s Diner at /user/samruby/Earth/45.2,-114.2/Joe’s%20Diner
and Leonard’s less glowing review at /user/leonardr/Earth/45.2,-114.2/Joe's%20Diner.
A URI like /Earth/USA?show=Joe's+Diner works like it
did before: it returns search results for places called “Joe’s Diner,”
anywhere in the U.S. The only difference is that now there are more
possible places to search: not only the built-in database of places,
but each user’s public list of places, and your own private
list.
Built-in places are still privileged. As it happens, there’s a
Japanese theme park that includes a one-third scale model of Mount
Rushmore. If a client creates a custom place called “Mount Rushmore”
north of Tokyo, /Earth/Mount%20Rushmore still points to the
original in South Dakota. It doesn’t suddenly become ambiguous which
“Mount Rushmore” resource that URI refers to. However, /Earth?show=Mount+Rushmore will show both
places.
Expose a Subset of the Uniform Interface
Clients can use GET and HEAD to retrieve representations of built-in places, their own places (whether public or private), and public places created by others. Clients can delete their own places with DELETE, and change the state of their places with PUT.
There are two ways a client might create a map annotation. The client might add a comment to an existing place on the map (“Mount Rushmore”), or it might give a new name to a certain point of latitude and longitude (“the cornfield where I kissed Betty”).
In the first case, the resource being created is “Mount Rushmore (from leonardr’s point of view).” When creating this resource the client shouldn’t have to know exactly where on the map Mount Rushmore is. “Mount Rushmore” is a consensus name and there’s a built-in place by that name. The client can rely on the server to look up the coordinates. In the second case, the resource being created is a brand new place that the server’s never heard of, and the client is responsible for knowing the coordinates.
How can I work this feature into my resource-oriented design?
“Mount Rushmore (from leonardr’s point of view)” is a
subordinate resource of another resource: the built-in place “Mount Rushmore.”
This resource already exists and has a URI: one of them is /Earth/Mount%20Rushmore. If the client wants
to reuse the consensus name for a place, it shouldn’t have to look up
its location. Instead of figuring out the final URI of the annotation
and sending a PUT request to it, the client can send a POST request to
the “Mount Rushmore” URI and let the server figure out the ultimate
URI.
Similarly, if the client wants to comment on the Alabama capitol
building, it can POST to /Earth/USA/AL/State%20capitol instead of
figuring out the exact coordinates or street address. Any URI that
identifies a built-in place can be the target of a POST request that
comments on that place.
What about custom names? What if a client wants to give the name “Mount Rushmore” not to the original in South Dakota, but to the scale model in Imaichi? What if the client wants to create an annotation for “the cornfield where I kissed Betty”?
Here the client must know the latitude and longitude of the
place it wants to create. This means it’ll have all the information
necessary to create the URI of the new resource: the world, a
geographic point on the world, the name of the place, and its own
username. The client could make a PUT request to a URI like /user/bob/Earth/42,-93.7/the%20cornfield%20where....
This would work just like creating a user account by sending a PUT
request to /user/bob.
Even here, it’s cleaner to use POST. A brand-new place on the map is a subordinate
resource: it’s subordinate to some point on the planet, just like a
comment on a built-in place is subordinate to a place on the planet.
So a client could also put a new place on the map by sending a POST
request to /Earth/42,-93.7. It
works the same way as a comment on existing places (a POST to /Earth/Mount%20Rushmore), except here the
place is identified by latitude and longitude, not by consensus
name.
My service will support POST for brand-new places because that’s simpler. The interface will be the same whether you’re adding a brand new place to the planet, or making a comment on some consensus place. Another service might support both methods: PUT to the final URI if the client is willing to figure out that URI, and POST to a parent URI if it’s not.
Finally, note that although I’m using POST, it’s not overloaded
POST. Clients of my service use POST only when they want to create a
resource “beneath” an existing one. The URI of the new resource
(/user/leonardr/Earth/43.9,-103.46/Mount%20Rushmore)
may not directly extend the URI of the old (/Earth/Mount%20Rushmore), but the resources
have a conceptual relationship.
Design the Representation(s) Accepted from the Client
When the client sticks a pin into a planet and creates a custom
place, what information does it need to provide? It must identify a
planet and a place on that planet: the spot where the pin goes. The
place can be identified either by latitude and longitude, or by
reference to a canonical name like “Mount Rushmore.” Call these
variables planet, latitude,
longitude, and name. The server
must know what type of place the client is putting
on the map. A place may be public or not, and the
client may provide a custom description of the
place. The final URI also incorporates a username, but the client is
already providing that, in the Authorization header. There’s no need to
make the client send that information twice.
These are all key-value pairs. I can have clients represent places the way they represent user accounts: as form-encoded strings. There are no complex data structures here that might call for a JSON or XML representation.
Client requests may choose to send some key-value pairs and omit
others. Information that’s in the URI as scoping information doesn’t
need to be repeated in the representation. When the client sends a
POST to /Earth/43.9,-103.46 it
doesn’t need to specify latitude and
longitude, because that information’s in the URI.
It does need to specify name and
type.
When the client sends a POST to /Earth/Mount%20Rushmore it shouldn’t specify
latitude,
longitude, or name. The client
is making a new place based on a well-known existing place, and the
new place will inherit the name and location of the existing place.
The client may specify a custom type
(“national-park,” “political,” “places in North Dakota”) or inherit
the default (“monument”).
The client may always choose to omit
description and public. My
service sets default values for those variables: descriptions are
empty by default, and places are public by default.
When the client modifies one of its custom places, anything and everything about the place might change: its name, its location, its type, its description, or its public status. The PUT request that modifies a place can specify the same key-value pairs used to create a place, in any combination. The server will make the appropriate changes, assuming the changes make sense.
Example 6-6 shows a sample HTTP POST
request that creates a new custom place. Combined, the
form-encoded representation and the scoping information in the URI
convey all required states for the new resource. The name and location
of the new resource come from the scoping information; its type and
description come from the representation. Since the representation
doesn’t specify a value for public,
the default takes over and this new resource is made public.
Design the Representation(s) Served to the Client
Most of the work here is already done. In Chapter 5 I defined an XHTML-based representation format for places. Custom places look the same as places from the built-in database.
The only new part is this: when an authenticated client requests
a representation of one of its custom places, our service will tack
onto the representation some hypermedia showing the client how to edit that place
(see Example 6-7). I don’t need to tell clients
how to delete the place: the uniform interface takes care of that. But
I do need to convey the information I wrote in prose above: that a
place is defined by planet,
latitude, longitude, and so
on.
<form id="modifyPlace" method="PUT" action=""> <p>Modify this place:</p> <p> Name: <input name="name" value="Mount Rushmore" type="text" /><br /> Type: <input name="type" value="national-park" type="text" /><br /> Position: <input name="latitude" value="43.9" type="text" />, <input name="longitude" value="-103.46" type="text" /><br /> Description: <textarea name="description">We visited on 3/5/2005</textarea><br /> Public? <input name="public" type="checkbox" value="on"/> <input type="submit" /> </p> </form>
The caveats from earlier apply here too. This isn’t valid XHTML
4, though it is valid XHTML 5, because it specifies PUT as its
method. Also, a client doesn’t know
what to do with this form unless it’s been programmed in advance.
Computers don’t know what “modifyPlace” means or what data might be a
good value for “latitude.”
Because clients have to be programmed in advance to understand these forms, most of today’s services don’t include a form for modifying a resource in that resource’s representation. They either serve all the forms up front (in a WSDL or WADL file), or they specify them in prose (as I did above) and leave it for the service programmer to figure out. It’s debatable whether it’s really helpful to serve forms along with representations, but serving them is better than just specifying the API in prose and making the programmer implement it.
Link This Resource to Existing Resources
I’ve got three kinds of integration to do. The first is data
integration. When you DELETE a user account, the account’s custom
places—everything under /user/—should
also be deleted. URIs to these resources used to work, but now they
will return a response code of 410 (“Gone”) or 404 (“Not
Found”).{username}
The other kinds of integration should be familiar by now. They involve changing the representations of existing resources to talk about the new one. I want search results to link to custom places. I want points on the globe to show how the user can create a custom place at that point. I want to improve my connectedness by connecting “custom place” resources to the resources I defined already.
The rather empty-looking representation of a user’s account, seen in Example 6-3, badly needs some link-based integration. This is the ideal place to list a user’s custom places. I’ll represent the place list with the same XHTML list of links I use to represent search results.
In the service defined in Chapter 5, a client
that searched for places called “Mount Rushmore” (/Earth?show=Mount+Rushmore) would only find
places from my built-in place database: probably only the “consensus”
location of Mount Rushmore in South Dakota. In the new version of the
service, there’s likely to be more than one result. In the new
version, that search will also return other users’ annotations for
Mount Rushmore, and other places that users have named “Mount
Rushmore,” like the scale model in Imaichi.
This is the same case as in Chapter 5, where the built-in place database contained more than one “Joe’s diner.” I present search results in a list, each linking to a specific resource. All I’m doing is expanding the search. A search result may be a place in the built-in database, a custom place created by some other user and exposed publicly, or a custom place created by the authenticated user (which may be public or private).
I also need to show the client how to create its own places on
the map. Custom places are created as subordinate resources of
existing places. The logical thing to do is to put that information in
the representations of those places: places with URIs like /Earth/Mount%20Rushmore and /Earth/42,-93.7.
Example 6-8 is a
possible representation of /Earth/43.9,-103.46 that brings together
most of what I’ve covered in the past two chapters. This
representation abounds in hypermedia. It links to a certain point on
several different maps, a place from the built-in database, custom
places from other users, and a custom place created by the
authenticated user. It also has a hypermedia form that will let the
authenticated user create a new custom place at these coordinates.
Compare this representation to the smaller representation of /Earth/43.9,-103.46 back in Example 5-9.
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
<head>
<title>43.9°N 103.46°W on Earth</title>
</head>
<body>
<p class="authenticated">
You are currently logged in as
<a class="user" href="/user/leonardr">leonardr</a>.
</p>
<p>
Welcome to
<a class="coordinates" href="/Earth/43.9,-103.46">43.9°N
103.46°W</a>
on scenic <a class="place" href="/Earth">Earth</a>.
</p>
<p>See this location on a map:</p>
<ul class="maps">
<li><a class="map" href="/road/Earth/43.9,-103.46">Road</a></li>
<li><a class="map" href="/satellite/Earth/43.9,-103.46">Satellite</a></li>
...
</ul>
<p>Places at this location:</p>
<ul class="places">
<li>
<a class="builtin" href="Mount%20Rushmore">Mount Rushmore</a>
System data says:
<span class="description">The world's largest sculpture</span>
</li>
<li>
<a class="custom" href="Mt.%20Rushmore/user1">Mt. Rushmore</a>
<a class="user" href="/users/user1">user1</a> says:
<span class="description">Built on land stolen from the Lakota tribe</span>
</li>
<li>
<a class="custom" href="Mount%20Rushmore%20Gift%20Shop/user2">
Mount Rushmore Gift Shop
</a>
<a class="user" href="/users/user1">user1</a> says:
<span class="description">Best fudge I've ever had</span>
</li>
<li>
<a class="custom-private" href="Mount%20Rushmore/leonardr">Mt. Rushmore</a>
You said: <span class="description">We visited on 3/5/2005</span>
</li>
</ul>
<form id="searchPlace" method="get" action="">
<p>
Show nearby places, features, or businesses:
<input name="show" repeat="template" /> <input class="submit" />
</p>
</form>
<form id="createPlace" method="post" action="">
<p>Create a new place here:</p>
<p>
Name: <input name="name" value="" type="text" /><br />
Type: <input name="type" value="" type="text" /><br />
Description:
<textarea name="description"></textarea><br />
Public?
<input name="public" type="checkbox" value="on"/>
<input type="submit" />
</p>
</form>
</body>
</html>What’s Supposed to Happen?
This new resource, the custom place, mostly works like other resources I’ve already defined. A custom place responds to GET just like a built-in place. It responds to PUT (with a representation consisting of key-value pairs) and DELETE (with no representation) just like “user account” resources do. I only have a couple new edge cases to consider here.
When the client creates a custom place, the response code is 201 (“Created”). This works the same way as users. But it was never possible to cause a user’s URI to change, because I prohibited users from changing their usernames. It’s possible to change the name of a place, or to move one (say, a ship) from one point on the map to another. Either of these actions will change the URI.
When the client modifies a custom place without changing its
location, the response code will be 200 (“OK”). If the location
changes, the response code will be 301 (“Moved Permanently”) and the
Location header will contain the
place’s new URI. The client is responsible for updating its data
structures to keep track of the new URI. This ties into a debate I’ll
revisit in Chapter 8, about whether it’s more
important to have URIs that contain useful information, or URIs that
never change. My URIs describe a custom place using two pieces of
resource state: coordinates and name (/user/leonardr/Earth/43.9,-103.46/Mt.%20Rushmore).
If either of those changes, the old URI breaks.
Broken URIs are no fun on the human web, and they’re even less
fun on the programmable web. If my custom “place” is a ship or
something else that’s constantly moving, it effectively has no
permanent URI. This is the single biggest design flaw in my system. If
I were exposing this as a real web service, I’d probably give a
“permalink” to every place: an alternate URI that doesn’t incorporate
any changeable resource state. Since everything about a place can
change except the planet it’s on and the person who owns it, these
URIs will not look very friendly: my annotation of Mount Rushmore
might be accessible from /user/leonardr/Earth/36028efa8. But at least
they’ll always refer to the same place.
What Might Go Wrong?
This new kind of resource introduces new error conditions, but most of them are variations of ones I’ve already covered, so I’ll pass over them quickly. The client might try to move an existing place off of the map by providing an invalid latitude or longitude: the response code is 400 (“Bad Request”), just as it was in a similar case in Chapter 5. The 400 response code is also appropriate when a client tries to create a place without providing all the information the server needs. This is similar to the 400 response code the server sends if the client tells the server to change a user’s password, but doesn’t actually provide the new password.
My service doesn’t allow a single user to define more than one
place with the same name at the same coordinates. /user/leonardr/Earth/43.9,-103.46/Mt.%20Rushmore
can only identify one place at a time. Suppose a client has two places
called “My car,” and makes a PUT request that would move one to the
location of the other. My service rejects this request with a response
code of 409 (“Conflict”). There’s nothing wrong with moving a place to
a certain set of coordinates; it’s just that right now there happens
to be another place with that name there. The same 409 response code
would happen if the client had two custom places at the same
coordinates, and tried to rename one to match the name of the other.
In either case, the client is making a syntactically valid request
that would put the system’s resources into an inconsistent state. It’s
the same as trying to create a user that already exists.
There’s one totally new error condition worthy of attention: the client may try to access a private place created by someone else. There are two possibilities. The first is to deny access with response code 403 (“Forbidden”). The 403 response code is used when the client provides no authentication, or insufficient authentication; the latter certainly applies in this case.
But a response code of 403 is a tacit admission that the resource exists. The server should not be giving out this information. If client A creates a custom place and marks it private, client B should not be able to figure out anything about it, even its name, even by guessing. When revealing the existence of a resource would compromise security, the HTTP standard allows the server to lie, and send a response code of 404 (“Not Found”).
A Look Back at the Map Service
This is still a simple design but it’s got quite a few features. In Chapter 5 my clients could get map images, navigate around a map, bookmark points on the globe, and do geographic searches against a built-in database of places. Now they can keep track of custom places, register comments on consensus places, and share places with other users. The representation in Example 5-6 shows off most of these features.
All of these features are made available through resources that expose the uniform interface. Occasionally I need to supplement the uniform interface with hypermedia forms (here, the XHTML 5 forms) that tell the client what representations can go with a PUT or POST request. The vast majority of requests will be GET requests. These need no hypermedia supplements, because GET always means the same thing.
A client can get right to its desired resource by constructing a URI, or it can get to that resource by navigating links in the hypermedia I serve. You can get anywhere from the service root (the list of planets) by following links and filling out forms. Each resource is fairly simple, but the service as a whole is very powerful. The power comes from the variety of resources, the links that connect them, and the fact that each resource is individually addressable.
The Resource-Oriented Architecture sets down a list of design questions you need to ask yourself. I embodied these questions in the previous chapter’s seven-step design procedure, and this chapter’s extended nine-step procedure. Like any architecture, the ROA imposes design constraints, but it doesn’t make all the design decisions for you. There are many other ways to define a map service in a RESTful and resource-oriented way. It all depends on how you split the data set into resources, what representations you define for those resources, and how you tie them together with hypermedia.
What I’ve designed should work and be useful to clients, but I won’t know for sure, because I don’t have to implement it. I just designed it to illustrate concepts in a book. When designing a real service, you also have implementation issues to consider. You have to write code to back up every decision you make: decisions about what resources you expose, what parts of the uniform interface they respond to, what URIs you choose, and which representations you serve and accept. In the next chapter, I’ll make all these decisions again for a different data set, and this time I’ll back it up with a real implementation.
[22] Remember, I’m using “planets” as a shorthand for “bodies that can be addressed with longitude and latitude.” I don’t just mean whatever 8 or 11 bodies the International Astronomical Union has decided are planets this week.
Get RESTful Web Services now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
