RESTful Web Clients

Preface

“The beginning is the most important part of the work.”

Plato

Web-based REST and hypermedia services are getting more common every day but there are very few client libraries that take advantage of these powerful API features. This is mostly because the techniques and patterns needed to create successful hypermedia clients have been ignored. Yet, when done right, hypermedia-based client applications exhibit more stability and flexibility than typical one-off custom client code.

The aim of this book is to give developers both a solid background and a source of working examples that provide clear recommendations for dealing with hypermedia-style APIs. One of the key ideas in this book is that client applications should rely on something I call the Request, Parse, Wait loop or RPW. This is the way all computer games are implemented and it is the way all event-driven interfaces work from windowing-style workstations to reactive machine interfaces.

I’ve been told that some frontend developers might find the RPW model unusual; one person even characterized my recommendations as “radical.” I can understand that viewpoint. So many of the client libraries and practices today focus on designing specially built one-off user interfaces that are difficult to modify and have a hard time reacting to new information provided by services at runtime. However, after reviewing the examples in this book, I hope that frontend developers—most all of whom are much more skilled than I—will be able to build on this initial work and help create a rich collection of best practices, tooling, and reusable libraries for creating high quality user experiences for the growing number of hypermedia APIs, which meet both the desire for high-quality user experience and the need for adaptive clients that can keep up with evolving service interfaces without the need for constant upgrades.

What’s in This Book

This book takes the reader on a journey from custom bespoke implementations to powerful general-purpose client applications and, along the way, shows how you can harness many of the basic principles that underpin the World Wide Web. The outline is a mix of code-centric chapters and ones that explore important related topics like the representor pattern, human–computer interaction (HCI) modeling, and the challenge of versioning web APIs. Along the way, there is quite a bit of code (over 20 GitHub repos were created for this book). I’ll use small snippets of code within the text of the book, but sometimes those snippets will be hard to understand by themselves. For that reason, I’ll always point the reader to the full code repository online where you’ll find fully functional examples of the ideas covered in the book.

The Chapters

This book explores the world of generic hypermedia-style clients—what they look like, how they differ from typical JSON-object style clients, and how both client and server developers can leverage this style to build systems that are easier to support and adapt over time. The book contains chapters targeting a small set of selected formats (HTML, plain JSON, HAL, Siren, and Collection+JSON) as well as chapters that go into the background on the theories and practices familiar to all web developers, including (1) server-side support for message formats, (2) human–computer interaction modeling, (3) versioning, and (4) implementing a single solution that supports multiple hypermedia formats while interacting with several independent backend services.

Most chapters can stand on their own and could be consumed in any order. But I encourage you to treat the book as a single journey and read it from start to finish in order to get the most out of it. Here is a brief look at how that journey will unfold:

Chapter 1, Our HTML Roots and Simple Web APIs

This chapter introduces us to a classic HTML-only client. We’ll use this to quickly review how browsers work and how they’ve affected some people’s view of what’s possible with hypermedia formats on the Web. This chapter also covers the process of converting an HTML-only service output into an initial JSON-only API service. This service will be the basis for all the other client apps we build for this book.

Chapter 2, JSON Clients

Most client-side web developers have learned to build “JSON clients.” They memorize URLs, consume static objects, and navigate over fixed workflow paths. This is a great way to start but turns out to be a terrible way to work over time. In this chapter we’ll explore ways to overcome the challenges of maintaining plain JSON-style client apps.

Chapter 3, The Representor Pattern

The representor pattern is a simple—and vital—way to deal with web API server output. It is the process of converting internal object models into external message models. We’ll review the pattern (including its roots) and cover how we’ll apply it for providers using the Web Service Transition Language (WeSTL) on the server and the HTML DOM on browser clients.

Chapter 4, HAL Clients

The HAL media type is currently one of the more popular hypermedia formats. It is used, for example, by the Amazon Web Services team for at least two of their APIs. HAL handles one of the three important elements all web clients deal with: Addresses. We’ll see what it’s like to build a generic client using HAL as the message format and introduce the HAL-FORMS extension for improving its action-handling capabilities.

Chapter 5, The Challenge of Reusable Client Apps

You’ll notice that most of the client applications we’ll build here look similar. Essentially, we’re building “explorers” that can—in some limited way—fend for themselves as they explore the world around them. These clients follow a pattern called the Request, Parse, Wait loop, or RPW. This pattern is based on several classic notions of how we interact with our world, and we’ll explore them here.

Chapter 6, Siren Clients

Another powerful hypermedia type is the Siren content type. Currently used as part of the Zetta IoT platform, Siren is designed to handle two of the three key tasks of a web client: Addresses and Actions. We’ll see what it’s like to build a generic client using Siren as the message format and we’ll explore a Siren extension (Profile for Object Display, or POD) that enhances its ability to handle metadata for UI displays.

Chapter 7, Versioning and the Web

What happens to the notion of API versions when you start using hypermedia types as the basis for client-side web applications? This chapter looks at the various attempts to manage change over time and how relying on message-based hypermedia-style APIs reduces the need to change the interface contract when you change the interface features.

Chapter 8, Collection+JSON Clients

In this chapter we’ll explore another hypermedia format—Collection+JSON, or Cj. Cj is able to handle all three of the key elements of web clients: Objects, Addresses, and Actions. We’ll see what it’s like to build a generic client using Collection+JSON as the message format and learn how to extend Cj’s data display and validation routines.

Chapter 9, Hypermedia and Microservices

What would it take to create a single generic hypermedia client that could seamlessly consume multiple services? And what if those services each emitted different hypermedia media types? What would it take to craft a single client application that is “multilingual” when it comes to message format? We’ll see how that looks in our final chapter.

The Dialogues

All of the chapters in this book start and end with short vignettes or dialogues. These are fictional conversations offered as a way to illustrate some of the challenges and thought processes that happen in any organization working to implement scalable, robust projects on the WWW. These dialogues also act as stage settings and summaries for the primary theme of each chapter.

The dialogues are also meant to act as prompts to give you a chance to start thinking about the challenge as presented and how you would approach it and come up with a solution. It may be helpful to start by reading through the dialogues and then take a few minutes to map out (maybe even write down) your ideas about how to solve the problem. Taking the time to interact with the material in this way can result in some additional insight into both the solution that is provided and your own skills in problem solving.

Finally, I’ve added these dialogues so that those who are interested in only skimming the book can still get quite a bit out of the experience. Hopefully, you can read through the dialogues alone and glean the basics of the material. The chapters are then the place where the details and subtleties of the material is explored in depth. You may find that, for at least some of the chapters, reading the dialogues gives you all the information you are looking for on a particular topic or challenge—and that’s fine, too.

The Artwork

The book includes diagrams and artwork by some very talented people. Dana Amundsen, an accomplished artist based in Louisville, KY, worked with me to create the characters Carol and Bob who appear throughout the book. She also designed the BigCo logo you see in the sample apps. It turns out Dana created much more material than I could include in this book and I hope I can share that art via other means in the near future.

The diagrams you see in this book were created by my good friend, Diogo Lucas; a very skilled software developer, architect, speaker, and teacher. I first met Diogo while on a trip to Brazil (where he lives) when he showed me his amazing dashed-off sketches of some of the ideas I’d been working on for this book. I jumped at the chance to invite him to create the illustrations and, luckily, Diogo agreed to lend his considerable artistic talents to this project.

Licensing

All the artwork in this book created by Dana and Diogo is licensed under Creative Commons - Attribution-NoDerivatives 4.0 International (CC BY-ND 4.0).

Both Dana and Diogo have added something special to the book and I thank them both very much.

What’s Not in This Book

This book covers quite a bit of ground in a relatively short number of pages. In order to do that, I needed to leave out quite a bit of helpful material. Here’s a quick rundown of what I decided to not include in this book.

User Interface Design

While I refer to some material on user interface design in the book, those references are cursory. I also do not spend any time on the details of human–computer interaction (HCI) or design methodology in general.

I should also give the reader a heads-up that this book provides a basic UI look-and-feel in the examples. My primary focus is on the network- and message-level techniques that allow services to provide recognizable and useful hypermedia hints to client applications. I also spend time on client-level coding best practices for parsing and activating these hypermedia signals. I leave the work of decorating these interfaces with improved visual appeal to others who are more skilled in this art.

Special thanks to Benjamin Young

I need to give special recognition to my long-time friend and skilled web developer, Benjamin Young. He took the time to extensively review my rudimentary UI designs and provided a coherent look-and-feel to all the client applications. Truth be told, I gave Benjamin limited room to be creative and he still produced a solid, consistent style guide for all of the apps. If they look at all pleasing, that’s a reflection of Benjamin’s talent and persistence. If they fall short in any way, that’s because I didn’t allow him the freedom he needed.

Basics of Hypermedia

This book also does not spend much time on the value of hypermedia as an implementation style or go very far to make the case for using hypermedia in your own projects. My work on previous books, RESTful Web APIs (with Leonard Richardson) and Building Hypermedia APIs with HTML5 and Node (both from O’Reilly), are better sources for the history and value of using hypermedia for web APIs, and I encourage the reader to look there and to other sources for this kind of information.

Programming with HTML, CSS, and JavaScript

Finally, I don’t devote any pages to the basics of programming on the Web in general, such as the topics of HTTP protocol, HTML5, JavaScript, and CSS. These are all very important subjects and well beyond the scope of this small book. There are many books on these subjects and I am confident the reader can find what they are looking for elsewhere.

Source Code

There is quite a bit of related source code for this book. Almost every chapter has at least one relevant project and some have two or more. Including all the source code in the pages of this book would make it tedious to read and difficult to navigate. For that reason, only short snippets of important code blocks appear in the text.

The entire set of source code is available in a public Git repo. The repos will be updated from time to time and should be considered the most accurate source for any code that appears in the pages of this book. Readers are encouraged to clone/fork the code in the repos and create pull requests for any needed changes.

External References

Throughout the book, I make references to external source material including other books, published papers, articles, public standards, and blog posts. In the text, I will mention the reference but not include a direct citation or footnote as that tends to interrupt the flow of the material. Instead, I will add a section at the end of each chapter that includes the name of the source and, where appropriate, online links to the cited material.

Conventions Used in This Book

The following typographical conventions are used in this book:

Italic

Indicates new terms, URLs, email addresses, filenames, and file extensions.

Constant width

Used for program listings, as well as within paragraphs to refer to program elements such as variable or function names, databases, data types, environment variables, statements, and keywords.

Constant width bold

Shows commands or other text that should be typed literally by the user.

Constant width italic

Shows text that should be replaced with user-supplied values or by values determined by context.

Tip

This element signifies a tip or suggestion.

Note

This element signifies a general note.

Warning

This element indicates a warning or caution.

O’Reilly Safari

Note

Safari (formerly Safari Books Online) is a membership-based training and reference platform for enterprise, government, educators, and individuals.

Members have access to thousands of books, training videos, Learning Paths, interactive tutorials, and curated playlists from over 250 publishers, including O’Reilly Media, Harvard Business Review, Prentice Hall Professional, Addison-Wesley Professional, Microsoft Press, Sams, Que, Peachpit Press, Adobe, Focal Press, Cisco Press, John Wiley & Sons, Syngress, Morgan Kaufmann, IBM Redbooks, Packt, Adobe Press, FT Press, Apress, Manning, New Riders, McGraw-Hill, Jones & Bartlett, and Course Technology, among others.

For more information, please visit http://oreilly.com/safari.

How to Contact Us

Please address comments and questions concerning this book to the publisher:

• O’Reilly Media, Inc.
• 1005 Gravenstein Highway North
• Sebastopol, CA 95472
• 800-998-9938 (in the United States or Canada)
• 707-829-0515 (international or local)
• 707-829-0104 (fax)

We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at http://bit.ly/restful-web-clients.

To comment or ask technical questions about this book, send email to [e-mail].

For more information about our books, courses, conferences, and news, see our website at http://www.oreilly.com.

Find us on Facebook: http://facebook.com/oreilly

Follow us on Twitter: http://twitter.com/oreillymedia

Watch us on YouTube: http://www.youtube.com/oreillymedia

Acknowledgments

A book like this does not happen without lots of input and assistance from several people. I am especially grateful to all those who volunteered to read and comment on early drafts of the book. In particular, I’d like to thank Todd Brackley, Carles Jove i Buxeda, Pedro Felix, Mark Foster, Toru Kawamura, Mike Kelly, Steve Klablnik, Ronnie Mitra, Erik Mogensen, Irakli Nadareishvili, Leonard Richardson, Kevin Swiber, Stefan Tilkov, Ruben Verborgh, and Jeremy Wilken for all their feedback and advice in shaping the book.

I’d also like to thank the team at O’Reilly for all their support and trust. Especially Meg Foley and Simon St.Laurent, who had the unenviable task of constantly prodding me and encouraging me through the long process of making this book a reality. Also, a big thanks to my production editor Colleen Lobner, who had to endure my endless picking at minor layout and font details while she worked to make my material presentable. And I’d especially like to thank my colleagues in the API Academy and CA Technologies for all their help and support throughout the project.

Finally, thanks to my family for once again having to deal with me as I worked through my ideas, hunkered down with headphones for long stretches, and as I stared off into space pondering details and testing scenarios in my head. They dealt with my temporary malady as best they could—with patience and humor. Thanks.