Building Web Reputation Systems

Preface

What Is This Book About?

Today’s Web is the product of over a billion hands and minds. Around the clock and around the globe, people are pumping out contributions small and large: full-length features on Vimeo, video shorts on YouTube, comments on Blogger, discussions on Yahoo! Groups, and tagged-and-titled Del.icio.us bookmarks. User-generated content and robust crowd participation have become the hallmarks of Web 2.0.

But the booming popularity of social media has brought with it a whole new set of challenges for those who create websites and online communities, as well as for the users who consume those sites:

• Problems of scale (how to manage—and present—an overwhelming inflow of user contributions)

• Problems of quality (how to tell the good stuff from the bad)

• Problems of engagement (how to reward contributors in a way that keeps them coming back)

• Problems of moderation (how to “stamp out” the worst stuff quickly and efficiently)

Reputation systems can provide an effective solution to all of these problems.

Reputation and Karma: Two Simple Definitions

Reputation

Information used to make a value judgment about an object or person

Karma

The reputation(s) for a user

What is reputation in an online community? In its broadest sense, reputation is information used to make a value judgment about an object or a person. There are potentially many components to reputation. For example, karma is a reputation score for a user in a community, and it may be an aggregation of answers to the following:

• How long has this person been a member of the community?

• What types of activities has she engaged in?

• How well has she performed at them?

• What do other people think about this person?

Having access to a person’s reputation might help you make better-informed judgments. Judgments like:

• Can I trust this person?

• Should I transact with him?

• Is it worth my time to listen to him?

The same principles of reputation can also be applied to that person’s individual contributions—the video, photo, or textual content that he’s authored. Content reputation can answer questions such as:

• What’s the best video in this category?

• What are the most interesting photos submitted today?

• Should I bother reading this blog entry?

• Which comments in a thread are truly worth my time?

For the site operator, reputation is at its most powerful when karma (people reputation) is used in concert with content reputation to increase your understanding of your online community and the relative value of its participants and contents. For the community itself, reputation is relevance. To the site, reputation represents value and return on investment.

So what does a reputation system do? It powers the whole process: it monitors the community, makes note of the actions that community members take, assesses the community’s response to those actions, and keeps a running tally of the history of it all. (Optionally, the system may then show the results of these calculations back to users, enabling them to make judgments about the people and things they encounter.)

Reputation systems are the underlying mechanisms behind some of the best-known consumer websites, including Amazon, eBay, Digg, Slashdot, Facebook, and especially media and gaming sites like iTunes and Xbox Live.

But, of course, developing a successful reputation system is not easy. Our experience is that reputation is one part each of engineering, social science, psychology, and economics. Social media architects and designers must pay careful attention to multiple factors: the context in which the community is situated, the incentives they’d like to provide contributors (which often demand a nuanced understanding of the actions they want to encourage and discourage), and the effects that decisions made early in the design process may have on the community spirit downstream.

Building Web Reputation Systems provides a complete, soup-to-nuts process for designing and developing your own community’s reputation program. In a guided fashion, you will start with the hard questions:

• How can reputation enhance my business? The community?

• What are the right types of behaviors to encourage?

• What are the right objects (people, things?) to accrue reputation?

• What types of actions should count toward a reputation?

and then you’ll learn how to build reputation models to accurately reflect all of these inputs. To do so, you’ll need a reputation grammar, which we’ve developed and applied to many successful reputation systems used across the Web.

We also provide well-diagrammed reputation patterns for common web needs (“Digg-style” voting systems, ratings and reviews, simple karma) and some extended reputation system reviews and a case study based on actual reputation deployments at industry-leading social sites, including Yahoo!, Flickr, and eBay.

Why Write a Book About Reputation?

We wrote this book because we saw how critical reputation has become to the survival and growth of the Web. Though there are many academic research papers on specific generational algorithms and social effects of reputation systems, we couldn’t find a single book that put it all in context—describing it as a separate domain of knowledge, complete with a grammar, emerging best-use patterns, and recurring antipatterns. Until now.

Our extensive experience creating and deploying several reputation systems across a wide range of social media at Yahoo! provided much of our base of knowledge for this book. We add a comprehensive review of other existing applications to venture forth into unexplained territory and offer a working definition of web reputation systems: a survivor’s guide for those who would follow. We hope this book will improve the quality of conversation about reputation systems, and help improve the quality of products that are using them.

Who Should Read This Book

This book and the supporting blog and wiki (http://buildingreputation.com) are targeted at several audiences:

Primary audience

Anyone who is building, operating, or participating in a website or online application that leverages user-generated content: social networking, ratings, votes, reviews, comments, posts, uploads, games, etc. You’ll need to understand reputation systems in order to maximize your user experience and search engine performance, and streamline your operational expenses.

A must-have reference for

System architects, product managers, community support staff, or user interface designers tasked with designing a reputation system for an online community. (Or any variant thereof: a social collaboration tool, voting mechanism, or a ratings-and-reviews experience.)

Also useful to

Game designers who deal with reward and incentive systems for social games or any technologist in a role that requires some familiarity with reputation systems who’d like to become conversant in their design and operations.

Successful social media design is hard. Many companies and organizations understand that. Rewarding points for user participation may generate activity and build audience, but site owners should be wary of possible negative impact on content quality. They need this book to help them choose the right reputation model for their site and help them evaluate the health of their deployed social collaboration system.

This book assumes that you have a rudimentary understanding of application logic, can read basic flowcharts, create content for the Web, and have direct experience interacting with users on socially active websites. No programming or web engineering experience is required, though you’ll be aided through some of the more technical aspects of this book by some passing familiarity with programmatic thinking.

Organization of This Book

This book is broken into three main parts: Reputation Defined and Illustrated, Extended Elements and Applied Examples, and Building Web Reputation Systems. It was written to be consumed sequentially, but if you are already experienced in developing websites heavy with user-generated content, you may be able to move more quickly through Part I.

Part I: Reputation Defined and Illustrated

Chapter 1 briefly introduces basic concepts and terminology of reputation and sets reputation systems in their proper historical context. With this context in mind, Chapter 2 goes on to define a complete grammar for thinking about reputation and provides a graphical language to clearly describe reputation systems for analysis and development.

Part II: Extended Elements and Applied Examples

In Chapter 3, the primitive reputation grammar elements described in Part I are combined into higher-level building blocks that can be used as-is for simple reputation systems or combined to create more complex ones. In Chapter 4, we show how the grammar and blocks we’ve described can be used to elucidate many of the reputation systems currently deployed across the Web, including Digg, eBay, and Flickr. In this part, practitioners’ tips become a regular feature of most chapters; the lessons we’ve learned are not to be missed—even by the casual reader.

Part III: Building Web Reputation Systems

The last section, comprising more than half the book, goes beyond cookie-cutter reputation needs to offer detailed advice on designing, building, deploying, and operating a custom reputation system. The project begins not by drawing a model or screen mock-ups, but by answering the three big questions posed in Chapter 5 that define and limit your reputation choices. With those answers, Chapter 6 helps to identify your application’s objects, methods, and inputs that power your reputation. With all of this in mind, you can start drawing the reputation model.

Chapter 7 helps you decide how to display reputation, and Chapter 8 describes other common uses for reputation, such as providing search relevance. At this point, the reputation model is designed, and the screen mocks are ready. Next up is implementation, testing, and deployment, for which strategies and tips are covered in Chapter 9. We wrap it all up with a case study in Chapter 10. This chapter ties the entire book together, revisiting each of the steps outlined in detail with a single concrete example. It is a successful reputation model that provides inspiration, many insights, and several notes of caution.

Appendix A is intended for technical readers, such as system architects and platform engineers. It describes the reputation framework—an execution environment for multiple reputation models. The Yahoo! Reputation Platform is described in detail as an example of a framework that can scale to tens of thousands of transactions per second but had to make some functional trade-offs that are quite illustrative.

Appendix B contains references for related materials for further reading, most of which are available on the Web.

Role-Based Reading (for Those in a Hurry)

Here are a few alternate chapter reading list recommendations, based on your professional role:

[Product | UX | game] designers and application product managers

We wrote this book primarily for you; Chapters 1 through 10 are all important. If you must skim, be sure to read all of the practitioners tips, warnings, notes, and sidebars to make sure you aren’t missing something important. User experience folks should pay extra attention to the pros and cons in Chapters 7 and 8.

System architects, software engineers, platform engineers

Assuming you’re reading this book as part of a plan to deploy a reputation system, read Chapters 1 and 2 completely—the definitions are important to later sections. Skim Chapter 3, but read all the practitioners tips, and pay close attention to the last half of Chapter 4. In Chapter 5, familiarize yourself with the Content Control Patterns and the limiting effects they have on reputation systems. Chapters 6, 9, and 10 are all worth your full attention. Also look at Appendix A and consider whether you need a reputation framework.

Community support staff, [program | project] managers, operations staff

If you’re involved in a support role with reputation systems, read Chapter 1 and review the definitions in Chapter 2. In Chapter 3, be sure to read the practitioners tips, and likewise the advice about why reputation sometimes fails at the end of Chapter 4. Chapters 7 and 8 provide patterns for how reputation faces the users and the company and explain when (and when not) to use them. You’re probably in a role that is detailed in Chapter 9; if so, read it. Chapter 10 may be the most important chapter in the book for you—nothing like a practical example to get oriented.

Conventions Used in This Book

The following typographical conventions are used in this book:

Italic

Indicates new terms, URLs, email addresses, filenames, and file extensions.

Constant width

Used for program listings, as well as within paragraphs to refer to program elements such as variable or function names, databases, data types, environment variables, statements, and keywords.

Constant width bold

Shows commands or other text that should be typed literally by the user.

Constant width italic

Shows text that should be replaced with user-supplied values or by values determined by context.

Tip

This icon signifies a tip, suggestion, or general note.

Caution

This icon indicates a warning or caution.

Safari® Books Online

Note

Safari Books Online is an on-demand digital library that lets you easily search over 7,500 technology and creative reference books and videos to find the answers you need quickly.

With a subscription, you can read any page and watch any video from our library online. Read books on your cell phone and mobile devices. Access new titles before they are available for print, and get exclusive access to manuscripts in development and post feedback for the authors. Copy and paste code samples, organize your favorites, download chapters, bookmark key sections, create notes, print out pages, and benefit from tons of other time-saving features.

O’Reilly Media has uploaded this book to the Safari Books Online service. To have full digital access to this book and others on similar topics from O’Reilly and other publishers, sign up for free at http://my.safaribooksonline.com.

How to Contact Us

Please address comments and questions concerning this book to the publisher:

O’Reilly Media, Inc.

1005 Gravenstein Highway North

Sebastopol, CA 95472

800-998-9938 (in the United States or Canada)

707-829-0515 (international or local)

707-829-0104 (fax)

We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at:

http://www.oreilly.com/catalog/9780596159795

The authors also have a site for this book at:

http://buildingreputation.com

To comment or ask technical questions about this book, send email to:

[e-mail]

For more information about our books, conferences, Resource Centers, and the O’Reilly Network, see our website at:

http://www.oreilly.com

Acknowledgments

As with any book that reports so much personal experience with a topic, there are more people to thank than we even know or recall—to any we’ve missed, know that we are grateful for the lessons you helped us learn, even if we’re forgetful of your names.

We are first-time authors, so our editorial and publishing supporting cast come foremost to mind:

Mary Treseler, our editor at O’Reilly and our mentor—you helped us learn the ropes and were always supportive when we stumbled.

Havi Hoffman, head of Yahoo! Press—you believed in this project from the beginning, and despite all logistical and legal challenges, you made it happen, along with the unbounded support of your fellow Yahoos: Douglas Crockford, Christian Crumlish, and Neal Sample. Without all of you, there’d be no book at all.

Cate DeHeer, at DutchGirl.com, our main copy editor—you unified our voices and made us both sound great without losing our personality.

Sanders Kleinfeld, Marlowe Shaeffer, Adam Witwer, and the rest of the support staff at O’Reilly—you made it all go as smoothly as possible.

The Yahoo! Reputation Platform team, in its various incarnations: Alex Chen, Matthias Eichstaedt, Yvonne French, Jason Harmon, Chip Morningstar, Dmitri Smirnov, Farhad Tanzad, Mammad Zadeh—you all helped define, implement, operate, and refine one of the world’s finest platforms that provided us with most of the grammar and technical lessons used in this book.

The Yahoo! reputation-enabled product managers: Micah Alpern, Frederique Dame, Miles Libby, Cheralyn Watson, Ori Zaltzman, and so many others—when others scoffed, you were visionary and saw reputation as an unique opportunity to improve your product. So many of the socially oriented stories we’ve used here are a direct result of your pioneering work.

Our author-mentors: Douglas Crockford, Christian Crumlish, Amy Jo Kim, and Erin Malone—you all helped us understand just what it takes (and what it doesn’t) to be an author.

To the readers/commentors on our blog, wiki, and manuscript—by letting us know what you thought as we went along, you significantly improved the first edition of this book. For those of you who comment after this is published—thank you so much for helping us keep this information up-to-date and accurate. Web publishing FTW!

From Randy

First and foremost, I’d like to thank my partner on this project, Bryce Glass, who presented the idea of us writing a book about reputation together just at the time I was feeling the desire to write something but too timid to do it on my own. I knew immediately that this was a great idea, and that he would be the perfect coauthor: I had some the product and engineering experience, and he really understood the UX design issues, as well as being world-class at creating wonderfully simple images to communicate complex concepts. Truly our combined talents produced a book that is greater than the sum of its parts.

Without the explicit encouragement from my wonderful wife, Pamela, this book would never have been started. I began working on it while being nominally unemployed, and at the worst of the 2008 economic downturn. Though I had enough contract work to just barely meet expenses, I could have just continued my search for full-time employment and simply deferred the opportunity to write down my experiences in favor of a steady paycheck. While I was dithering, unsure about taking on the mantle of authorship, she said, “You should go for it!” Her faith in and support for me is an inspiration.

To my parents, Frank and Kathy Farmer, for your constant encouragement to dig ever-deeper into whatever topic I was interested in, I am forever grateful. I hope that sharing my knowledge will help others along a similar path.

Reeve, Cassi, Amanda, and Alice Farmer—you are my pride and joy, and the reason I keep striving to improve the world you will inherit.

I’d also like to acknowledge folks who personally influenced me in significant ways that eventually led me here:

• Thomas Hartsig, Sr., formerly head of the Macomb Intermediate School District Computer Based Instruction group. Tom had the foresight to hire untested high-school programmers to create educational software in the late 1970s. At the MISD I learned that anyone can build a good reputation through hard work and inspiration.

• Steve Arnold, former head of Lucasfilm Games/LucasArts, and everyone there who I worked with during the early 1980s. Nothing convinces you that anything is possible like working for George Lucas.

• Phil Salin, free-market economist, who encouraged me to create reputation systems for his lifelong project The American Information Exchange in the pre-Web 1990s. If he’d only survived and we’d timed it a bit better, we could have been eBay.

• Mark Hull, who hired me into Yahoo! first to create the business plan to build and leverage a reputation platform, then to co-design the Yahoo! 360° social network and help found the Community Platforms group, where the reputation platform would eventually be built.

• Scott Moore and Han Qu, who helped me clarify the Content Control Patterns—thanks, guys!

From Bryce

I, too, would like to thank my coauthor, Randy Farmer. His enthusiasm for, and absolute grasp of, social media and online communities was a large part of what drew me to Yahoo!’s Community Platforms team. Randy—you don’t just work at this stuff, you love it, and your energy is contagious.

The real, untold hero of this book—for me—has been my wife, LeeAnn. While I stole precious evenings and weekends away to work on the book, you cared for our son, Edison, and carried our new son, Evan. You have my endless gratitude and—of course—my undying love.

Thank you to my sons’ wonderful grandparents for the many weekends of babysitting that freed Daddy up for…yes, more writing.

I’d also like to thank several past and present Yahoos: Christian Crumlish—you’ve been a great champion of our book, and a great friend as well; Erin Malone—thank you for your friendship and mentoring, and assigning me to work with the Reputation Platform team; Matt Leacock, who supported that platform before me, and is an all-around amazing UX designer and longtime friend; and finally my last manager at Yahoo!, Amanda Linden, who threw her unabashed support and approval behind the book and my involvement in it.

And finally, I’d like to thank my new team at Manta Media, Inc., particularly my manager, Marty Vian, and fellow designer David Roe. You have been supportive in the extreme in helping me get it to the finish line.