Designing with Data

Preface

Design and Data: A Perfect Synergy

THE MYTH OF THE “genius designer,” someone whose instincts and intuition lead to great design decisions, is a popular one. This seductive myth can lead some to conclude that design is never grounded in data, that design is never an empirical discipline, and that design practice stands in opposition to data and data sciences.

We understand where this myth comes from. On the surface, data science and design practice are not obviously compatible. Design philosophy and practice emphasizes empathy with users, and the creation and crafting of artful user experiences in the form of effective products. For some designers (and for many outside the design world who valorize design as “inspired genius”), design is a nonlinear, exploratory journey. It is a fundamentally human, fluid, and creative process, unlike the procedures and rigors of “science.” Design is emotional. Design cannot be articulated as a set of procedures or steps to be followed. Design cannot be rationalized and constrained. For some, incorporating data into the design process is a cause for concern.

Some concerns we have heard expressed include:

• Data undermines and undervalues the designer’s intuition and experience

• Data stifles creativity and removes the “art” from the design process

• Data dehumanizes the design process, reducing human experience and design evaluation to “just numbers”

• Data overemphasizes minutiae and optimization of small variations in the design

• Data enslaves you—believing in data as way to evaluate designs takes the power away from design as a practice

On the other side, proponents of design through experimentation and data science value measurement. Some see data as rational, and numbers as irrefutable. Data science reveals the truth—data science is a proceduralized, scientific endeavor where rigor leads to irrefutable results and therefore to certainty. Data science is a trustworthy and precise craft. This view is reinforced by the increasing fascination with measures and metrics for business, commonly referred to today as “big data.” An extreme view is that large-scale experiments can be run to gather data from millions of users to answer all design questions and that such analytics can, therefore, replace design. Under this view, font types, colors, and sizes, and questions such as “Should we have a blue or a red dialogue box?”, “Do people engage more with a list or a carousel?”, or “Does a wizard help with new user onboarding flow?” fall under the purview of data science and not design practice. This could be characterized as “Let the crowd speak with their clicks, and what emerges will necessarily be the best design.”

We deliberately present these extreme positions to illustrate a point. We believe that the extreme views we just outlined draw a false dichotomy between data and design. In reality, data sciences and design practices are working toward the same goal: understanding users and crafting elegant experiences for them. Design is and always has been informed by data. The “data” may be an accumulated set of experiences and informally gathered observations that provide the basis for design genius and “craft knowledge.” The data may also be derived from more systematic studies of users’ activities and opinions, such as lab-based studies, field observations, and surveys. Design practice has always been about different forms of data. In an ever-changing marketplace and industry where new applications and new behaviors are constantly emerging, data can play a big role in helping us learn and respond in a timely way to shifts in user interests and needs. By harnessing and leveraging the power of data at scale—that is, data in high volume, often arriving in streams from millions of users, and which may be of disparate types—new ways to understand people, “users,” are emerging. Data at all scales from individuals to millions and hundreds of millions of users—systematically collected, analyzed, communicated, and leveraged—can empower design.

We want to acknowledge that designers’ concerns about large-scale data collection have some grain of truth. Personally, we have all experienced some circumstances and work situations where these criticisms held—for example, where data gathered at scale contradicts what we know or believe to be true about the human experience. Personally, we believe that these potential misalignments in belief and data reflections arise precisely because designers have historically not been included in the experimental process, data collection, and analysis that informs design. We believe that design intent and evaluation are often poorly matched to the data capture and analysis because designers with a desire to understand user experience have not been in effective dialogue with data scientists and machine-learning experts. This is a two-way conversation: design can bring deeper meaning to data. By developing an awareness of and an affinity for data, such conversations will benefit both disciplines. Similarly, design practice can be enhanced by data. When managed well, data science in the form of large-scale experiments can demonstrate the worth of creativity in design rather than stifle it.

In sum, we believe designers have to engage in the practice and business of designing experiments and managing the data collection process, by being part of the conversation about what data should be collected, when, and most importantly why.

Our Focus: A/B Testing

In writing this book, our intended audience is people who know nothing about large-scale experimentation. We will focus on A/B testing, the most common form of large-scale experimentation and data collection in the internet industry. A/B testing is a methodology to compare two or more versions of an experience to see which one performs the best relative to some objective measure. Essentially, A/B testing is the scientific method applied online, at scale. A primary advantage of A/B testing is that you can test “in the wild”—that is, in the often-messy context of the real world—by launching different experiences to a subset of your actual users. Through an A/B test, you can causally attribute changes in user behavior to design changes that you make. This is the best way to get a true understanding of how each of your experiences will impact your users if launched.

There are many steps and considerations in crafting an A/B test, and we will spend the majority of this book providing practical thoughts on how to be involved. At a high level, though, engaging with quantitative testing of this kind can help you create:

• A direct feedback loop with your users, elevating the way you understand and think about user behavior, ultimately helping you to hone your instinct about your users over time

• A stronger bond between user needs and how your business measures success, helping to align a cross-functional team

• A rigorous approach that can help to eliminate hierarchy, rank, and politics from the process of decision making, allowing you to focus on your users’ needs

Using carefully captured and analyzed data will give you a framework for discussions about user behaviors and needs, product effectiveness (including potential innovations), and business goals.

Some Orienting Principles

In writing this book, we have three orienting principles.

One, that design always advocates for users and is accountable to users. Good design brings with it a responsibility toward reflecting and addressing user needs through well-designed products and experiences.

Two, that design practice therefore needs to be invested in representing users accurately and appropriately. This requires a curiosity, engagement, and drive for understanding and developing new methods to create that understanding of users and user behaviors. Data is an integral part of that process.

Three, that a design perspective is needed to ensure that optimal user experiences are appropriately represented in business goals, measures, and metrics.

From these three orienting principles, we believe that designers are, or should be, fundamentally interested in being disciplined about data and its collection, analysis, and use.

We hope that the audience for this book is familiar with the concepts of iteration and continuous learning and that our readers want to bring that perspective to the design of data gathering and analysis, and to the practice of design itself.

Who Is This Book For?

This book is written for designers and product managers who are involved in launching digital products but who have little to no experience with leveraging data in their approach to product development. You might be part of a small startup with just a few people working on building your product, or you might be part of a team in a larger company and looking to apply data methodology in your group. You should have a basic understanding of design thinking, and likely have been working with partners in product and technology to build your products. You might work as an in-house designer or you may be working in an agency with clients. But fundamentally, you have an interest in understanding how blending data and design can help you solve problems for your product.

Based on our experience, the biggest adjustment for a designer or organization looking to leverage data is that they first need to get a solid understanding of data types and how they work together. Very rarely does doing so require a significant change to the fundamentals of how they design; however, it does require an open mind.

We note that our definition of design as a discipline is broad, and encompasses different roles in the industry. It is therefore worth reviewing who we think will gain most from this book:

If you formally trained in design...

We are writing for people whose formal or informal training is in design, but who are unfamiliar with large-scale online experiments with large groups of users. Perhaps you started your career with a strong art or creative background, as opposed to one based in engineering, or you have only worked at companies that haven’t had access to rich data.

If you are a user experience researcher...

We are also writing this book for user researchers who are focused on people’s everyday experience. Perhaps you started your career with a social science or anthropology background or are trained in qualitative user research, but you haven’t had the opportunity to think about how quantitative data at scale can give you a new tool to study human behavior. If you are empathetic and a humanist, with a desire to truly understand how people feel, but want to expand your methods and get a sense of what the fuss is all about with experimentation at scale—for good or bad—you will get something out of reading this book.

If you are a data scientist...

If you are familiar with analyzing logs, but have never done experimentation at scale, then this book can also be useful to you because it will present you with a different perspective on the user experiences about which you are collecting data. We hope this book will encourage you to get more proactively involved at the start of the conversation on user experience, and actively seek out collaborations with designers who you might not have otherwise thought to work with.

If you are a product manager, developer, or something else...

If you are interested in the blending of design and data, then this book could help you gain insight into how designers are beginning to approach their job by incorporating data. We know that great products are only built when you have product, technology, and design working together hand in hand, so we’re of course excited to have as many people from other disciplines engage with this book as well.

Scope

Our aim in this book is to help you understand the basics of designing with data, to help you recognize the value of incorporating data into your workflow, and to help you avoid some common pitfalls. There are many types of data that you might use as a designer. As we said, this book focuses specifically on large-scale experiments and A/B testing. We’re doing this specifically because we have found this to be where analytics and design have converged the least, but where we believe there is the most potential for fruitful gains and collaborations.

We aim to offer some perspectives on how to develop solid data practices in your organization. Throughout the book, we’ll share our experiences and the experiences of others, suggest ways that your team can change the way you work, and get the most benefit from your data. We want you to be able to argue your case with confidence about what should be measured, when and how to best illustrate, test, and further your design intention. Ultimately, we would love to make you feel excited and eager to embark on an approach within your company or organization that better leverages data.

About Us

We believe that designers, data scientists, developers, and business leaders need to work together to determine what data to collect, as well as when, why, and how to manage, curate, summarize, and communicate with and through data. We started to write this book because we wanted to encourage designers to influence and change the conversation around measures and metrics that purport to reflect the value of a product or service. We believe data and design need to be seen as two sides of the same coin.

Rochelle and Elizabeth have managed and driven user-centered design and evaluation processes in the internet industry. Caitlin is just beginning her career in this field, and we found that having a fresh perspective in the book was extremely helpful in making our point of view accessible to our readers.

All three of us care deeply about understanding the ways in which people interface with, interact with, and derive value from technology. We feel strongly that carefully gathered and analyzed data can and does help develop that understanding but that we need to broaden the conversation. We hope that by sharing our enthusiasm for integrating different forms of programmatically gathered data into the design process, we can persuade others who have a human-centered approach to design to be more centrally part of the data design process. We believe this will lead to better products and better business.

Although we focus on A/B testing in this book, we understand there are many ways to gather data. For example, surveys, interviews, field studies, diary studies, and lab studies are all excellent ways of collecting meaningful data about the users of a wide variety of products. In our daily work, we leverage data from all of these methodologies and others, from gathering feedback by talking with users in a usability session about how effective, usable, or delightful your product seems to them (or not!), to tracking data instrumented to measure exactly what actions users are doing in a product. We believe that if companies consider only one form of “data” (e.g., “click” or “clickstream” data—literally a record of what a user does at a screen), they will not get a full picture of a user’s experience. Therefore, we believe that the definition of data needs to be broad and constantly reviewed, as new methodologies emerge with unique capacities to help us capture a holistic view of our users.

We are excited to write this book because we are passionate about empowering designers to be able to formulate, inform, and evaluate their approach to product design with data.

A Word from Rochelle

I remember the first time I started to incorporate data in my design work. It was 2001, I was at a small startup, and someone had mentioned how Amazon was applying A/B testing as a technique to make decisions about user experience. Our startup was always great about reviewing the business metrics on a weekly basis and we were keen to get our hands on as much data as possible. However, with the introduction of A/B testing, we really got our heads around using data in an even more effective and sophisticated way than we had previously. During the next few years, our startup took a very DIY and self-taught approach to determining the ins and outs of being more data driven. We studied and learned as much as we could about what other data-driven companies like Netflix and Amazon were doing and tried to apply those learnings in practice.

Over the years, I’ve gotten more exposure to companies that excelled in gathering data and information from their users—my company was acquired by Intuit, which had a well-established and widely respected approach toward user research. Later, I joined Netflix, which is one of the most disciplined technology companies at using data to make decisions. Witnessing world-class environments that embrace the use of data has made me appreciate how incredible a tool it can be in helping to transform the way an entire product organization works. I’ve learned that there are many nuances to combining data and design, and that while there are many benefits, there are also many pitfalls as well. My goal is to help more designers appreciate and take advantage of the benefits while avoiding some of the pitfalls that I, and others, have fallen prey to in the past.

I’m writing this book because I hope that I can share my enthusiasm and passion for working in a data-centric environment with other designers and product managers and that this book can help you elevate the way in which you work with data.

A Word from Elizabeth

For me, a psychology undergraduate degree seeded my passion for design—the design of carefully crafted studies that help us understand people, their traits and motivations, as well as their experiences and behaviors.

Beyond focusing on individuals, though, this degree also seeded a passion for understanding the ways in which people interact with environments and social settings, and how those environments and social settings mold and shape their behaviors.

And thus began my career-long fascination with the human-centered design of situations and settings, including online situations and settings. I have worked in the area of human–computer interaction (HCI) in the decades since that undergraduate degree, working in academia and in a number of technology companies. HCI as a field of enquiry addresses how to build interactive systems and services that work for people. As a discipline, it draws on various academic areas, including applied psychology, computer science, anthropology, and ergonomics.

My desire to write this book stems from the belief that human-centered design practice and the “data sciences” work best when conducted together. As someone concerned with users and their experiences, I believe these two disciplines should be in a more productive dialogue than has been the case to date.

While data science has become a well-regarded tool for companies, the human-focused art of designing effective studies, posing the right questions (hypotheses), designing the right measures, and conducting exploratory analyses has not been equally emphasized. I have seen many designers whose intuitions are excellent and whose sensitivity to human psychology are evident in the products they create, but who, when it comes time for rollout, shy away from engaging with the design of studies that test what they have created. Over the years, talking to designers and working with them, it has become apparent that some designers would like to be part of the conversation around experimentation and data science, but they don’t feel empowered. If that sounds like you, this book is for you—this book is intended to help you push the conversation and deepen the connection between design, experimentation, and data analysis. This book is intended to help you participate in the creation of a design-focused data science, to help you demonstrate, with evidence, what good design is, and how good design can have a positive impact on people and on the world.

A Word from Caitlin

Like most college students nearing graduation, I struggled with the prospects of how to combine my seemingly disparate interests into a professional career. My undergraduate degree from MIT taught me the value of applying an experimental, quantitative approach to problem solving. It instilled in me a deep confidence in how numerical measurement can uncover surprising and generalizable truths about the world. However, I couldn’t shake the feeling that pursuing a strictly “scientific” or “technological” postgrad job would neglect my humanistic desire to understand the complexity of people, which in many ways felt irreducible to measurements.

Now, just under a year since beginning my role as a user researcher, I’ve come to realize that I drew a false dichotomy between the sciences and the arts. The discipline of design—and more broadly, the fields and industries that design is now applied to—provide a space for the arts and the sciences to come together for one unified goal: understanding people. What has been most exciting for me about this synergy is how data can be applied to create deep empathy for not only the humans behind our products, but also our cross-disciplinary collaborators. We need a humanistic view of products in order to ask the right questions, while leaning on the tools of scientific endeavor help answer those questions. Through the shared language of data, folks from a variety of backgrounds can have effective conversations about their users, to challenge our beliefs and assumptions in service of seeking truth. When applied in this way, data both retains and provides nuance to the complexity of human behavior, and gives us the words and tools to speak about that complexity. And, of equal importance, the practice of designing with data opens the channels for communication and collaboration across disciplines that have been previously siloed, but whose sum is greater than the individual parts.

I hope that this book helps infuse that empathy—for our users and coworkers whose backgrounds may lean toward the art or the science side of design—throughout the practice of building better experiences. And equally, I hope that it demonstrates to young people like myself that pursuing interdisciplinary fields like the intersection of design and data can bring together their interests and skillsets in unique and exciting ways. Doing so will inspire the creativity necessary to apply technology and product development to the human problems and needs of the 21st century.

How This Book Is Organized

We’ve structured the book into eight chapters and give a brief summary of each one here:

Chapter 1: Introducing a Data Mindset

In this chapter, we hope to motivate you to understand our perspective on how we see working with data—that it is both an exciting time to be working at the intersection of data and design, but also that working with data is truly a creative process. We talk about the kind of data that you have access to as a designer and how different roles in a company interact with data, in addition to covering some of our basic terminology.

Chapter 2: The ABCs of Using Data

In this chapter, we’ll give you the necessary foundation around data. We talk in more depth about data types and how you collect them. We introduce the experimental methodology that is necessary to make sure you are using your data to its fullest and define the basic components of A/B testing that you will need for the rest of the book.

Chapter 3: A Framework for Experimentation

Here we’ll take the concepts and put them into action. We’re going to tell you how we’ve crafted a framework for experimentation, which you can apply. This is our own take on how to do this, but many folks are doing something similar.

Chapter 4: The Definition Phase (How to Frame Your Experiments)

Starting in Chapter 4, we’ll get even more concrete about how to apply the framework we’ve outlined in Chapter 3 by starting with the importance of grounded question asking. We discuss what a hypothesis is and show how you build just one, and then scale to considering many divergent hypotheses. We also show you how to generate multiple statements and then whittle them down to the ones you should focus on.

Chapter 5: The Execution Phase (How to Put Your Experiments Into Action)

In this chapter, we focus on how to create the experiences that you will ultimately be testing. We again talk about how important it is to go broad and show that the design you craft will ultimately have an impact on the data you collect.

Chapter 6: The Analysis Phase (Getting Answers From Your Experiments)

Here we’ll take you through some of the considerations when you launch your A/B test. We’ll also discuss how to interpret your results, and make decisions about what to do next after running an A/B test.

Chapter 7: Creating the Right Environment for Data-Aware Design

In this chapter, we focus on building and driving a culture focused on learning at your company, and making data a core part of that learning. We also talk about some of the softer side of culture, including the kinds of people that work well in that environment, making data more accessible to everyone at your company, and the longer-term benefits of using data in the design process.

Chapter 8: Conclusion

In this short chapter, we offer a brief summary of the concepts we introduced in this book, and highlight the need for an ethical stance toward data collection and toward experimentation.

How to Read This Book

If you don’t know much about designing with data, we’d suggest that you start by reading the first six chapters in order. If you are familiar with working with data, then we suggest you read Chapter 3 to orient yourself to our framework and then hop around to the chapters that you are most interested in.

No matter how experienced you are with incorporating data into your design process, real-world examples from our friends in the industry are a great way to learn more about how the theories are being applied in reality. We’ve interviewed some of these folks in the field to hear their thoughts on designing with data. We will share a few of their vignettes to provide texture and complement our own perspective. Of course, such vignettes may not be directly applicable to your particular case—you may have a different user base, product, or constraints. But hopefully they will illustrate how to take the more abstract concepts we introduce throughout this book into practice.

In addition to these real-world examples, we also introduce an illustrative example in some of the early chapters of this book to help explain some of the different concepts we cover. Our illustrative example focuses on running a summer camp. These places in the book will be noted with a small camping icon.

At the end of every chapter, we also include a set of questions to provoke you to think more deeply about what you have read; these come under the section header “Questions to Ask Yourself.” These questions are included as a way to spark some conversation either with yourself or within your company, and to help you take the concepts introduced in the chapter and apply them to your own work.

We hope this book will give you what you need to start out on this journey and to build a shared understanding with your peers of your product, based on objective feedback direct from your users.

Introducing our “Running a Camp” Metaphor

You may notice throughout this book that we like metaphors. Starting in Chapter 2 and continuing throughout the book, we’ll be using an illustrative metaphor to help explain some of the concepts and situations that we are describing. We’ve often found that having a strong metaphor that is abstracted a bit can help to crystallize some of the concepts we are covering in a way that real-world examples from companies can’t. It’s sometimes easier to see how an illustrative example can apply to your own world rather than trying to take something from another company and then do the translation from an example that might feel very specific to their situation and needs.

So periodically, we are going to ask you to pretend that you are the owner of a summer camp. Every year, you welcome back ~200 kids to your summer camp, where they get to go hiking, play outdoors, and enjoy eating meals together. Because your camp is so large and diverse, feedback you hear from just a few campers may not accurately capture the overall experience of your campers. You’ve been running this camp for a number of years, and have some dedicated families that return every year, but as you are running a business, you also want to make sure you are continuing to attract new families as well. Because you run your camp on a recurring basis, we thought it made a nice tie-in to why you might care about experimenting with new ways to improve your summer camp experience and therefore your business.

In Chapter 2 through Chapter 5 you’ll see us refer you back to this summer camp metaphor as we ask you to think about:

• How to define and measure your goals

• How to think about your users (both existing users and new ones)

• How to design experiences you can test and learn from as you try to improve your business

As a quick illustration, you can see how this metaphor might apply to running experiments. If you’re trying to improve your camp experience, you’ll want to get an understanding of what’s working and what’s not. You’ll want to try some new things out on a smaller scale before you commit to investing in big changes across the camp and as you try to expand your business you’ll also want to make sure that any changes you consider will have the results and impact that you’re looking for. We’ll show you how you can use this metaphor to understand statistical terms like cohorts and segmentation (Chapter 2), and how it can be used to illustrate considerations when you are defining your goals (Chapter 4), designing your experiences (Chapter 5), and then analyzing your results (Chapter 6). Yes, it might seem a bit goofy, but if you bear with us, we think it will help!

And now let’s get going... Designing with Data!

O’Reilly Safari

Safari (formerly Safari Books Online) is a membership-based training and reference platform for enterprise, government, educators, and individuals.

Members have access to thousands of books, training videos, Learning Paths, interactive tutorials, and curated playlists from over 250 publishers, including O’Reilly Media, Harvard Business Review, Prentice Hall Professional, Addison-Wesley Professional, Microsoft Press, Sams, Que, Peachpit Press, Adobe, Focal Press, Cisco Press, John Wiley & Sons, Syngress, Morgan Kaufmann, IBM Redbooks, Packt, Adobe Press, FT Press, Apress, Manning, New Riders, McGraw-Hill, Jones & Bartlett, and Course Technology, among others.

For more information, please visit http://oreilly.com/safari.

How to Contact Us

Please address comments and questions concerning this book to the publisher:

• O’Reilly Media, Inc.

• 1005 Gravenstein Highway North

• Sebastopol, CA 95472

• 800-998-9938 (in the United States or Canada)

• 707-829-0515 (international or local)

• 707-829-0104 (fax)

We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at http://bit.ly/designing-with-data.

To comment or ask technical questions about this book, send email to [e-mail].

For more information about our books, courses, conferences, and news, see our website at http://www.oreilly.com.

Find us on Facebook: http://facebook.com/oreilly

Follow us on Twitter: http://twitter.com/oreillymedia

Watch us on YouTube: http://www.youtube.com/oreillymedia

Acknowledgments

Writing this book has been a long project that has progressed in fits and starts, and we wouldn’t have made it across the finish line without the help and support of so many people to whom we owe so much. We have been blessed with incredible colleagues from whom we’ve learned throughout our careers and we’ve been fortunate enough to work at companies that have set the gold standard for doing the type of work that we do. Thank you to our friends and peers for your generosity and insights.

We’d also like to thank the good people at O’Reilly. Mary Treseler initiated this project with Rochelle and never gave up on seeing the book come to life. Angela Rufino patiently stuck with us as our editor, prodding us along the way and providing encouragement and support when we needed it most. This thanks extends to the rest of the O’Reilly team as well: Colleen Lobner, Jasmine Kwityn, and José Marzan, Jr.

We are grateful to the many people who were generous with the time they gave to us in interviews; their reflections and perspectives have been invaluable and the conversations they shared with us have given a three-dimensionality to this book. Thanks go to Arianna McClain, John Ciancutti, Katie Dill, Eric Colson, Dan McKinley, Patty McCord, Jon Wiley, Josh Brewer, Chris Maliwat, David Ayman Shamma, David Draper, Amy Bruckman, Casey Fiesler, and Jeff Hancock. We would also like to thank the folks behind the case studies that served as anchors to Chapter 4 through Chapter 6 in our book: Marcus Persson, Julian Kirby, Natasa Soltic, Chris Smith, Alvin Lee, Dantley Davis, Steven Dreyer, Neil Hunt, Matt Marenghi, and Todd Yellin. A special thanks to Ben Dressler and Annina Koskinen for providing a sounding board as we worked through various concepts and themes in this book. We are indebted to our peer reviewers and others whose comments helped us immeasurably in terms of shaping the book: Tim Lynch, Kevin Ho, Khoi Vihn, Martin Charlier, and Sean Power.

We owe special thanks to Colin McFarland, who went above and beyond in terms of his generosity, support, and guidance. His insight was invaluable throughout the process. He read rough drafts, allowed us to try out different concepts and metaphors with him, sat through long Google hangouts to share his comments with us, and poked holes in our manuscript when our language got sloppy.

Rochelle

I am grateful to my husband Warren and our two sons, Genta and Tatsuya. They patiently allowed me to spend countless weekend hours at my computer instead of hanging out with them, and I now plan to pay those hours back in full. The constant reminders from my sons that “we don’t have quitters in this family” was also helpful on the many, many occasions when I would suffer from writer’s block. I’d also like to thank my mom, who worked tirelessly to make so much possible for me and whose support and belief in me has never faltered. Finally, a huge hug to Elizabeth and Caitlin, without whom I am certain this project would not have been completed.

Elizabeth

First and foremost, I would like to thank my coauthors for sticking with this project. It’s been quite a journey. I look forward to conversations and laughter in the future, when no conversation involves discussion of a possible chapter edit. I’d also like to thank my friends and family, whose company I have missed while focusing on this book over the course of many weekends. Finally, I want to thank my colleagues, who have patiently indulged me time and again in random conversations about the art, science, and practice of experimentation in these internet days. Thank you all.

Caitlin

I’d like to thank my boyfriend Harvey for the endless support and endless cups of home-brewed coffee during many long weekends of writing and editing. I owe you many weekend adventures now that we’ve wrapped up to make up for all that time! I’m also grateful to my family—Mom, Dad, Darrien, and Auntie Amelia—for being my biggest cheerleaders and advocates, and for always believing I could do more than I thought possible. Thanks to all of my lovely friends, especially Cecile, Michelle, and Zoë, who lent me their best listening ears and accommodated my crazy schedule throughout this whole process. And finally, a huge thanks to Rochelle and Elizabeth. I went into this process proud to be your coauthor, but I’m even more grateful to finish up alongside two exceptional mentors and friends.

ROCHELLE KING

NEW YORK, NY, USA

ELIZABETH F. CHURCHILL

SAN FRANCISCO, CA, USA

CAITLIN TAN

BROOKLYN, NY, USA