Designing Gestural Interfaces

Preface

For the past several years, I've been learning to play the cello. As any adult who has ever tried to learn a musical instrument—especially a fretless stringed instrument—can tell you, it's painfully hard. You feel like you don't really know your body at all: your muscles get sore in odd places, you develop strange calluses on your fingers, and you perform stretches with your hands you never thought possible. It requires almost all of your concentration simply to coax the instrument into making a pleasing sound, which is very different from actually making music.

But, ah, when you do begin to make music, it is a thing of joy. When you are able to get a sweet sound with emotional resonance from an inanimate object using only your body, it is an amazing feeling. Gestures create meaning.

That's what this book is about.

Technological, social, and market forces have converged to create a fertile new ground for designers and engineers to plow. The price of processing speed has dropped and sensors are readily available. Touchscreens on our mobile devices, ATMs, and airline check-in kiosks have taught us to expect to be able to manipulate things on-screen with our hands. Games have shown us we can make gestures in space to control objects on-screen. Public restrooms are, believe it or not, test laboratories for interactive gestures: placing your hands under a faucet to turn it on, waving your hands to get a paper towel, stepping into a room to turn on the lights.

All of these things have ushered in a new era of interaction design, one where gestures on a surface and in the air replace (or at least supplement) keyboards, mice, and styli. This new era, however, means those who design and develop more "traditional" systems need to grow their skills, adding in knowledge about kinesiology, sensors, ergonomics, physical computing, touchscreen technology, and new interface patterns.

That's what this book is about.

I wrote this book because, on my third gestural interface project, I was frustrated that I could find only bits and pieces of information about a subject—this subject—that was obviously important and growing rapidly. Why wasn't there a book about these things? I groused. How come I have no idea how big a touchscreen button needs to be? What is a capacitive touchscreen? What kinds of gestures can I use? And thus the idea for the book you are reading was born.

Aside from the constraints of the human body, many of the other parts of this book were changing even as I wrote about them, and new products that utilized interactive gestures seemingly came out every week, if not several times a week, over the course of the nine months I wrote this book (November 2007–July 2008). The newness of the subject was glaringly apparent, but I have tried, when and where possible, to emphasize what is unlikely to change and techniques that are, if not proven, at least being used by practicing designers and developers in the field today.

WHO SHOULD READ THIS BOOK

This book is mostly for interaction and industrial designers who have found themselves in the same position I found myself in several years ago: being asked to design gestural interfaces (especially touchscreens) and, coming from the world of web and desktop software or physical products, having no central place to go to for information to get started. I hope this book is a starting place and a reference point.

And even though there isn't a line of code in this book, I also wrote it with developers in mind, knowing full well that they are often the people who have to take what designers dream up and make it real. I hope everyone involved in creating new products that make use of sensors and interactive surfaces will find something of use in these pages.