Twisted Network Programming Essentials, 2nd Edition

Foreword to the First Edition

“My name is Ozymandius, king of kings:
Look on my words, ye Mighty, and despair!”
Nothing beside remains. Round the decay
Of that colossal wreck, boundless and bare
The lone and level sands stretch far away.

—Percy Bysshe Shelly, “Ozymandius”

As the Twisted project’s originator and nominal leader—and as someone who is not being paid for writing this—I can very honestly say that this is a fine book, and it has made me proud of what I’ve started. You now hold in your hands a wondrous key that contains the knowledge to unlock a very powerful software system—a software system borne of a consistent, methodical vision; a vision half a decade in realization and hundreds of man-years in implementation; a vision for a video game that has yet to be written, called “Divunal.”

I have been lauded many times for my role in Twisted’s creation, and in this foreword I will attempt to disabuse you of the notion that any of it was on purpose. Not only was it an accident, but neither I, nor anyone else, has made one iota of progress towards my original goal of writing a game.

When I was eight years old, I decided I wanted to be a writer. I was going to write video games just like my favorite ones, the text-based games from Infocom. They were like books, but better. I knew how to write already—at a fourth-grade level, or so I’m told—and all I needed to figure out was the part where the computer wrote back. Lucky for you nobody thought to tell me how hard that step between the input and the output was, or Twisted would be a series of detective novels instead of a Python program.

Tolkien said it best: “The tale grew in the telling,” and I’ll say it worse: the code grew in the hacking. Twisted began over a decade after my aforementioned first plunge into the netherworld of software, as a solitary attempt to create a networking subsystem for a small online fantasy world. Since then, it has become an ongoing community quest to unify all manner of asynchronous communications. This book will take you on an adventure through Twisted for the Web, Twisted for email, Twisted for chat, and of course, Twisted for whatever new kind of networked application you want to dream up—maybe even an online video game.

Much as the tale of Twisted has grown and changed, its origins still have a profound effect on its nature, and on its future. Having origins in an eclectic^[1] problem domain has attracted an eclectic^[2] audience. The community in the online support forum engages in discussions that are “often funny.” To put it more directly: we’re weird.

“Weird” is a badge I have long worn with pride, dear reader, so please take it as a compliment that I bestow it upon you. You’re not simply non-average, you’re better than average. Almost by definition, Twisted hackers are the ones for whom “good enough” isn’t good enough. You are the web programmers who can’t use their operating system’s stock HTTP daemon because you need more power and more control over how it’s run; the chat developers who aren’t content with chatting on a perfectly working network just because it doesn’t support some cool new features you want; the (dare I say it?) gamers who aren’t content with the market’s offerings of online games. You want to create something newer, different, better. To build higher than those who have come before, because you are building not merely upon the shoulders of giants, but upon the apex of an acrobatic balancing act of giants, or more literally an interlocking network of frameworks and libraries for different tasks, rather than just one at a time.

Twisted will let you do that, by letting you leverage code written by far more and far better programmers than I. Twisted provides a common method for that code to cooperate, which means you can use all of that code without performing a complex integration pass. In this spirit, I’d like to invite you to release your Twisted-based projects, or the infrastructure components of them, as open source software, so that we might together build a Twisted commons upon which many more fantastic applications will be built.

Don’t mistake this friendly vision for altruism, however. I didn’t have anything to do with the start of the Free Software or Open Source movements, respectively, but they came along at a convenient time for me. This feeling of share-and-share-alike has been a feature of the Twisted community since day one, but not because I care about sharing.^[3] It is because—I may have mentioned this—I want to write a video game one day. A game that effortlessly connects to the Web and to your email, that politely requests that you play when you have time, and that reminds you to get back to work when you do not.

You see, the majority of Twisted’s core developers, including myself, suffer from Attention Deficit Disorder. This malady is the grease that makes the magic wheels of integration turn. While most developers—sane developers—would be content to write a perfectly good web server that could work only as a web server and leave it at that, we are always afraid we’ll suddenly lose interest and need a chat application instead—or maybe it should be a mail server? Hey, there’s a squirrel! I don’t like this song.

What was I saying? Oh yes. The essence of Twisted is apparently paradoxical. Created on a whim by crazed eccentrics, designed to be a toy, and yet powerful enough to drive massive email systems, high-traffic web sites, transaction-processing systems, and inventory management applications.

However, the paradox is an illusion. People produce the best work when they are working and having fun at the same time. It takes a sense of humor to call yourself a crazed eccentric (whether it’s true or not). You have to have a sense of fun to try and build a toy. In enjoying ourselves, we have brought to life a system that many of us have tried and been unable to create in more serious surroundings.

So, when I look out upon the “lone and level sands” of Divunal, a game whose incarnation today is little more than its name, I am not concerned. I am having a good time with Twisted. With this book in hand, I have no doubt that you will, too.

—Matthew “the Glyph” Lefkowitz
CTO at Divmod, Inc.
(not a game company)
(yet)
August 2005

---

^[1] And difficult! Making an online game work properly is hard.

^[2] And intelligent! People who solve unusual problems are always learning.

^[3] Caution for the humorless: this is a joke. I am not actually an enemy of freedom. Still, there is some truth to this.