Exploring Expect

Preface

This book is a tutorial for Expect, a software suite for automating interactive tools.

Expect has turned out to be very popular. This is good and bad. It’s good because people are able to do things more easily than before and in some cases do things that they would never have even tried. Expect is not simply another language. It is a completely new type of tool that addresses problems that were not even recognized as problems in the past.

The bad news is that since writing Expect, I’ve been plagued by people asking me questions about it. While it was not originally intended to be a rich or complex piece of software, it has admittedly become more sophisticated as various features have been added. But more importantly, the nature of automating interactive programs involves dealing with issues that are quite unlike the issues involved in traditional programming.

While Expect comes with a “man page” (which is quite a misnomer at 25 pages), I continue to get requests for information that does not properly belong there. Often the requests are for examples. Sometimes they are simply for advice. This book is an attempt to write down all of these things and to describe everything in the man page in a tutorial fashion.

This book draws upon the thousands of Expect applications that people have described to me, the common and not-so-common problems that people have discussed with me—and explanations of the limitations—what you can’t do with Expect.

I pull no punches. Expect is not meant to do everything for everyone, and I am quite frank about discussing its limitations. While some may represent my own limitations, others mirror my beliefs about how UNIX should work rather than what can be accomplished. Indeed, I have resisted requests for enhancements that do not add anything specifically useful to the original intent of Expect or that can already be done more easily by another program. Expect is not yet-another kitchen-sink language.

I am convinced^[1] that Expect is very easy to use for the majority of applications that users put it to. And I am convinced that many people can learn to do useful Expect scripting in an hour or two.

Nevertheless, I recognize that the language is substantial. Using it is one thing. Mastering it is quite another. Just reading through the 500+ pages in this book may be an onerous task for some. However, as I said above, I believe that much of the reason for the length of this book is due to the unique nature of automating interactive programs. No matter how you accomplish it—whether using Expect, a commercial product, or a home-grown set of kludges—automating interactive programs is a task that is full of surprises. And while the examples in this book are specific to Expect, the knowledge you gain from them can be taken and applied to other interaction automation tools.

Indeed, Expect represents only the tip of the iceberg in the field of interactive automation. Already, GUI automators are on the market. Eventually, hypermedia automators will make their debut, combining simulations of human voice, images, and all sorts of other sensory data.

Expect—Why another tool?

I initially did not view Expect as something that would last very long. It struck me as solving a very simple problem but not in the best way possible. Indeed, I originally wrote Expect as an experiment to demonstrate the need for a general way of handling interaction automation. I expected that, having shown the utility of it, the popular shells of the day would all soon incorporate these functions, allowing Expect-like things to be accomplished from the shells without requiring the use of another tool.

But to my surprise that has not been the case. Some shells (e.g., Korn shell, Z shell) provide co-processes but offer no access to pseudo-terminals and no in-line stream pattern matching. Most shells don’t even provide co-processes. Thus, Expect remains very important to shell programmers.

Expect is also useful with environments that have limited or baffling Expect-like functions. For example, Emacs, a popular editor, has actually had the ability to do Expect-like processing for years. However, Emacs is a fairly unusual programming environment and few people do real Emacs programming. Perl is another unusual programming environment, again, with its own Expect-like functionality; however, the implementation is difficult to use and many Perl programmers find it easier to call Expect scripts for these tasks. Perl is also a large language, which makes its use for Expect-like programming all the more formidable for the casual user.^[2]

By comparison, Expect is simple. It really only does one thing. But it does it very well. Everything in Expect is optimized to help you automate interactive programs.

That doesn’t mean it can’t do other things. These other things simply aren’t the focus of Expect. You can use Expect to work with non-interactive applications. Expect rests on top of Tcl which provides a very pleasant environment in which to work. I have implemented and used a number of complex software packages that use Tcl as a scripting language, and I look forward to using any other application that does similarly.

---

^[1] Meaning that I have no proof.

^[2] Although Exploring Expect is based on Tcl, programmers attempting to automate interactive programs using Emacs, Perl, Python, Scheme, C, or any other tool will find this book helpful because many of the concepts underlying Expect-like programming are common to all of those tools.