Learning the Korn Shell

Preface

The long, tortuous history of the UNIX operating system has resulted in systems with all kinds of permutations and combinations of features. This means that whenever you walk up to an unfamiliar UNIX system, you need to find out certain things about it in order to use it properly. And even on a given system, you may have a number of choices you can make about what features you want to use.

The most important such decision—if you get to make it—is what shell to use. “Shell” is UNIX jargon for the program that allows you to communicate with the computer by entering commands and getting responses. The shell is completely separate from the UNIX operating system per se; it’s just a program that runs on UNIX. With other systems such as MS-DOS, the Macintosh, and VM/CMS, the command interpreter or user interface is an integral part of the operating system.

Nowadays there are dozens of different shells floating around, ranging from the original standard, the Bourne shell, to menu-based and graphical interfaces. The most important shells have been the Bourne shell, the C shell, and now the Korn shell—the subject of this book.

Korn Shell Versions

Specifically, this book describes the 1988 version of the Korn shell, which is distributed with all UNIX systems based on System V Release 4. There are various other versions, variations, and implementations on other operating systems; these are described in Appendix A.

To find out which version you have, type the command set -o emacs, then press CTRL-V. You should see a date followed by a version letter (the letter is unimportant). If you do, you have one of the official versions, whether it be the 1988 version or an older one. But if you don’t, then you have a non-standard version such as pdksh, the public domain Korn shell discussed in Appendix A.

Summary of Korn Shell Features

The Korn shell is the most advanced of the shells that are “officially” distributed with UNIX systems. It’s a backward-compatible evolutionary successor to the Bourne shell that includes most of the C shell’s major advantages as well as a few new features of its own.

Features appropriated from the C shell include:

• Job control, including the fg and bg commands and the ability to stop jobs with CTRL-Z.

• Aliases, which allow you to define shorthand names for commands or command lines.

• Functions (included in some C shell versions), which increase programmability and allow you to store your own shell code in memory instead of files.

• Command history, which lets you recall previously entered commands.

The Korn shell’s major new features include:

• Command-line editing, allowing you to use vi or emacs-style editing commands on your command lines.

• Integrated programming features: the functionality of several external UNIX commands, including test, expr, getopt, and echo, has been integrated into the shell itself, enabling common programming tasks to be done more cleanly and without creating extra processes.

• Control structures, especially the select construct, which enables easy menu generation.

• Debugging primitives that make it possible to write tools that help programmers debug their shell code.

• Regular expressions, well known to users of UNIX utilities like grep and awk, have been added to the standard set of filename wildcards and to the shell variable facility.

• Advanced I/O features, including the ability to do two-way communication with concurrent processes (coroutines).

• New options and variables that give you more ways to customize your environment.

• Increased speed of shell code execution.

• Security features that help protect against “Trojan horses” and other types of break-in schemes.

Intended Audience

This book is designed to appeal most closely to casual UNIX users who are just above the “raw beginner” level. You should be familiar with the process of logging in, entering commands, and doing simple things with files. Although Chapter 1, reviews concepts such as the tree-like file and directory scheme, you may find that it moves too quickly if you’re a complete neophyte. In that case, we recommend the O’Reilly & Associates Nutshell Handbook, Learning the UNIX Operating System, by Grace Todino and John Strang.

If you’re an experienced user, you may wish to skip Chapter 1 altogether. But if your experience is with the C shell, you may find that Chapter 1 reveals a few subtle differences between the Korn and C shells.

No matter what your level of experience is, you will undoubtedly learn many things in this book that make you a more productive Korn shell user—from major features down to details at the “nook-and-cranny” level that you weren’t aware of.

If you are interested in shell programming (writing shell scripts and functions that automate everyday tasks or serve as system utilities), you should find this book useful too. However, we have deliberately avoided drawing a strong distinction between interactive shell use (entering commands during a login session) and shell programming. We see shell programming as a natural, inevitable outgrowth of increasing experience as a user.

Accordingly, each chapter depends on those previous to it, and although the first three chapters are oriented toward interactive use only, subsequent chapters describe interactive user-oriented features in addition to programming concepts.

In fact, if this book has an overriding message, it is: “The Korn shell is an incredibly powerful and grossly undervalued UNIX programming environment. You—yes, you—can write useful shell programs, even if you just learned how to log on last week and have never programmed before.”

Toward that end, we have decided not to spend much time on features of interest exclusively to low-level systems programmers. Concepts like file descriptors, errno error numbers, special file types, etc., can only confuse the casual user, and anyway, we figure that those of you who understand such things are smart enough to extrapolate the necessary information from our cursory discussions.

Code Examples

This book is full of examples of shell commands and programs that are designed to be useful in your everyday life as a user, not just to illustrate the feature being explained. In Chapter 4 and onwards, we include various programming problems, which we call tasks, that illustrate particular shell programming concepts. Some tasks have solutions that are refined in subsequent chapters. The later chapters also include programming exercises, many of which build on the tasks in the chapter.

You should feel free to use any code you see in this book and to pass it along to friends and colleagues. We especially encourage you to modify and enhance it yourself.

If you want to try examples but you don’t use the Korn shell as your login shell, you must put the following line at the top of each shell script:

#!/bin/ksh

If your Korn shell isn’t installed as the file /bin/ksh, substitute its pathname in the above.

Chapter Summary

If you want to investigate specific topics rather than read the entire book through, here is a chapter-by-chapter summary:

Chapter 1

introduces the Korn shell and tells you how to install it as your login shell. Then it gives an introduction to the basics of interactive shell use, including overviews of the UNIX file and directory scheme, standard I/O, and background jobs.

Chapter 2,

discusses the shell’s command history mechanism, including the emacs- and vi-editing modes and the fc history command.

Chapter 3,

covers ways to customize your shell environment without programming, by using the .profile and environment files. Aliases, options, and shell variables are the customization techniques discussed.

Chapter 4

is an introduction to shell programming. It explains the basics of shell scripts and functions, and discusses several important “nuts-and-bolts” programming features: string manipulation operators, regular expressions, command-line arguments (positional parameters), and command substitution.

Chapter 5,

continues the discussion of shell programming by describing command exit status, conditional expressions, and the shell’s flow-control structures: if, for, case, select, while, and until.

Chapter 6,

goes into depth about positional parameters and command-line option processing, then discusses special types and properties of variables, such as integer arithmetic and arrays, and the typeset command.

Chapter 7,

gives a detailed description of Korn shell I/O, filling in the information omitted in Chapter 1. All of the shell’s I/O redirectors are covered, as are the line-at-a-time I/O commands read and print. Then the chapter discusses the shell’s command-line processing mechanism and the eval command.

Chapter 8,

covers process-related issues in detail. It starts with a discussion of job control, then gets into various low-level information about processes, including process IDs, signals, and traps. The chapter then moves out to a higher level of abstraction to discuss coroutines, two-way pipes, and subshells.

Chapter 9,

discusses various debugging techniques, starting with simple ones like trace and verbose modes and “fake signal” traps. Then it presents kshdb, a Korn shell debugging tool that you can use to debug your own code.

Chapter 10,

gives information for system administrators, including techniques for implementing system-wide shell customization and features related to system security.

Appendix A

compares the 1988 UNIX Korn shell to several similar shells, including the standard Bourne shell, the IEEE 1003.2 POSIX shell standard, the Windowing Korn shell (wksh), public domain Korn shell (pdksh), the Free Software Foundation’s bash, and the MKS Toolkit shell for MS-DOS and OS/2.

Appendix B,

contains lists of shell invocation options, built-in commands, built-in variables, conditional test operators, options, typeset command options, and emacs and vi editing mode commands.

Appendix C,

lists the ways that you can obtain the major scripts in this book for free, using anonymous FTP or electronic mail.

Conventions Used in This Handbook

We leave it as understood that, when you enter a shell command, you press RETURN at the end. RETURN is labeled ENTER on some keyboards.

Characters called CTRL-X, where X is any letter, are entered by holding down the CTRL (or CTL, or CONTROL) key and pressing that letter. Although we give the letter in uppercase, you can press the letter without the SHIFT key.

Other special characters are LINEFEED (which is the same as CTRL-J), BACKSPACE (same as CTRL-H), ESC, TAB, and DEL (sometimes labeled DELETE or RUBOUT).

This book uses the following font conventions:

Italic	is used for UNIX filenames, commands not built into the shell, (which are files anyway), and shell functions. Italic is also used for dummy parameters that should be replaced with an actual value, to distinguish the vi and emacs programs from their Korn-shell modes, and to highlight special terms the first time they are defined.
Bold	is used for Korn shell built-in commands, aliases, variables, and options, as well as command lines when they are within regular text. Bold is used for all elements typed in by the user.
Constant Width	is used in examples to show the contents of files or the output from commands.
Constant Bold	is used in examples to show interaction between the user and the shell; any text the user types in is shown in Constant Bold. For example: $ pwd /users/billr/ora/kb $
Constant Italic	is used in displayed command lines for dummy parameters that should be replaced with an actual value.
Reverse Video	is used in Chapter 2 to show the position of the cursor on the command line being edited. For example: grep -l Bob < ~pete/wk/names

Standard UNIX utility commands are sometimes mentioned with a number in parentheses (usually 1) following the command’s name. The number refers to the section of the UNIX User’s Manual in which you’ll find reference documentation (a.k.a. “man page”) on the utility in question. For example, grep(1) means you will find the man page for grep in Section 1.

Acknowledgments

Many people contributed to this book in many ways. I’d like to thank the following people for technical advice and/or assistance: for system administration help, John van Vlaanderen and Alexis Rosen. For information on alternative shells, John (again), Sean Wilson (of MKS), Ed Ravin, Mel Rappaport, and Chet Ramey. For identifying the need for a shell debugger, expertise in SunOS and system security, and, indeed, a significant portion of my career, Hal Stern. For debugger suggestions, Tan Bronson. For humanitarian aid, Jessica Lustig. And much thanks to David Korn for all kinds of great “horse’s mouth” information—and, of course, for the Korn shell itself.

Thanks to our technical reviewers: Jim Baumbach, Jim Falk, David Korn, Ed Miner, Eric Pearce, and Ed Ravin. I especially appreciate the cooperation of Ed and Ed (in that order) during my “Whaddya mean, it doesn’t work?!?” phase.

Several people at O’Reilly & Associates contributed to this effort: Gigi Estabrook and Clairemarie Fisher O’Leary proofread multiple drafts of the manuscript, Kismet McDonough and Donna Woonteiler copyedited the manuscript, Len Muellner implemented the book design macro package, Jennifer Niederst designed the cover and the format of the book, and Chris Reilley created the figures. Finally, an ocean of gratitude to Mike Loukides—editor, motivator, facilitator, constructive nit-picker, and constant voice of reason. He and the other folks at O’Reilly & Associates are some of the most innovative, interesting, and motivated people I’ve ever had the privilege to work with.

We’d Like to Hear From You

We have tested and verified all of the information in this book to the best of our ability, but you may find that features have changed (or even that we have made mistakes!). Please let us know about any errors you find, as well as your suggestions for future editions, by writing:

O’Reilly & Associates, Inc.

101 Morris Street

Sebastopol, CA 95472

1-800-998-9938 (in the US or Canada)

1-707-829-0515 (international/local)

1-707-829-0104 (FAX)

You can also send us messages electronically. To be put on the mailing list or request a catalog, send email to:

[e-mail]

To ask technical questions or comment on the book, send email to:

[e-mail]