The user or programmer new to Unix[1] is suddenly faced with a bewildering variety of programs, each of which often has multiple options. Questions such as "What purpose do they serve?" and "How do I use them?" spring to mind.
This book's job is to answer those questions. It teaches you how to combine the Unix tools, together with the standard shell, to get your job done. This is the art of shell scripting. Shell scripting requires not just a knowledge of the shell language, but also a knowledge of the individual Unix programs: why each one is there, and how to use them by themselves and in combination with the other programs.
Why should you learn shell scripting? Because often, medium-size to large problems can be decomposed into smaller pieces, each of which is amenable to being solved with one of the Unix tools. A shell script, when done well, can often solve a problem in a mere fraction of the time it would take to solve the same problem using a conventional programming language such as C or C++. It is also possible to make shell scripts portableâi.e., usable across a range of Unix and POSIX-compliant systems, with little or no modification.
When talking about Unix programs, we use the term tools deliberately. The Unix toolbox approach to problem solving has long been known as the "Software Tools" philosophy.[2]
A long-standing analogy summarizes this approach to problem solving. A Swiss Army knife is a useful thing to carry around in one's pocket. It has several blades, a screwdriver, a can opener, a toothpick, and so on. Larger models include more tools, such as a corkscrew or magnifying glass. However, there's only so much you can do with a Swiss Army knife. While it might be great for whittling or simple carving, you wouldn't use it, for example, to build a dog house or bird feeder. Instead, you would move on to using specialized tools, such as a hammer, saw, clamp, or planer. So too, when solving programming problems, it's better to use specialized software tools.
This book is intended for computer users and software developers who find themselves in a Unix environment, with a need to write shell scripts. For example, you may be a computer science student, with your first account on your school's Unix system, and you want to learn about the things you can do under Unix that your Windows PC just can't handle. (In such a case, it's likely you'll write multiple scripts to customize your environment.) Or, you may be a new system administrator, with the need to write specialized programs for your company or school. (Log management and billing and accounting come to mind.) You may even be an experienced Mac OS developer moving into the brave new world of Mac OS X, where installation programs are written as shell scripts. Whoever you are, if you want to learn about shell scripting, this book is for you. In this book, you will learn:
- Software tool design concepts and principles
A number of principles guide the design and implementation of good software tools. We'll explain those principles to you and show them to you in use throughout the book.
- What the Unix tools are
A core set of Unix tools are used over and over again when shell scripting. We cover the basics of the shell and regular expressions, and present each core tool within the context of a particular kind of problem. Besides covering what the tools do, for each tool we show you why it exists and why it has particular options.
Learning Unix is an introduction to Unix systems, serving as a primer to bring someone with no Unix experience up to speed as a basic user. By contrast, Unix in a Nutshell covers the broad swath of Unix utilities, with little or no guidance as to when and how to use a particular tool. Our goal is to bridge the gap between these two books: we teach you how to exploit the facilities your Unix system offers you to get your job done quickly, effectively, and (we hope) elegantly.
- How to combine the tools to get your job done
In shell scripting, it really is true that "the whole is greater than the sum of its parts." By using the shell as "glue" to combine individual tools, you can accomplish some amazing things, with little effort.
- About popular extensions to standard tools
If you are using a GNU/Linux or BSD-derived system, it is quite likely that your tools have additional, useful features and/or options. We cover those as well.
- About indispensable nonstandard tools
Some programs are not "standard" on most traditional Unix systems, but are nevertheless too useful to do without. Where appropriate, these are covered as well, including information about where to get them.
For longtime Unix developers and administrators, the software tools philosophy is nothing new. However, the books that popularized it, while still being worthwhile reading, are all on the order of 20 years old, or older! Unix systems have changed since these books were written, in a variety of ways. Thus, we felt it was time for an updated presentation of these ideas, using modern versions of the tools and current systems for our examples. Here are the highlights of our approach:
Our presentation is POSIX-based. "POSIX" is the short name for a series of formal standards describing a portable operating system environment, at the programmatic level (C, C++, Ada, Fortran) and at the level of the shell and utilities. The POSIX standards have been largely successful at giving developers a fighting chance at making both their programs and their shell scripts portable across a range of systems from different vendors. We present the shell language, and each tool and its most useful options, as described in the most recent POSIX standard.
The official name for the standard is IEEE Std. 1003.1-2001.[3] This standard includes several optional parts, the most important of which are the X/Open System Interface (XSI) specifications. These features document a fuller range of historical Unix system behaviors. Where it's important, we'll note changes between the current standard and the earlier 1992 standard, and also mention XSI-related features. A good starting place for Unix-related standards is http://www.unix.org/.[4]
The home page for the Single UNIX Specification is http://www.unix.org/version3/. Online access to the current standard is available, but requires registration at http://www.unix.org/version3/online.html.
Occasionally, the standard leaves a particular behavior as "unspecified." This is done on purpose, to allow vendors to support historical behavior as extensions, i.e., additional features above and beyond those documented within the standard itself.
Besides just telling you how to run a particular program, we place an emphasis on why the program exists and on what problem it solves. Knowing why a program was written helps you better understand when and how to use it.
Many Unix programs have a bewildering array of options. Usually, some of these options are more useful for day-to-day problem solving than others are. For each program, we tell you which options are the most useful. In fact, we typically do not cover all the options that individual programs have, leaving that task to the program's manual page, or to other reference books, such as Unix in a Nutshell (O'Reilly) and Linux in a Nutshell (O'Reilly).
By the time you've finished this book, you should not only understand the Unix toolset, but also have internalized the Unix mindset and the Software Tools philosophy.
[1] Throughout this book, we use the term Unix to mean not only commercial variants of the original Unix system, such as Solaris, Mac OS X, and HP-UX, but also the freely available workalike systems, such as GNU/Linux and the various BSD systems: BSD/OS, NetBSD, FreeBSD, and OpenBSD.
[2] This approach was popularized by the book Software Tools (Addison-Wesley).
[3] A 2004 edition of the standard was published after this book's text was finalized. For purposes of learning about shell scripting, the differences between the 2001 and 2004 standard don't matter.
[4] A technical frequently asked questions (FAQ) file about IEEE Std. 1003.1-2001 may be found at http://www.opengroup.org/austin/papers/posix_faq.html. Some background on the standard is at http://www.opengroup.org/austin/papers/backgrounder.html.
Get Classic Shell Scripting now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
