You are previewing bash Cookbook.

bash Cookbook

Cover of bash Cookbook by Carl Albing... Published by O'Reilly Media, Inc.
  1. bash Cookbook
    1. SPECIAL OFFER: Upgrade this ebook with O’Reilly
    2. A Note Regarding Supplemental Files
    3. Preface
      1. Who Should Read This Book
      2. About This Book
      3. Conventions Used in This Book
      4. Using Code Examples
      5. We'd Like to Hear from You
      6. Safari® Enabled
      7. Acknowledgments
    4. 1. Beginning bash
      1. 0. Why bash?
      2. 0. The bash Shell
      3. 1.1. Decoding the Prompt
      4. 1.2. Showing Where You Are
      5. 1.3. Finding and Running Commands
      6. 1.4. Getting Information About Files
      7. 1.5. Showing All Hidden (dot) Files in the Current Directory
      8. 1.6. Using Shell Quoting
      9. 1.7. Using or Replacing Built-ins and External Commands
      10. 1.8. Determining If You Are Running Interactively
      11. 1.9. Setting bash As Your Default Shell
      12. 1.10. Getting bash for Linux
      13. 1.11. Getting bash for xBSD
      14. 1.12. Getting bash for Mac OS X
      15. 1.13. Getting bash for Unix
      16. 1.14. Getting bash for Windows
      17. 1.15. Getting bash Without Getting bash
      18. 1.16. Learning More About bash Documentation
    5. 2. Standard Output
      1. 2.1. Writing Output to the Terminal/Window
      2. 2.2. Writing Output but Preserving Spacing
      3. 2.3. Writing Output with More Formatting Control
      4. 2.4. Writing Output Without the Newline
      5. 2.5. Saving Output from a Command
      6. 2.6. Saving Output to Other Files
      7. 2.7. Saving Output from the ls Command
      8. 2.8. Sending Both Output and Error Messages to Different Files
      9. 2.9. Sending Both Output and Error Messages to the Same File
      10. 2.10. Appending Rather Than Clobbering Output
      11. 2.11. Using Just the Beginning or End of a File
      12. 2.12. Skipping a Header in a File
      13. 2.13. Throwing Output Away
      14. 2.14. Saving or Grouping Output from Several Commands
      15. 2.15. Connecting Two Programs by Using Output As Input
      16. 2.16. Saving a Copy of Output Even While Using It As Input
      17. 2.17. Connecting Two Programs by Using Output As Arguments
      18. 2.18. Using Multiple Redirects on One Line
      19. 2.19. Saving Output When Redirect Doesn't Seem to Work
      20. 2.20. Swapping STDERR and STDOUT
      21. 2.21. Keeping Files Safe from Accidental Overwriting
      22. 2.22. Clobbering a File on Purpose
    6. 3. Standard Input
      1. 3.1. Getting Input from a File
      2. 3.2. Keeping Your Data with Your Script
      3. 3.3. Preventing Weird Behavior in a Here-Document
      4. 3.4. Indenting Here-Documents
      5. 3.5. Getting User Input
      6. 3.6. Getting Yes or No Input
      7. 3.7. Selecting from a List of Options
      8. 3.8. Prompting for a Password
    7. 4. Executing Commands
      1. 4.1. Running Any Executable
      2. 4.2. Telling If a Command Succeeded or Not
      3. 4.3. Running Several Commands in Sequence
      4. 4.4. Running Several Commands All at Once
      5. 4.5. Deciding Whether a Command Succeeds
      6. 4.6. Using Fewer if Statements
      7. 4.7. Running Long Jobs Unattended
      8. 4.8. Displaying Error Messages When Failures Occur
      9. 4.9. Running Commands from a Variable
      10. 4.10. Running All Scripts in a Directory
    8. 5. Basic Scripting: Shell Variables
      1. 5.1. Documenting Your Script
      2. 5.2. Embedding Documentation in Shell Scripts
      3. 5.3. Promoting Script Readability
      4. 5.4. Separating Variable Names from Surrounding Text
      5. 5.5. Exporting Variables
      6. 5.6. Seeing All Variable Values
      7. 5.7. Using Parameters in a Shell Script
      8. 5.8. Looping Over Arguments Passed to a Script
      9. 5.9. Handling Parameters with Blanks
      10. 5.10. Handling Lists of Parameters with Blanks
      11. 5.11. Counting Arguments
      12. 5.12. Consuming Arguments
      13. 5.13. Getting Default Values
      14. 5.14. Setting Default Values
      15. 5.15. Using null As a Valid Default Value
      16. 5.16. Using More Than Just a Constant String for Default
      17. 5.17. Giving an Error Message for Unset Parameters
      18. 5.18. Changing Pieces of a String
      19. 5.19. Using Array Variables
    9. 6. Shell Logic and Arithmetic
      1. 6.1. Doing Arithmetic in Your Shell Script
      2. 6.2. Branching on Conditions
      3. 6.3. Testing for File Characteristics
      4. 6.4. Testing for More Than One Thing
      5. 6.5. Testing for String Characteristics
      6. 6.6. Testing for Equal
      7. 6.7. Testing with Pattern Matches
      8. 6.8. Testing with Regular Expressions
      9. 6.9. Changing Behavior with Redirections
      10. 6.10. Looping for a While
      11. 6.11. Looping with a read
      12. 6.12. Looping with a Count
      13. 6.13. Looping with Floating-Point Values
      14. 6.14. Branching Many Ways
      15. 6.15. Parsing Command-Line Arguments
      16. 6.16. Creating Simple Menus
      17. 6.17. Changing the Prompt on Simple Menus
      18. 6.18. Creating a Simple RPN Calculator
      19. 6.19. Creating a Command-Line Calculator
    10. 7. Intermediate Shell Tools I
      1. 7.1. Sifting Through Files for a String
      2. 7.2. Getting Just the Filename from a Search
      3. 7.3. Getting a Simple True/False from a Search
      4. 7.4. Searching for Text While Ignoring Case
      5. 7.5. Doing a Search in a Pipeline
      6. 7.6. Paring Down What the Search Finds
      7. 7.7. Searching with More Complex Patterns
      8. 7.8. Searching for an SSN
      9. 7.9. Grepping Compressed Files
      10. 7.10. Keeping Some Output, Discarding the Rest
      11. 7.11. Keeping Only a Portion of a Line of Output
      12. 7.12. Reversing the Words on Each Line
      13. 7.13. Summing a List of Numbers
      14. 7.14. Counting String Values
      15. 7.15. Showing Data As a Quick and Easy Histogram
      16. 7.16. Showing a Paragraph of Text After a Found Phrase
    11. 8. Intermediate Shell Tools II
      1. 8.1. Sorting Your Output
      2. 8.2. Sorting Numbers
      3. 8.3. Sorting IP Addresses
      4. 8.4. Cutting Out Parts of Your Output
      5. 8.5. Removing Duplicate Lines
      6. 8.6. Compressing Files
      7. 8.7. Uncompressing Files
      8. 8.8. Checking a tar Archive for Unique Directories
      9. 8.9. Translating Characters
      10. 8.10. Converting Uppercase to Lowercase
      11. 8.11. Converting DOS Files to Linux Format
      12. 8.12. Removing Smart Quotes
      13. 8.13. Counting Lines, Words, or Characters in a File
      14. 8.14. Rewrapping Paragraphs
      15. 8.15. Doing More with less
    12. 9. Finding Files: find, locate, slocate
      1. 9.1. Finding All Your MP3 Files
      2. 9.2. Handling Filenames Containing Odd Characters
      3. 9.3. Speeding Up Operations on Found Files
      4. 9.4. Finding Files Across Symbolic Links
      5. 9.5. Finding Files Irrespective of Case
      6. 9.6. Finding Files by Date
      7. 9.7. Finding Files by Type
      8. 9.8. Finding Files by Size
      9. 9.9. Finding Files by Content
      10. 9.10. Finding Existing Files and Content Fast
      11. 9.11. Finding a File Using a List of Possible Locations
    13. 10. Additional Features for Scripting
      1. 10.1. "Daemon-izing" Your Script
      2. 10.2. Reusing Code with Includes and Sourcing
      3. 10.3. Using Configuration Files in a Script
      4. 10.4. Defining Functions
      5. 10.5. Using Functions: Parameters and Return Values
      6. 10.6. Trapping Interrupts
      7. 10.7. Redefining Commands with alias
      8. 10.8. Avoiding Aliases, Functions
    14. 11. Working with Dates and Times
      1. 11.1. Formatting Dates for Display
      2. 11.2. Supplying a Default Date
      3. 11.3. Automating Date Ranges
      4. 11.4. Converting Dates and Times to Epoch Seconds
      5. 11.5. Converting Epoch Seconds to Dates and Times
      6. 11.6. Getting Yesterday or Tomorrow with Perl
      7. 11.7. Figuring Out Date and Time Arithmetic
      8. 11.8. Handling Time Zones, Daylight Saving Time, and Leap Years
      9. 11.9. Using date and cron to Run a Script on the Nth Day
    15. 12. End-User Tasks As Shell Scripts
      1. 12.1. Starting Simple by Printing Dashes
      2. 12.2. Viewing Photos in an Album
      3. 12.3. Loading Your MP3 Player
      4. 12.4. Burning a CD
      5. 12.5. Comparing Two Documents
    16. 13. Parsing and Similar Tasks
      1. 13.1. Parsing Arguments for Your Shell Script
      2. 13.2. Parsing Arguments with Your Own Error Messages
      3. 13.3. Parsing Some HTML
      4. 13.4. Parsing Output into an Array
      5. 13.5. Parsing Output with a Function Call
      6. 13.6. Parsing Text with a read Statement
      7. 13.7. Parsing with read into an Array
      8. 13.8. Getting Your Plurals Right
      9. 13.9. Taking It One Character at a Time
      10. 13.10. Cleaning Up an SVN Source Tree
      11. 13.11. Setting Up a Database with MySQL
      12. 13.12. Isolating Specific Fields in Data
      13. 13.13. Updating Specific Fields in Data Files
      14. 13.14. Trimming Whitespace
      15. 13.15. Compressing Whitespace
      16. 13.16. Processing Fixed-Length Records
      17. 13.17. Processing Files with No Line Breaks
      18. 13.18. Converting a Data File to CSV
      19. 13.19. Parsing a CSV Data File
    17. 14. Writing Secure Shell Scripts
      1. 14.1. Avoiding Common Security Problems
      2. 14.2. Avoiding Interpreter Spoofing
      3. 14.3. Setting a Secure $PATH
      4. 14.4. Clearing All Aliases
      5. 14.5. Clearing the Command Hash
      6. 14.6. Preventing Core Dumps
      7. 14.7. Setting a Secure $IFS
      8. 14.8. Setting a Secure umask
      9. 14.9. Finding World-Writable Directories in Your $PATH
      10. 14.10. Adding the Current Directory to the $PATH
      11. 14.11. Using Secure Temporary Files
      12. 14.12. Validating Input
      13. 14.13. Setting Permissions
      14. 14.14. Leaking Passwords into the Process List
      15. 14.15. Writing setuid or setgid Scripts
      16. 14.16. Restricting Guest Users
      17. 14.17. Using chroot Jails
      18. 14.18. Running As a Non-root User
      19. 14.19. Using sudo More Securely
      20. 14.20. Using Passwords in Scripts
      21. 14.21. Using SSH Without a Password
      22. 14.22. Restricting SSH Commands
      23. 14.23. Disconnecting Inactive Sessions
    18. 15. Advanced Scripting
      1. 15.1. Finding bash Portably for #!
      2. 15.2. Setting a POSIX $PATH
      3. 15.3. Developing Portable Shell Scripts
      4. 15.4. Testing Scripts in VMware
      5. 15.5. Using for Loops Portably
      6. 15.6. Using echo Portably
      7. 15.7. Splitting Output Only When Necessary
      8. 15.8. Viewing Output in Hex
      9. 15.9. Using bash Net-Redirection
      10. 15.10. Finding My IP Address
      11. 15.11. Getting Input from Another Machine
      12. 15.12. Redirecting Output for the Life of a Script
      13. 15.13. Working Around "argument list too long" Errors
      14. 15.14. Logging to syslog from Your Script
      15. 15.15. Sending Email from Your Script
      16. 15.16. Automating a Process Using Phases
    19. 16. Configuring and Customizing bash
      1. 16.1. bash Startup Options
      2. 16.2. Customizing Your Prompt
      3. 16.3. Change Your $PATH Permanently
      4. 16.4. Change Your $PATH Temporarily
      5. 16.5. Setting Your $CDPATH
      6. 16.6. Shortening or Changing Command Names
      7. 16.7. Adjusting Shell Behavior and Environment
      8. 16.8. Adjusting readline Behavior Using .inputrc
      9. 16.9. Keeping a Private Stash of Utilities by Adding ~/bin
      10. 16.10. Using Secondary Prompts: $PS2, $PS3, $PS4
      11. 16.11. Synchronizing Shell History Between Sessions
      12. 16.12. Setting Shell History Options
      13. 13. Creating a Better cd Command
      14. 16.14. Creating and Changing into a New Directory in One Step
      15. 16.15. Getting to the Bottom of Things
      16. 16.16. Adding New Features to bash Using Loadable Built-ins
      17. 16.17. Improving Programmable Completion
      18. 16.18. Using Initialization Files Correctly
      19. 16.19. Creating Self-Contained, Portable RC Files
      20. 16.20. Getting Started with a Custom Configuration
    20. 17. Housekeeping and Administrative Tasks
      1. 17.1. Renaming Many Files
      2. 17.2. Using GNU Texinfo and Info on Linux
      3. 17.3. Unzipping Many ZIP Files
      4. 17.4. Recovering Disconnected Sessions Using screen
      5. 17.5. Sharing a Single bash Session
      6. 17.6. Logging an Entire Session or Batch Job
      7. 17.7. Clearing the Screen When You Log Out
      8. 17.8. Capturing File Metadata for Recovery
      9. 17.9. Creating an Index of Many Files
      10. 17.10. Using diff and patch
      11. 17.11. Counting Differences in Files
      12. 17.12. Removing or Renaming Files Named with Special Characters
      13. 17.13. Prepending Data to a File
      14. 17.14. Editing a File in Place
      15. 17.15. Using sudo on a Group of Commands
      16. 17.16. Finding Lines in One File But Not in the Other
      17. 17.17. Keeping the Most Recent N Objects
      18. 17.18. Grepping ps Output Without Also Getting the grep Process Itself
      19. 17.19. Finding Out Whether a Process Is Running
      20. 17.20. Adding a Prefix or Suffix to Output
      21. 17.21. Numbering Lines
      22. 17.22. Writing Sequences
      23. 17.23. Emulating the DOS Pause Command
      24. 17.24. Commifying Numbers
    21. 18. Working Faster by Typing Less
      1. 18.1. Moving Quickly Among Arbitrary Directories
      2. 18.2. Repeating the Last Command
      3. 18.3. Running Almost the Same Command
      4. 18.4. Substituting Across Word Boundaries
      5. 18.5. Reusing Arguments
      6. 18.6. Finishing Names for You
      7. 18.7. Playing It Safe
    22. 19. Tips and Traps: Common Goofs for Novices
      1. 19.1. Forgetting to Set Execute Permissions
      2. 19.2. Fixing "No such file or directory" Errors
      3. 19.3. Forgetting That the Current Directory Is Not in the $PATH
      4. 19.4. Naming Your Script Test
      5. 19.5. Expecting to Change Exported Variables
      6. 19.6. Forgetting Quotes Leads to "command not found" on Assignments
      7. 19.7. Forgetting That Pattern Matching Alphabetizes
      8. 19.8. Forgetting That Pipelines Make Subshells
      9. 19.9. Making Your Terminal Sane Again
      10. 19.10. Deleting Files Using an Empty Variable
      11. 19.11. Seeing Odd Behavior from printf
      12. 19.12. Testing bash Script Syntax
      13. 19.13. Debugging Scripts
      14. 19.14. Avoiding "command not found" When Using Functions
      15. 19.15. Confusing Shell Wildcards and Regular Expressions
    23. A. Reference Lists
      1. A.1. bash Invocation
      2. A.2. Prompt String Customizations
      3. A.3. ANSI Color Escape Sequences
      4. A.4. Built-in Commands and Reserved Words
      5. A.5. Built-in Shell Variables
      6. A.6. set Options
      7. A.7. shopt options
      8. A.8. Adjusting Shell Behavior Using set, shopt, and Environment Variables
      9. A.9. Test Operators
      10. A.10. I/O Redirection
      11. A.11. echo Options and Escape Sequences
      12. A.12. printf
      13. A.13. Date and Time String Formatting with strftime
      14. A.14. Pattern-Matching Characters
      15. A.15. extglob Extended Pattern-Matching Operators
      16. A.16. tr Escape Sequences
      17. A.17. Readline Init File Syntax
      18. A.18. emacs Mode Commands
      19. A.19. vi Control Mode Commands
      20. A.20. Table of ASCII Values
    24. B. Examples Included with bash
      1. B.1. Startup-Files Directory Examples
    25. C. Command-Line Processing
      1. C.1. Command-Line Processing Steps
    26. D. Revision Control
      1. D.1. CVS
      2. D.2. Subversion
      3. D.3. RCS
      4. D.4. Other
    27. E. Building bash from Source
      1. E.1. Obtaining bash
      2. E.2. Unpacking the Archive
      3. E.3. What's in the Archive
      4. E.4. Who Do I Turn To?
    28. Index
    29. About the Authors
    30. Colophon
    31. SPECIAL OFFER: Upgrade this ebook with O’Reilly
O'Reilly logo

Chapter 4. Executing Commands

The main purpose of bash (or of any shell) is to allow you to interact with the computer's operating system so that you can accomplish whatever you need to do. Usually that involves launching programs, so the shell takes the commands you type, determines from that input what programs need to be run, and launches them for you.

Let's take a look at the basic mechanism for launching jobs and explore some of the features bash offers for launching programs in the foreground or the background, sequentially or in parallel, indicating whether programs succeeded and more.

Running Any Executable

Problem

You need to run a command on a Linux or Unix system.

Solution

Use bash and type the name of the command at the prompt.

	$someprog

Discussion

This seems rather simple, and in a way it is, but a lot goes on behind the scenes that you never see. What's important to understand about bash is that its basic operation is to load and execute programs. All the rest is just window dressing to get ready to run programs. Sure there are shell variables and control statements for looping and if/then/else branching, and there are ways to control input and output, but they are all icing on the cake of program execution.

So where does it get the program to run?

bash will use a shell variable called $PATH to locate your executable. The $PATH variable is a list of directories. The directories are separated by colons (:). bash will search in each of those directories for a file with the name that you specified. The order of the directories is important—bash looks at the order in which the directories are listed in the variable, and takes the first executable found.

	$ echo $PATH
	/bin:/usr/bin:/usr/local/bin:.
	$

In the $PATH variable shown above, four directories are included. The last directory in that list is just a single dot (called the dot directory, or just dot), which represents the current directory. The dot is the name of the directory found within every directory on a Linux or Unix file system—wherever you are, that's the directory to which dot refers. For example, when you copy a file from someplace to dot (i.e., cp /other/place/file.), you are copying the file into the current directory. By having the dot directory listed in our path, bash will look for commands not just in those other directories, but also in the current directory (.).

Many people feel that putting dot on your $PATH is too great a security risk—some-one could trick you and get you to run their own (malicious) version of a command in place of one that you were expecting. Now if dot were listed first, then someone else's version of ls would supersede the normal ls command and you might unwittingly run that command. Don't believe us? Try this:

	$ bash
	$ cd
	$ touch ls
	$ chmod 755 ls
	$ PATH=".:$PATH"
	$ ls
	$

Suddenly, the ls appears not to work in your home directory. You get no output. When you cd to some other location (e.g., cd /tmp), then ls will work, but not in your home directory. Why? Because in that directory there is an empty file called ls that is run (and does nothing—it's empty) instead of the normal ls command located at /bin/ls. Since we started this example by running a new copy of bash , you can exit from this mess by exiting this subshell; but you might want to remove the bogus ls command first:

	$ cd
	$ rm ls
	$ exit
	$

Can you see the mischief potential of wandering into a strange directory with your path set to search the dot directory before anywhere else?

If you put dot as the last directory in your $PATH variable, at least you won't be tricked that easily. Of course, if you leave it off altogether it is arguably even safer and you can still run commands in your local directory by typing a leading dot and slash character, as in:

	$./myscript

The choice is yours.

Warning

Never allow a dot or writable directories in root's $PATH. For more, see Finding World-Writable Directories in Your $PATH, "Finding World-Writable Directories in Your $PATH" and Adding the Current Directory to the $PATH, "Adding the Current Directory to the $PATH."

Don't forget to set the file's permissions to execute permission before you invoke your script:

	$ chmod a+x ./myscript
	$ ./myscript

You only need to set the permissions once. Thereafter you can invoke the script as a command.

A common practice among some bash experts is to create a personal bin directory, analogous to the system directories /bin and /usr/bin where executables are kept. In your personal bin you can put copies of your favorite shell scripts and other customized or private commands. Then add your home directory to your $PATH, even to the front (PATH=~/bin:$PATH). That way, you can still have your own customized favorites without the security risk of running commands from strangers.

See Also

The best content for your career. Discover unlimited learning on demand for around $1/day.