Shell Scripting Recipes: A Problem-Solution Approach

Table of contents

1. Copyright
2. About the Author
3. About the Technical Reviewer
4. Acknowledgments
5. Introduction
   1. Why Shell Scripts?
   2. Which Shells Will Run These Scripts?
   3. Looking Through the Book
   4. Thinking Outside the Book
6. 1. The POSIX Shell and Command-Line Utilities
   1. 1.1. Shell Commands
      1. 1.1.1. echo
      2. 1.1.2. printf
      3. 1.1.3. set
      4. 1.1.4. shift
      5. 1.1.5. type
      6. 1.1.6. getopts
      7. 1.1.7. case
      8. 1.1.8. eval
      9. 1.1.9. local
   2. 1.2. Parameters and Variables
      1. 1.2.1. Positional Parameters
      2. 1.2.2. Special Parameters
      3. 1.2.3. standard-vars—A Collection of Useful Variables
   3. 1.3. Patterns
      1. 1.3.1. Pathname Expansion
      2. 1.3.2. Regular Expressions
   4. 1.4. Parameter Expansion
      1. 1.4.1. The Bourne Shell Expansions
      2. 1.4.2. POSIX Parameter Expansions
      3. 1.4.3. Shell-Specific Expansions, bash2, and ksh93
   5. 1.5. Shell Arithmetic
   6. 1.6. Aliases
   7. 1.7. Sourcing a File
   8. 1.8. Functions
      1. 1.8.1. Functions Are Fast
      2. 1.8.2. Command Substitution Is Slow
      3. 1.8.3. Using the Functions in This Book
   9. 1.9. standard-funcs: A Collection of Useful Commands
      1. 1.9.1. get_key—Get a Single Keystroke from the User
      2. 1.9.2. getline—Prompt User to Enter a Line
      3. 1.9.3. press_any_key—Prompt for a Single Keypress
      4. 1.9.4. menu1—Print a Menu and Execute a Selected Command
      5. 1.9.5. arg—Prompt for Required Argument If None Supplied
      6. 1.9.6. die—Print Error Message and Exit with Error Status
      7. 1.9.7. show_date—Display Date in D[D] MMM YYYY Format
      8. 1.9.8. date_vars—Set Date and Time Variables
      9. 1.9.9. is_num—Is This a Positive Integer?
      10. 1.9.10. abbrev_num—Abbreviate Large Numbers
      11. 1.9.11. commas—Add Thousands Separators to a Number
      12. 1.9.12. pr1—Print Arguments, One to a Line
      13. 1.9.13. checkdirs—Check for Directories; Create If Necessary
      14. 1.9.14. checkfiles—Check That a Directory Contains Certain Files
      15. 1.9.15. zpad—Pad a Number with Leading Zeroes
      16. 1.9.16. cleanup—Remove Temporary Files and Reset Terminal on Exit
   10. 1.10. The Unix Utilities
       1. 1.10.1. cat: Concatenate Files to the Standard Output
       2. 1.10.2. sed: A Text Stream Editor
       3. 1.10.3. awk: Pattern Scanning and Processing Language
       4. 1.10.4. grep: Print Lines Matching a Regular Expression
       5. 1.10.5. date: Show or Set the System Date
       6. 1.10.6. tr: A Character Translation Utility
       7. 1.10.7. wc: Count Characters, Words, and Lines in a File
       8. 1.10.8. file: Determine the File Type
       9. 1.10.9. ls: Sort and Provide Details About Files
       10. 1.10.10. uniq: Remove Consecutive Duplicate Lines
       11. 1.10.11. sudo: Execute Commands as the Superuser
       12. 1.10.12. split: Divide a File into Equal-Sized Pieces
       13. 1.10.13. which: Show the Full Path to a Command
       14. 1.10.14. gs, gv: Render, Convert, or View PostScript and PDF Files
   11. 1.11. Summary
7. 2. Playing with Files: Viewing, Manipulating, and Editing Text Files
   1. 2.1. End of the Line for OS Differences: Converting Text Files
      1. 2.1.1. dos2unix—Convert Windows Text Files to Unix
      2. 2.1.2. unix2dos—Convert a Unix File to Windows
      3. 2.1.3. mac2unix—Convert Macintosh Files to Unix
      4. 2.1.4. unix2mac—Convert Unix Files to Mac Format
      5. 2.1.5. dos2mac—Convert Windows Files to Macintosh
      6. 2.1.6. mac2dos—Convert Macintosh Files to Windows
   2. 2.2. Displaying Files
      1. 2.2.1. prn—Print File with Line Numbers
      2. 2.2.2. prw—Print One Word per Line
      3. 2.2.3. wbl—Sort Words by Length
   3. 2.3. Formatting File Facts
      1. 2.3.1. finfo—File Information Display
      2. 2.3.2. wfreq—Word Frequency
      3. 2.3.3. lfreq—Letter Frequency
      4. 2.3.4. fed—A Simple Batch File Editor
   4. 2.4. Summary
8. 3. String Briefs
   1. 3.1. Character Actions: The char-funcs Library
      1. 3.1.1. chr—Convert a Decimal Number to an ASCII Character
      2. 3.1.2. asc—Convert a Character to Its Decimal Equivalent
      3. 3.1.3. nxt—Get the Next Character in ASCII Sequence
      4. 3.1.4. upr—Convert Character(s) to Uppercase
      5. 3.1.5. lwr—Convert Character(s) to Lowercase
   2. 3.2. String Cleaning: The string-funcs Library
      1. 3.2.1. sub—Replace First Occurrence of a Pattern
      2. 3.2.2. gsub—Globally Replace a Pattern in a String
      3. 3.2.3. repeat—Build a String of a Specified Length
      4. 3.2.4. index, rindex—Find Position of One String Within Another
      5. 3.2.5. substr—Extract a Portion of a String
      6. 3.2.6. insert_str—Place One String Inside Another
   3. 3.3. Summary
9. 4. What's in a Word?
   1. 4.1. Finding and Massaging Word Lists
   2. 4.2. wf-funcs: WordFinder Function Library
      1. 4.2.1. write_config—Write User's Information to the Configuration File
      2. 4.2.2. do_config—Check For and Source Default Configuration File
      3. 4.2.3. set_sysdict—Select the Dictionary Directory
      4. 4.2.4. mkwsig—Sort Letters in a Word
      5. 4.2.5. wf-clean—Remove Carriage Returns and Accents
      6. 4.2.6. wf-compounds—Squish Compound Words and Save with Lengths
      7. 4.2.7. wf-setup—Prepare Word and Anagram Lists
   3. 4.3. Playing with Matches
      1. 4.3.1. wf—Find Words That Match a Pattern
      2. 4.3.2. wfb—Find Words That Begin with a Given Pattern
      3. 4.3.3. wfe—Find Words That End with a Given Pattern
      4. 4.3.4. wfc—Find Words That Contain a Given Pattern
      5. 4.3.5. wfit—Find Words That Fit Together in a Grid
      6. 4.3.6. anagram—Find Words That Are Anagrams of a Given Word
      7. 4.3.7. aplus—Find Anagrams of a Word with a Letter Added
      8. 4.3.8. aminus—Remove Each Letter in Turn and Anagram What's Left
   4. 4.4. Summary
10. 5. Scripting by Numbers
    1. 5.1. The math-funcs Library
       1. 5.1.1. fpmul—Multiply Decimal Fractions
       2. 5.1.2. int—Return the Integer Portion of a Decimal Fraction
       3. 5.1.3. round—Round the Argument to the Nearest Integer
       4. 5.1.4. pow—Raise a Number to Any Given Power
       5. 5.1.5. square—Raise a Number to the Second Power
       6. 5.1.6. cube—Raise a Number to the Third Power
       7. 5.1.7. calc—A Simple Command-Line Calculator
    2. 5.2. Adding and Averaging
       1. 5.2.1. total—Add a List of Numbers
       2. 5.2.2. mean—Find the Arithmetic Mean of a List of Numbers
       3. 5.2.3. median—Find the Median of a List of Numbers
       4. 5.2.4. mode—Find the Number That Appears Most in a List
       5. 5.2.5. range—Find the Range of a Set of Numbers
       6. 5.2.6. stdev—Finding the Standard Deviation
    3. 5.3. Converting Between Unit Systems
       1. 5.3.1. conversion-funcs—Converting Metric Units
       2. 5.3.2. conversion—A Menu System for Metric Conversion
    4. 5.4. Summary
11. 6. Loose Names Sink Scripts: Bringing Sanity to Filenames
    1. 6.1. What's in a Name?
       1. 6.1.1. POSIX Portable Filenames
       2. 6.1.2. OK Filenames
    2. 6.2. Functioning Filenames: The filename-funcs Library
       1. 6.2.1. basename—Extract the Last Element of a Pathname
       2. 6.2.2. dirname—Return All but the Last Element of a Pathname
       3. 6.2.3. is_pfname—Check for POSIX Portable Filename
       4. 6.2.4. is_OKfname—Check Whether a Filename Is Acceptable
       5. 6.2.5. pfname—Convert Nonportable Characters in Filename
       6. 6.2.6. OKfname—Make a Filename Acceptable
       7. 6.2.7. is_whitespc—Does the Filename Contain Whitespace Characters?
       8. 6.2.8. whitespc—Fix Filenames Containing Whitespace Characters
       9. 6.2.9. is_dir—Is This a Directory I See Before Me?
       10. 6.2.10. nodoublechar—Remove Duplicate Characters from a String
       11. 6.2.11. new_filename—Change Filename to Desired Character Set
       12. 6.2.12. fix_pwd—Fix All the Filenames in the Current Directory
       13. 6.2.13. fixfname—Convert Filenames to Sensible Names
    3. 6.3. Summary
12. 7. Treading a Righteous PATH
    1. 7.1. The path-funcs Library
       1. 7.1.1. path—Display a User's PATH
       2. 7.1.2. unslash—Remove Extraneous Slashes
       3. 7.1.3. checkpath—Clean Up the PATH Variable
       4. 7.1.4. addpath—Add Directories to the PATH Variable
       5. 7.1.5. rmpath—Remove One or More Directories from $PATH
    2. 7.2. Summary
13. 8. The Dating Game
    1. 8.1. The date-funcs Library
       1. 8.1.1. split_date—Divide a Date into Day, Month, and Year
       2. 8.1.2. is_leap_year—Is There an Extra Day This Year?
       3. 8.1.3. days_in_month—How Many Days Hath September?
       4. 8.1.4. date2julian—Calculate the Julian Day Number
       5. 8.1.5. julian2date—Convert Julian Back to Year, Month, and Day
       6. 8.1.6. dateshift—Add or Subtract a Number of Days
       7. 8.1.7. diffdate—Find the Number of Days Between Two Dates
       8. 8.1.8. day_of_week—Find the Day of the Week for Any Date
       9. 8.1.9. display_date—Show a Date in Text Format
       10. 8.1.10. parse_date—Decipher Various Forms of Date String
       11. 8.1.11. valid_date—Where Was I on November 31st?
    2. 8.2. Summary
14. 9. Good Housekeeping: Monitoring and Tidying Up File Systems
    1. 9.1.
       1. 9.1.1. dfcmp—Notify User of Major Changes in Disk Usage
       2. 9.1.2. symfix—Remove Broken Symbolic Links
       3. 9.1.3. sym2file—Converts Symbolic Links to Regular Files
       4. 9.1.4. zrm—Remove Empty Files
       5. 9.1.5. undup—Remove Duplicate Files
       6. 9.1.6. lsr—List the Most Recent (or Oldest) Files in a Directory
    2. 9.2. Summary
15. 10. POP Goes the E-Mail
    1. 10.1.
       1. 10.1.1. popcheck—Display Number of Messages in POP3 Mailbox
       2. 10.1.2. pop3-funcs—Functions for Managing a POP3 Mailbox
       3. 10.1.3. pop3list—Examine a POP3 Mailbox
       4. 10.1.4. pop3filter—Delete and Retrieve Messages from a POP3 Server
       5. 10.1.5. pflog—Print a Summary of a Day's Activity
       6. 10.1.6. viewlog—Continuously View the pop3filter Log File
    2. 10.2. Summary
16. 11. PostScript: More Than an Afterthought
    1. 11.1.
       1. 11.1.1. ps-labels—Producing Mailing Labels from a List
       2. 11.1.2. ps-envelopes—Print Envelopes from List
       3. 11.1.3. ps-grid—Convert a Crossword Grid to PostScript
    2. 11.2. Summary
17. 12. Screenplay: The screen—funcs Library
    1. 12.1.
       1. 12.1.1. screen-vars—Variables for Screen Manipulation
       2. 12.1.2. set_attr—Set Screen-Printing Attributes
       3. 12.1.3. set_fg, set_bg, set_fgbg-—Set Colors forPrinting to the Screen
       4. 12.1.4. cls—Clear the Screen
       5. 12.1.5. printat—Position Cursor by Row and Column
       6. 12.1.6. put_block_at—Print Lines in a Column Anywhere on the Screen
       7. 12.1.7. get_size—Set LINES and COLUMNS Variables
       8. 12.1.8. max_length—Find the Length of the Longest Argument
       9. 12.1.9. print_block_atxy—Print a Block of Lines Anywhere on the Screen
       10. 12.1.10. vbar, hbar—Print a Vertical or Horizontal Bar
       11. 12.1.11. center—Center a String on N Columns
       12. 12.1.12. flush_right—Align String with the Right Margin
       13. 12.1.13. ruler—Draw a Ruler Across the Width and Height of the Window
       14. 12.1.14. box_block, box_block_at—Print Text Surrounded by a Box
       15. 12.1.15. clear_area, clear_area_at—Clear an Area of the Screen
       16. 12.1.16. box_area, box_area_at—Draw a Box Around an Area
       17. 12.1.17. screen-demo—Saving and Redisplaying Areas of the Screen
    2. 12.2. Summary
18. 13. Backing Up the Drive
    1. 13.1.
       1. 13.1.1. bu—Backup Directories
       2. 13.1.2. bin-pack—Fill Fixed-Capacity Directories with Files
       3. 13.1.3. unpack—Extract All Files from One or More Archives
    2. 13.2. Summary
19. 14. Aging, Archiving, and Deleting Files
    1. 14.1.
       1. 14.1.1. date-file—Add a Datestamp to a Filename
       2. 14.1.2. rmold—Remove Old Files
       3. 14.1.3. keepnewest—Remove All but the Newest or Oldest Files
    2. 14.2. Summary
20. 15. Covering All Your Databases
    1. 15.1.
       1. 15.1.1. lookup—Find the Corresponding Value for a Key
    2. 15.2. shdb-funcs: Shell Database Function Library
       1. 15.2.1. load_db—Import Database into Shell Array
       2. 15.2.2. split_record—Split a Record into Fields
       3. 15.2.3. csv_split—Extract Fields from CSV Records
       4. 15.2.4. put_record—Assemble a Database Record from an Array
       5. 15.2.5. put_csv—Assemble Fields into a CSV Record
       6. 15.2.6. db-demo—View and Edit a Password File
    3. 15.3. PhoneBase: A Simple Phone Number Database
       1. 15.3.1. ph—Look Up a Phone Number
       2. 15.3.2. phadd—Add an Entry to PhoneBase
       3. 15.3.3. phdel—Delete an Entry from PhoneBase
       4. 15.3.4. phx—Show ph Search Results in an X Window
    4. 15.4. Summary
21. 16. Home on the Web
    1. 16.1. Playing with Hypertext: The html-funcs Library
       1. 16.1.1. get_element—Extract the First Occurrence of an Element
       2. 16.1.2. split_tags—Put Each Tag on Its Own Line
       3. 16.1.3. html-title—Get the Title from an HTML File
    2. 16.2. HTML on the Fly: The cgi-funcs Library
       1. 16.2.1. x2d2—Convert a Two-Digit Hexadecimal Number to Decimal
       2. 16.2.2. dehex—Convert Hex Strings (%XX) to Characters
       3. 16.2.3. filedate—Find and Format the Modification Date of a File
    3. 16.3. Creating HTML Files
       1. 16.3.1. mk-htmlindex—Create an HTML Index
       2. 16.3.2. pretext—Create a Wrapper Around a Text File
       3. 16.3.3. text2html—Convert a Text File to HTML
       4. 16.3.4. demo.cgi—A CGI Script
    4. 16.4. Summary
22. 17. Taking Care of Business
    1. 17.1.
       1. 17.1.1. prcalc—A Printing Calculator
       2. 17.1.2. gle—Keeping Records Without a Shoebox
    2. 17.2. Summary
23. 18. Random Acts of Scripting
    1. 18.1. The rand-funcs Library
       1. 18.1.1. random—Return One or More Random Integers in a Given Range
       2. 18.1.2. toss—Simulate Tossing a Coin
       3. 18.1.3. randstr—Select a String at Random
    2. 18.2. A Random Sampling of Scripts
       1. 18.2.1. rand-date—Generate Random Dates in ISO Format
       2. 18.2.2. randsort—Print Lines in Random Order
       3. 18.2.3. randomword—Generate Random Words According to Format Specifications
       4. 18.2.4. dice—Roll a Set of Dice
       5. 18.2.5. throw—Throw a Pair of Dice
    3. 18.3. Summary
24. 19. A Smorgasbord of Scripts
    1. 19.1.
       1. 19.1.1. topntail—Remove Top and Bottom Lines from a File
       2. 19.1.2. flocate—Locate Files by Filename Alone
       3. 19.1.3. sus—Display a POSIX Man Page
       4. 19.1.4. cwbw—Count Words Beginning With
       5. 19.1.5. cci—Configure, Compile, and Install from Tarball
       6. 19.1.6. ipaddr—Find a Computer's Network Address
       7. 19.1.7. ipaddr.cgi—Print the Remote Address of an HTTP Connection
       8. 19.1.8. iprev—Reverse the Order of Digits in an IP Address
       9. 19.1.9. intersperse—Insert a String Between Doubled Characters
       10. 19.1.10. ll—Use a Pager for a Directory Listing Only If Necessary
       11. 19.1.11. name-split—Divide a Person's Full Name into First, Last, and Middle Names
       12. 19.1.12. rot13—Encode or Decode Text
       13. 19.1.13. showfstab—Show Information from /etc/fstab
       14. 19.1.14. unique—Remove All Duplicate Lines from a File
    2. 19.2. Summary
25. 20. Script Development Management
    1. 20.1.
       1. 20.1.1. script-setup—Prepare the Scripting Environment
       2. 20.1.2. cpsh—Install Script and Make Backup Copy
       3. 20.1.3. shgrep—Search Scripts for String or Regular Expression
       4. 20.1.4. shcat—Display a Shell Script
    2. 20.2. Summary
26. A. Internet Scripting Resources
    1. A.1. Introductions to Shell Scripting
    2. A.2. Intermediate and Advanced Scripting
    3. A.3. Collections of Scripts
    4. A.4. Home Pages for Shells
    5. A.5. Regular Expressions, sed, and awk
    6. A.6. Miscellaneous Pages
    7. A.7. History of the Shell