You are previewing Linux® Programming by Example.
O'Reilly logo
Linux® Programming by Example

Book Description

“This is an excellent introduction to Linux programming. The topics are well chosen and lucidly presented. I learned things myself, especially about internationalization, and I’ve been at this for quite a while.”

—Chet Ramey, Coauthor and Maintainer of the Bash shell

“This is a good introduction to Linux programming. Arnold’s technique of showing how experienced programmers use the Linux programming interfaces is a nice touch, much more useful than the canned programming examples found in most books.”

—Ulrich Drepper, Project Lead, GNU C library

“A gentle yet thorough introduction to the art of UNIX system programming, Linux Programming by Example uses code from a wide range of familiar programs to illustrate each concept it teaches. Readers will enjoy an interesting mix of in-depth API descriptions and portability guidelines, and will come away well prepared to begin reading and writing systems applications. Heartily recommended.”

—Jim Meyering, Coauthor and Maintainer of the GNU Core Utility Programs

Learn Linux® programming, hands-on… from real source code

This book teaches Linux programming in the most effective way possible: by showing and explaining well-written programs. Drawing from both V7 Unix® and current GNU source code, Arnold Robbins focuses on the fundamental system call APIs at the core of any significant program, presenting examples from programs that Linux/Unix users already use every day. Gradually, one step at a time, Robbins teaches both high-level principles and “under the hood” techniques. Along the way, he carefully addresses real-world issues like performance, portability, and robustness. Coverage includes:

  • Memory management

  • File I/O

  • File metadata

  • Processes

  • Users and groups

  • Sorting and searching

  • Argument parsing

  • Extended interfaces

  • Signals

  • Internationalization

  • Debugging

  • And more…

  • Just learning to program? Switching from Windows®? Already developing with Linux but interested in exploring the system call interface further? No matter which, quickly and directly, this book will help you master the fundamentals needed to build serious Linux software.

    Companion Web Sites,authors.phptr.com/robbins and www.linux-by-example.com, include all code examples.

    Table of Contents

    1. Copyright
      1. Dedication
    2. Preface
      1. Audience
      2. What You Will Learn
      3. Small Is Beautiful: Unix Programs
      4. Standards
      5. Features and Power: GNU Programs
      6. Summary of Chapters
      7. Typographical Conventions
      8. Where to Get Unix and GNU Source Code
      9. Unix Code
      10. GNU Code
      11. Where to Get the Example Programs Used in This Book
      12. About the Cover
      13. Acknowledgments
    3. I. Files and Users
      1. 1. Introduction
        1. 1.1. The Linux/Unix File Model
          1. 1.1.1. Files and Permissions
          2. 1.1.2. Directories and Filenames
          3. 1.1.3. Executable Files
          4. 1.1.4. Devices
        2. 1.2. The Linux/Unix Process Model
          1. 1.2.1. Pipes: Hooking Processes Together
        3. 1.3. Standard C vs. Original C
        4. 1.4. Why GNU Programs Are Better
          1. 1.4.1. Program Design
          2. 1.4.2. Program Behavior
          3. 1.4.3. C Code Programming
          4. 1.4.4. Things That Make a GNU Program Better
          5. 1.4.5. Parting Thoughts about the “GNU Coding Standards”
        5. 1.5. Portability Revisited
        6. 1.6. Suggested Reading
        7. 1.7. Summary
        8. Exercises
      2. 2. Arguments, Options, and the Environment
        1. 2.1. Option and Argument Conventions
          1. 2.1.1. POSIX Conventions
          2. 2.1.2. GNU Long Options
        2. 2.2. Basic Command-Line Processing
          1. 2.2.1. The V7 echo Program
        3. 2.3. Option Parsing: getopt() and getopt_long()
          1. 2.3.1. Single-Letter Options
          2. 2.3.2. GNU getopt() and Option Ordering
          3. 2.3.3. Long Options
            1. 2.3.3.1. Long Options Table
            2. 2.3.3.2. Long Options, POSIX Style
            3. 2.3.3.3. getopt_long() Return Value Summary
            4. 2.3.3.4. GNU getopt() or getopt_long() in User Programs
        4. 2.4. The Environment
          1. 2.4.1. Environment Management Functions
          2. 2.4.2. The Entire Environment: environ
          3. 2.4.3. GNU env
        5. 2.5. Summary
        6. Exercises
      3. 3. User-Level Memory Management
        1. 3.1. Linux/Unix Address Space
        2. 3.2. Memory Allocation
          1. 3.2.1. Library Calls: malloc(), calloc(), realloc(), free()
            1. 3.2.1.1. Examining C Language Details
            2. 3.2.1.2. Initially Allocating Memory: malloc()
            3. 3.2.1.3. Releasing Memory: free()
            4. 3.2.1.4. Changing Size: realloc()
            5. 3.2.1.5. Allocating and Zero-filling: calloc()
            6. 3.2.1.6. Summarizing from the GNU Coding Standards
            7. 3.2.1.7. Using Private Allocators
            8. 3.2.1.8. Example: Reading Arbitrarily Long Lines
            9. 3.2.1.9. GLIBC Only: Reading Entire Lines: getline() and getdelim()
          2. 3.2.2. String Copying: strdup()
          3. 3.2.3. System Calls: brk() and sbrk()
          4. 3.2.4. Lazy Programmer Calls: alloca()
          5. 3.2.5. Address Space Examination
        3. 3.3. Summary
        4. Exercises
      4. 4. Files and File I/O
        1. 4.1. Introducing the Linux/Unix I/O Model
        2. 4.2. Presenting a Basic Program Structure
        3. 4.3. Determining What Went Wrong
          1. 4.3.1. Values for errno
          2. 4.3.2. Error Message Style
        4. 4.4. Doing Input and Output
          1. 4.4.1. Understanding File Descriptors
          2. 4.4.2. Opening and Closing Files
            1. 4.4.2.1. Mapping FILE * Variables to File Descriptors
            2. 4.4.2.2. Closing All Open Files
          3. 4.4.3. Reading and Writing
          4. 4.4.4. Example: Unix cat
        5. 4.5. Random Access: Moving Around within a File
        6. 4.6. Creating Files
          1. 4.6.1. Specifying Initial File Permissions
          2. 4.6.2. Creating Files with creat()
          3. 4.6.3. Revisiting open()
        7. 4.7. Forcing Data to Disk
        8. 4.8. Setting File Length
        9. 4.9. Summary
        10. Exercises
      5. 5. Directories and File Metadata
        1. 5.1. Considering Directory Contents
          1. 5.1.1. Definitions
          2. 5.1.2. Directory Contents
          3. 5.1.3. Hard Links
            1. 5.1.3.1. The GNU link Program
            2. 5.1.3.2. Dot and Dot-Dot
          4. 5.1.4. File Renaming
          5. 5.1.5. File Removal
            1. 5.1.5.1. Removing Open Files
            2. 5.1.5.2. Using ISO C: remove()
          6. 5.1.6. Symbolic Links
        2. 5.2. Creating and Removing Directories
        3. 5.3. Reading Directories
          1. 5.3.1. Basic Directory Reading
            1. 5.3.1.1. Portability Considerations
            2. 5.3.1.2. Linux and BSD Directory Entries
          2. 5.3.2. BSD Directory Positioning Functions
        4. 5.4. Obtaining Information about Files
          1. 5.4.1. Linux File Types
          2. 5.4.2. Retrieving File Information
          3. 5.4.3. Linux Only: Specifying Higher-Precision File Times
          4. 5.4.4. Determining File Type
            1. 5.4.4.1. Device Information
            2. 5.4.4.2. The V7 cat Revisited
          5. 5.4.5. Working with Symbolic Links
        5. 5.5. Changing Ownership, Permission, and Modification Times
          1. 5.5.1. Changing File Ownership: chown(), fchown(), and lchown()
          2. 5.5.2. Changing Permissions: chmod() and fchmod()
          3. 5.5.3. Changing Timestamps: utime()
            1. 5.5.3.1. Faking utime(file, NULL)
          4. 5.5.4. Using fchown() and fchmod() for Security
        6. 5.6. Summary
        7. Exercises
      6. 6. General Library Interfaces — Part 1
        1. 6.1. Times and Dates
          1. 6.1.1. Retrieving the Current Time: time() and difftime()
          2. 6.1.2. Breaking Down Times: gmtime() and localtime()
          3. 6.1.3. Formatting Dates and Times
            1. 6.1.3.1. Simple Time Formatting: asctime() and ctime()
            2. 6.1.3.2. Complex Time Formatting: strftime()
          4. 6.1.4. Converting a Broken-Down Time to a time_t
          5. 6.1.5. Getting Time-Zone Information
            1. 6.1.5.1. BSD Systems Gotcha: timezone(), Not timezone
        2. 6.2. Sorting and Searching Functions
          1. 6.2.1. Sorting: qsort()
            1. 6.2.1.1. Example: Sorting Employees
            2. 6.2.1.2. Example: Sorting Directory Contents
          2. 6.2.2. Binary Searching: bsearch()
        3. 6.3. User and Group Names
          1. 6.3.1. User Database
          2. 6.3.2. Group Database
        4. 6.4. Terminals: isatty()
        5. 6.5. Suggested Reading
        6. 6.6. Summary
        7. Exercises
      7. 7. Putting It All Together: ls
        1. 7.1. V7 ls Options
        2. 7.2. V7 ls Code
        3. 7.3. Summary
        4. Exercises
      8. 8. Filesystems and Directory Walks
        1. 8.1. Mounting and Unmounting Filesystems
          1. 8.1.1. Reviewing the Background
          2. 8.1.2. Looking at Different Filesystem Types
          3. 8.1.3. Mounting Filesystems: mount
          4. 8.1.4. Unmounting Filesystems: umount
        2. 8.2. Files for Filesystem Administration
          1. 8.2.1. Using Mount Options
          2. 8.2.2. Working with Mounted Filesystems: getmntent()
        3. 8.3. Retrieving Per-Filesystem Information
          1. 8.3.1. POSIX Style: statvfs() and fstatvfs()
          2. 8.3.2. Linux Style: statfs() and fstatfs()
        4. 8.4. Moving Around in the File Hierarchy
          1. 8.4.1. Changing Directory: chdir() and fchdir()
          2. 8.4.2. Getting the Current Directory: getcwd()
          3. 8.4.3. Walking a Hierarchy: nftw()
            1. 8.4.3.1. The nftw() Interface
            2. 8.4.3.2. The nftw() Callback Function
        5. 8.5. Walking a File Tree: GNU du
        6. 8.6. Changing the Root Directory: chroot()
        7. 8.7. Summary
        8. Exercises
    4. II. Processes, IPC, and Internationalization
      1. 9. Process Management and Pipes
        1. 9.1. Process Creation and Management
          1. 9.1.1. Creating a Process: fork()
            1. 9.1.1.1. After the fork(): Shared and Distinct Attributes
            2. 9.1.1.2. File Descriptor Sharing
            3. 9.1.1.3. File Descriptor Sharing and Close()
          2. 9.1.2. Identifying a Process: getpid() and getppid()
          3. 9.1.3. Setting Process Priority: nice()
            1. 9.1.3.1. POSIX vs. Reality
          4. 9.1.4. Starting New Programs: The exec() Family
            1. 9.1.4.1. The execve() System Call
            2. 9.1.4.2. Wrapper Functions: excel() et al.
            3. 9.1.4.3. Program Names and argv[0]
            4. 9.1.4.4. Attributes Inherited across exec()
          5. 9.1.5. Terminating a Process
            1. 9.1.5.1. Defining Process Exit Status
            2. 9.1.5.2. Returning from main()
            3. 9.1.5.3. Exiting Functions
          6. 9.1.6. Recovering a Child’s Exit Status
            1. 9.1.6.1. Using POSIX Functions: wait() and waitpid()
            2. 9.1.6.2. Using BSD Functions: wait3() and wait4()
        2. 9.2. Process Groups
          1. 9.2.1. Job Control Overview
          2. 9.2.2. Process Group Identification: getpgrp() and getpgid()
          3. 9.2.3. Process Group Setting: setpgid() and setpgrp()
        3. 9.3. Basic Interprocess Communication: Pipes and FIFOs
          1. 9.3.1. Pipes
            1. 9.3.1.1. Creating Pipes
            2. 9.3.1.2. Pipe Buffering
          2. 9.3.2. FIFOs
        4. 9.4. File Descriptor Management
          1. 9.4.1. Duplicating Open Files: dup() and dup2()
          2. 9.4.2. Creating Nonlinear Pipelines: /dev/fd/XX
          3. 9.4.3. Managing File Attributes: fcntl()
            1. 9.4.3.1. The Close-on-exec Flag
            2. 9.4.3.2. File Descriptor Duplication
            3. 9.4.3.3. Manipulation of File Status Flags and Access Modes
            4. 9.4.3.4. Nonblocking I/O for Pipes and FIFOs
            5. 9.4.3.5. fcntl() Summary
        5. 9.5. Example: Two-Way Pipes in gawk
        6. 9.6. Suggested Reading
        7. 9.7. Summary
        8. Exercises
      2. 10. Signals
        1. 10.1. Introduction
        2. 10.2. Signal Actions
        3. 10.3. Standard C Signals: signal() and raise()
          1. 10.3.1. The signal() Function
          2. 10.3.2. Sending Signals Programmatically: raise()
        4. 10.4. Signal Handlers in Action
          1. 10.4.1. Traditional Systems
          2. 10.4.2. BSD and GNU/Linux
          3. 10.4.3. Ignoring Signals
          4. 10.4.4. Restartable System Calls
            1. 10.4.4.1. Example: GNU Coreutils safe_read() and safe_write()
            2. 10.4.4.2. GLIBC Only: TEMP_FAILURE_RETRY()
          5. 10.4.5. Race Conditions and sig_atomic_t (ISO C)
          6. 10.4.6. Additional Caveats
          7. 10.4.7. Our Story So Far, Episode I
        5. 10.5. The System V Release 3 Signal APIs: sigset() et al.
        6. 10.6. POSIX Signals
          1. 10.6.1. Uncovering the Problem
          2. 10.6.2. Signal Sets: Sigset_t and Related Functions
          3. 10.6.3. Managing the Signal Mask: sigprocmask() et al.
          4. 10.6.4. Catching Signals: sigaction()
          5. 10.6.5. Retrieving Pending Signals: sigpending()
          6. 10.6.6. Making Functions Interruptible: siginterrupt()
          7. 10.6.7. Sending Signals: kill() and killpg()
          8. 10.6.8. Our Story So Far, Episode II
        7. 10.7. Signals for Interprocess Communication
        8. 10.8. Important Special-Purpose Signals
          1. 10.8.1. Alarm Clocks: sleep(), alarm(), and SIGALRM
            1. 10.8.1.1. Harder but with More Control: alarm() and SIGALRM
            2. 10.8.1.2. Simple and Easy: sleep()
          2. 10.8.2. Job Control Signals
          3. 10.8.3. Parental Supervision: Three Different Strategies
            1. 10.8.3.1. Poor Parenting: Ignoring Children Completely
            2. 10.8.3.2. Permissive Parenting: Supervising Minimally
            3. 10.8.3.3. Strict Parental Control
        9. 10.9. Signals Across fork() and exec()
        10. 10.10. Summary
        11. Exercises
      3. 11. Permissions and User and Group ID Numbers
        1. 11.1. Checking Permissions
          1. 11.1.1. Real and Effective IDs
          2. 11.1.2. Setuid and Setgid Bits
        2. 11.2. Retrieving User and Group IDs
        3. 11.3. Checking as the Real User: access()
        4. 11.4. Checking as the Effective User: euidaccess() (GLIBC)
        5. 11.5. Setting Extra Permission Bits for Directories
          1. 11.5.1. Default Group for New Files and Directories
          2. 11.5.2. Directories and the Sticky Bit
        6. 11.6. Setting Real and Effective IDs
          1. 11.6.1. Changing the Group Set
          2. 11.6.2. Changing the Real and Effective IDs
          3. 11.6.3. Using the Setuid and Setgid Bits
        7. 11.7. Working with All Three IDs: getresuid() and setresuid() (Linux)
        8. 11.8. Crossing a Security Minefield: Setuid root
        9. 11.9. Suggested Reading
        10. 11.10. Summary
        11. Exercises
      4. 12. General Library Interfaces — Part 2
        1. 12.1. Assertion Statements: assert()
        2. 12.2. Low-Level Memory: The memXXX() Functions
          1. 12.2.1. Setting Memory: memset()
          2. 12.2.2. Copying Memory: memcpy(),memmove(), and memccpy()
          3. 12.2.3. Comparing Memory Blocks: memcmp()
          4. 12.2.4. Searching for a Byte Value: memchr()
        3. 12.3. Temporary Files
          1. 12.3.1. Generating Temporary Filenames (Bad)
          2. 12.3.2. Creating and Opening Temporary Files (Good)
          3. 12.3.3. Using the TMPDIR Environment Variable
        4. 12.4. Committing Suicide: abort()
        5. 12.5. Nonlocal Gotos
          1. 12.5.1. Using Standard Functions: setjmp() and longjmp()
          2. 12.5.2. Handling Signal Masks: sigsetjmp() and siglongjmp()
          3. 12.5.3. Observing Important Caveats
        6. 12.6. Pseudorandom Numbers
          1. 12.6.1. Standard C: rand() and srand()
          2. 12.6.2. POSIX Functions: random() and srandom()
          3. 12.6.3. The /dev/random and /dev/urandom Special Files
        7. 12.7. Metacharacter Expansions
          1. 12.7.1. Simple Pattern Matching: fnmatch()
          2. 12.7.2. Filename Expansion: glob() and globfree()
          3. 12.7.3. Shell Word Expansion: wordexp() and wordfree()
        8. 12.8. Regular Expressions
        9. 12.9. Suggested Reading
        10. 12.10. Summary
        11. Exercises
      5. 13. Internationalization and Localization
        1. 13.1. Introduction
        2. 13.2. Locales and the C Library
          1. 13.2.1. Locale Categories and Environment Variables
          2. 13.2.2. Setting the Locale: setlocale()
          3. 13.2.3. String Collation: strcoll() and strxfrm()
          4. 13.2.4. Low-Level Numeric and Monetary Formatting: localeconv()
          5. 13.2.5. High-Level Numeric and Monetary Formatting: strfmon() and printf()
          6. 13.2.6. Example: Formatting Numeric Values in gawk
          7. 13.2.7. Formatting Date and Time Values: ctime() and strftime()
          8. 13.2.8. Other Locale Information: nl_langinfo()
        3. 13.3. Dynamic Translation of Program Messages
          1. 13.3.1. Setting the Text Domain: textdomain()
          2. 13.3.2. Translating Messages: gettext()
          3. 13.3.3. Working with Plurals: ngettext()
          4. 13.3.4. Making gettext() Easy to Use
            1. 13.3.4.1. Portable Programs: "gettext.h"
            2. 13.3.4.2. GLIBC Only: <libintl.h>
          5. 13.3.5. Rearranging Word Order with printf()
          6. 13.3.6. Testing Translations in a Private Directory
          7. 13.3.7. Preparing Internationalized Programs
          8. 13.3.8. Creating Translations
        4. 13.4. Can You Spell That for Me, Please?
          1. 13.4.1. Wide Characters
          2. 13.4.2. Multibyte Character Encodings
          3. 13.4.3. Languages
          4. 13.4.4. Conclusion
        5. 13.5. Suggested Reading
        6. 13.6. Summary
        7. Exercises
      6. 14. Extended Interfaces
        1. 14.1. Allocating Aligned Memory: posix_memalign() and memalign()
        2. 14.2. Locking Files
          1. 14.2.1. File Locking Concepts
          2. 14.2.2. POSIX Locking: fcntl() and lockf()
            1. 14.2.2.1. Describing a Lock
            2. 14.2.2.2. Obtaining and Releasing Locks
            3. 14.2.2.3. Observing Locking Caveats
          3. 14.2.3. BSD Locking: flock()
          4. 14.2.4. Mandatory Locking
        3. 14.3. More Precise Times
          1. 14.3.1. Microsecond Times: gettimeofday()
          2. 14.3.2. Microsecond File Times: utimes()
          3. 14.3.3. Interval Timers: setitimer() and getitimer()
          4. 14.3.4. More Exact Pauses: nanosleep()
        4. 14.4. Advanced Searching with Binary Trees
          1. 14.4.1. Introduction to Binary Trees
          2. 14.4.2. Tree Management Functions
          3. 14.4.3. Tree Insertion: tsearch()
          4. 14.4.4. Tree Lookup and Use of A Returned Pointer: tfind() and tsearch()
          5. 14.4.5. Tree Traversal: twalk()
          6. 14.4.6. Tree Node Removal and Tree Deletion: tdelete() and tdestroy()
        5. 14.5. Summary
        6. Exercises
    5. III. Debugging and Final Project
      1. 15. Debugging
        1. 15.1. First Things First
        2. 15.2. Compilation for Debugging
        3. 15.3. GDB Basics
          1. 15.3.1. Running GDB
          2. 15.3.2. Setting Breakpoints, Single-Stepping, and Setting Watchpoints
        4. 15.4. Programming for Debugging
          1. 15.4.1. Compile-Time Debugging Code
            1. 15.4.1.1. Use Debugging Macros
            2. 15.4.1.2. Avoid Expression Macros If Possible
            3. 15.4.1.3. Reorder Code If Necessary
            4. 15.4.1.4. Use Debugging Helper Functions
            5. 15.4.1.5. Avoid Unions When Possible
          2. 15.4.2. Runtime Debugging Code
            1. 15.4.2.1. Add Debugging Options and Variables
            2. 15.4.2.2. Use Special Environment Variables
            3. 15.4.2.3. Add Logging Code
            4. 15.4.2.4. Runtime Debugging Files
            5. 15.4.2.5. Add Special Hooks for Breakpoints
        5. 15.5. Debugging Tools
          1. 15.5.1. The dbug Library — A Sophisticated printf()
          2. 15.5.2. Memory Allocation Debuggers
            1. 15.5.2.1. GNU/Linux mtrace
            2. 15.5.2.2. Electric Fence
            3. 15.5.2.3. Debugging Malloc: dmalloc
            4. 15.5.2.4. Valgrind: A Versatile Tool
            5. 15.5.2.5. Other Malloc Debuggers
          3. 15.5.3. A Modern lint
        6. 15.6. Software Testing
        7. 15.7. Debugging Rules
        8. 15.8. Suggested Reading
        9. 15.9. Summary
        10. Exercises
      2. 16. A Project that Ties Everything Together
        1. 16.1. Project Description
        2. 16.2. Suggested Reading
    6. IV. Appendixes
      1. A. Teach Yourself Programming in Ten Years
        1. Why Is Everyone in Such a Rush?
          1. Teach Yourself Programming in Ten Years
        2. References
        3. Answers
        4. Footnotes
      2. B. Caldera Ancient UNIX License
      3. C. GNU General Public License
        1. Preamble
        2. Terms and Conditions for Copying, Distribution and Modification
        3. How to Apply These Terms to Your New Programs
        4. Example Use