Advanced Programming in the UNIX® Environment: Second Edition

Advanced Programming in the UNIX® Environment: Second Edition

Released June 2005
Publisher(s): Addison-Wesley Professional
ISBN: 9780201433074

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

"Stephen Rago's update is a long overdue benefit to the community of professionals using the versatile family of UNIX and UNIX-like operating environments. It removes obsolescence and includes newer developments. It also thoroughly updates the context of all topics, examples, and applications to recent releases of popular implementations of UNIX and UNIX-like environments. And yet, it does all this while retaining the style and taste of the original classic."

--Mukesh Kacker, cofounder and former CTO of Pronto Networks, Inc.

"One of the essential classics of UNIX programming."

--Eric S. Raymond, author of The Art of UNIX Programming

"This is the definitive reference book for any serious or professional UNIX systems programmer. Rago has updated and extended the classic Stevens text while keeping true to the original. The APIs are illuminated by clear examples of their use. He also mentions many of the pitfalls to look out for when programming across different UNIX system implementations and points out how to avoid these pitfalls using relevant standards such as POSIX 1003.1, 2004 edition and the Single UNIX Specification, Version 3."

--Andrew Josey, Director, Certification, The Open Group, and Chair of the POSIX 1003.1 Working Group

"Advanced Programming in the UNIX® Environment, Second Edition, is an essential reference for anyone writing programs for a UNIX system. It's the first book I turn to when I want to understand or re-learn any of the various system interfaces. Stephen Rago has successfully revised this book to incorporate newer operating systems such as GNU/Linux and Apple's OS X while keeping true to the first edition in terms of both readability and usefulness. It will always have a place right next to my computer."

--Dr. Benjamin Kuperman, Swarthmore College

Praise for the First Edition

"Advanced Programming in the UNIX® Environment is a must-have for any serious C programmer who works under UNIX. Its depth, thoroughness, and clarity of explana-tion are unmatched."

--UniForum Monthly

"Numerous readers recommended Advanced Programming in the UNIX® Environment by W. Richard Stevens (Addison-Wesley), and I'm glad they did; I hadn't even heard of this book, and it's been out since 1992. I just got my hands on a copy, and the first few chapters have been fascinating."

--Open Systems Today

"A much more readable and detailed treatment of UNIX internals can be found in Advanced Programming in the UNIX® Environment by W. Richard Stevens (Addison-Wesley). This book includes lots of realistic examples, and I find it quite helpful when I have systems programming tasks to do."

--RS/Magazine

"This is the definitive reference book for any serious or professional UNIX systems programmer. Rago has updated and extended the original Stevens classic while keeping true to the original."

--Andrew Josey, Director, Certification, The Open Group, and Chair of the POSIX 1003.1 Working Group

For over a decade, serious C programmers have relied on one book for practical, in-depth knowledge of the programming interfaces that drive the UNIX and Linux kernels: W. Richard Stevens' Advanced Programming in the UNIX® Environment. Now, Stevens' colleague Stephen Rago has thoroughly updated this classic to reflect the latest technical advances and add support for today's leading UNIX and Linux platforms.

Rago carefully retains the spirit and approach that made this book a classic. Building on Stevens' work, he begins with basic topics such as files, directories, and processes, carefully laying the groundwork for understanding more advanced techniques, such as signal handling and terminal I/O.

Substantial new material includes chapters on threads and multithreaded programming, using the socket interface to drive interprocess communication (IPC), and extensive coverage of the interfaces added to the latest version of the POSIX.1 standard. Nearly all examples have been tested on four of today's most widely used UNIX/Linux platforms: FreeBSD 5.2.1; the Linux 2.4.22 kernel; Solaris 9; and Darwin 7.4.0, the FreeBSD/Mach hybrid underlying Apple's Mac OS X 10.3.

As in the first edition, you'll learn through example, including more than 10,000 lines of downloadable, ANSI C source code. More than 400 system calls and functions are demonstrated with concise, complete programs that clearly illustrate their usage, arguments, and return values. To tie together what you've learned, the book presents several chapter-length case studies, each fully updated for contemporary environments.

Advanced Programming in the UNIX® Environment has helped a generation of programmers write code with exceptional power, performance, and reliability. Now updated for today's UNIX/Linux systems, this second edition will be even more indispensable.

Table of contents

  1. Copyright
    1. Dedication
  2. Praise for Advanced Programming in the UNIX® Environment, Second Edition
  3. Praise for the First Edition
  4. Addison-Wesley Professional Computing Series
  5. Foreword
  6. Preface
    1. Introduction
    2. Changes from the First Edition
    3. Acknowledgments
  7. Preface to the First Edition
    1. Introduction
    2. Unix Standards
    3. Organization of the Book
    4. Examples in the Text
    5. Systems Used to Test the Examples
    6. Acknowledgments
  8. 1. UNIX System Overview
    1. 1.1. Introduction
    2. 1.2. UNIX Architecture
    3. 1.3. Logging In
      1. Login Name
      2. Shells
    4. 1.4. Files and Directories
      1. File System
      2. Filename
      3. Pathname
      4. Working Directory
      5. Home Directory
    5. 1.5. Input and Output
      1. File Descriptors
      2. Standard Input, Standard Output, and Standard Error
      3. Unbuffered I/O
      4. Standard I/O
    6. 1.6. Programs and Processes
      1. Program
      2. Processes and Process ID
      3. Process Control
      4. Threads and Thread IDs
    7. 1.7. Error Handling
      1. Error Recovery
    8. 1.8. User Identification
      1. User ID
      2. Group ID
      3. Supplementary Group IDs
    9. 1.9. Signals
    10. 1.10. Time Values
    11. 1.11. System Calls and Library Functions
    12. 1.12. Summary
    13. Exercises
  9. 2. UNIX Standardization and Implementations
    1. 2.1. Introduction
    2. 2.2. UNIX Standardization
      1. 2.2.1. ISO C
      2. 2.2.2. IEEE POSIX
      3. 2.2.3. The Single UNIX Specification
      4. 2.2.4. FIPS
    3. 2.3. UNIX System Implementations
      1. 2.3.1. UNIX System V Release 4
      2. 2.3.2. 4.4BSD
      3. 2.3.3. FreeBSD
      4. 2.3.4. Linux
      5. 2.3.5. Mac OS X
      6. 2.3.6. Solaris
      7. 2.3.7. Other UNIX Systems
    4. 2.4. Relationship of Standards and Implementations
    5. 2.5. Limits
      1. 2.5.1. ISO C Limits
      2. 2.5.2. POSIX Limits
      3. 2.5.3. XSI Limits
      4. 2.5.4. sysconf, pathconf, and fpathconf Functions
      5. 2.5.5. Indeterminate Runtime Limits
        1. Pathnames
        2. Maximum Number of Open Files
    6. 2.6. Options
    7. 2.7. Feature Test Macros
    8. 2.8. Primitive System Data Types
    9. 2.9. Conflicts Between Standards
    10. 2.10. Summary
    11. Exercises
  10. 3. File I/O
    1. 3.1. Introduction
    2. 3.2. File Descriptors
    3. 3.3. open Function
      1. Filename and Pathname Truncation
    4. 3.4. creat Function
    5. 3.5. close Function
    6. 3.6. lseek Function
    7. 3.7. read Function
    8. 3.8. write Function
    9. 3.9. I/O Efficiency
    10. 3.10. File Sharing
    11. 3.11. Atomic Operations
      1. Appending to a File
      2. pread and pwrite Functions
      3. Creating a File
    12. 3.12. dup and dup2 Functions
    13. 3.13. sync, fsync, and fdatasync Functions
    14. 3.14. fcntl Function
    15. 3.15. ioctl Function
    16. 3.16. /dev/fd
    17. 3.17. Summary
    18. Exercises
  11. 4. Files and Directories
    1. 4.1. Introduction
    2. 4.2. stat, fstat, and lstat Functions
    3. 4.3. File Types
    4. 4.4. Set-User-ID and Set-Group-ID
    5. 4.5. File Access Permissions
    6. 4.6. Ownership of New Files and Directories
    7. 4.7. access Function
    8. 4.8. umask Function
    9. 4.9. chmod and fchmod Functions
    10. 4.10. Sticky Bit
    11. 4.11. chown, fchown, and lchown Functions
    12. 4.12. File Size
      1. Holes in a File
    13. 4.13. File Truncation
    14. 4.14. File Systems
    15. 4.15. link, unlink, remove, and rename Functions
    16. 4.16. Symbolic Links
    17. 4.17. symlink and readlink Functions
    18. 4.18. File Times
    19. 4.19. utime Function
    20. 4.20. mkdir and rmdir Functions
    21. 4.21. Reading Directories
    22. 4.22. chdir, fchdir, and getcwd Functions
    23. 4.23. Device Special Files
    24. 4.24. Summary of File Access Permission Bits
    25. 4.25. Summary
    26. Exercises
  12. 5. Standard I/O Library
    1. 5.1. Introduction
    2. 5.2. Streams and FILE Objects
    3. 5.3. Standard Input, Standard Output, and Standard Error
    4. 5.4. Buffering
    5. 5.5. Opening a Stream
    6. 5.6. Reading and Writing a Stream
      1. Input Functions
      2. Output Functions
    7. 5.7. Line-at-a-Time I/O
    8. 5.8. Standard I/O Efficiency
    9. 5.9. Binary I/O
    10. 5.10. Positioning a Stream
    11. 5.11. Formatted I/O
      1. Formatted Output
      2. Formatted Input
    12. 5.12. Implementation Details
    13. 5.13. Temporary Files
    14. 5.14. Alternatives to Standard I/O
    15. 5.15. Summary
    16. Exercises
  13. 6. System Data Files and Information
    1. 6.1. Introduction
    2. 6.2. Password File
    3. 6.3. Shadow Passwords
    4. 6.4. Group File
    5. 6.5. Supplementary Group IDs
    6. 6.6. Implementation Differences
    7. 6.7. Other Data Files
    8. 6.8. Login Accounting
    9. 6.9. System Identification
    10. 6.10. Time and Date Routines
    11. 6.11. Summary
    12. Exercises
  14. 7. Process Environment
    1. 7.1. Introduction
    2. 7.2. main Function
    3. 7.3. Process Termination
      1. Exit Functions
      2. atexit Function
    4. 7.4. Command-Line Arguments
    5. 7.5. Environment List
    6. 7.6. Memory Layout of a C Program
    7. 7.7. Shared Libraries
    8. 7.8. Memory Allocation
      1. Alternate Memory Allocators
      2. libmalloc
      3. vmalloc
      4. quick-fit
      5. alloca Function
    9. 7.9. Environment Variables
    10. 7.10. setjmp and longjmp Functions
      1. Automatic, Register, and Volatile Variables
      2. Potential Problem with Automatic Variables
    11. 7.11. getrlimit and setrlimit Functions
    12. 7.12. Summary
    13. Exercises
  15. 8. Process Control
    1. 8.1. Introduction
    2. 8.2. Process Identifiers
    3. 8.3. fork Function
      1. File Sharing
    4. 8.4. vfork Function
    5. 8.5. exit Functions
    6. 8.6. wait and waitpid Functions
    7. 8.7. waitid Function
    8. 8.8. wait3 and wait4 Functions
    9. 8.9. Race Conditions
    10. 8.10. exec Functions
    11. 8.11. Changing User IDs and Group IDs
      1. setreuid and setregid Functions
      2. seteuid and setegid Functions
      3. Group IDs
    12. 8.12. Interpreter Files
    13. 8.13. system Function
      1. Set-User-ID Programs
    14. 8.14. Process Accounting
    15. 8.15. User Identification
    16. 8.16. Process Times
    17. 8.17. Summary
    18. Exercises
  16. 9. Process Relationships
    1. 9.1. Introduction
    2. 9.2. Terminal Logins
      1. BSD Terminal Logins
      2. Mac OS X Terminal Logins
      3. Linux Terminal Logins
      4. Solaris Terminal Logins
    3. 9.3. Network Logins
      1. BSD Network Logins
      2. Mac OS X Network Logins
      3. Linux Network Logins
      4. Solaris Network Logins
    4. 9.4. Process Groups
    5. 9.5. Sessions
    6. 9.6. Controlling Terminal
    7. 9.7. tcgetpgrp, tcsetpgrp, and tcgetsid Functions
    8. 9.8. Job Control
    9. 9.9. Shell Execution of Programs
    10. 9.10. Orphaned Process Groups
    11. 9.11. FreeBSD Implementation
    12. 9.12. Summary
    13. Exercises
  17. 10. Signals
    1. 10.1. Introduction
    2. 10.2. Signal Concepts
    3. 10.3. signal Function
      1. Program Start-Up
      2. Process Creation
    4. 10.4. Unreliable Signals
    5. 10.5. Interrupted System Calls
    6. 10.6. Reentrant Functions
    7. 10.7. SIGCLD Semantics
    8. 10.8. Reliable-Signal Terminology and Semantics
    9. 10.9. kill and raise Functions
    10. 10.10. alarm and pause Functions
    11. 10.11. Signal Sets
      1. Implementation
    12. 10.12. sigprocmask Function
    13. 10.13. sigpending Function
    14. 10.14. sigaction Function
    15. 10.15. sigsetjmp and siglongjmp Functions
    16. 10.16. sigsuspend Function
    17. 10.17. abort Function
    18. 10.18. system Function
      1. Return Value from system
    19. 10.19. sleep Function
    20. 10.20. Job-Control Signals
    21. 10.21. Additional Features
      1. Signal Names
      2. Signal Mappings
    22. 10.22. Summary
    23. Exercises
  18. 11. Threads
    1. 11.1. Introduction
    2. 11.2. Thread Concepts
    3. 11.3. Thread Identification
    4. 11.4. Thread Creation
    5. 11.5. Thread Termination
    6. 11.6. Thread Synchronization
      1. Mutexes
      2. Deadlock Avoidance
      3. Reader–Writer Locks
      4. Condition Variables
    7. 11.7. Summary
    8. Exercises
  19. 12. Thread Control
    1. 12.1. Introduction
    2. 12.2. Thread Limits
    3. 12.3. Thread Attributes
      1. More Thread Attributes
    4. 12.4. Synchronization Attributes
      1. Mutex Attributes
      2. Reader–Writer Lock Attributes
      3. Condition Variable Attributes
    5. 12.5. Reentrancy
    6. 12.6. Thread-Specific Data
    7. 12.7. Cancel Options
    8. 12.8. Threads and Signals
    9. 12.9. Threads and fork
    10. 12.10. Threads and I/O
    11. 12.11. Summary
    12. Exercises
  20. 13. Daemon Processes
    1. 13.1. Introduction
    2. 13.2. Daemon Characteristics
    3. 13.3. Coding Rules
    4. 13.4. Error Logging
    5. 13.5. Single-Instance Daemons
    6. 13.6. Daemon Conventions
    7. 13.7. Client–Server Model
    8. 13.8. Summary
    9. Exercises
  21. 14. Advanced I/O
    1. 14.1. Introduction
    2. 14.2. Nonblocking I/O
    3. 14.3. Record Locking
      1. History
      2. fcntl Record Locking
      3. Implied Inheritance and Release of Locks
      4. FreeBSD Implementation
      5. Locks at End of File
      6. Advisory versus Mandatory Locking
    4. 14.4. STREAMS
      1. STREAMS Messages
      2. putmsg and putpmsg Functions
      3. STREAMS ioctl Operations
      4. write to STREAMS Devices
      5. Write Mode
      6. getmsg and getpmsg Functions
      7. Read Mode
    5. 14.5. I/O Multiplexing
      1. 14.5.1. select and pselect Functions
      2. 14.5.2. poll Function
      3. Interruptibility of select and poll
    6. 14.6. Asynchronous I/O
      1. 14.6.1. System V Asynchronous I/O
      2. 14.6.2. BSD Asynchronous I/O
    7. 14.7. readv and writev Functions
    8. 14.8. readn and writen Functions
    9. 14.9. Memory-Mapped I/O
    10. 14.10. Summary
    11. Exercises
  22. 15. Interprocess Communication
    1. 15.1. Introduction
    2. 15.2. Pipes
    3. 15.3. popen and pclose Functions
    4. 15.4. Coprocesses
    5. 15.5. FIFOs
    6. 15.6. XSI IPC
      1. 15.6.1. Identifiers and Keys
      2. 15.6.2. Permission Structure
      3. 15.6.3. Configuration Limits
      4. 15.6.4. Advantages and Disadvantages
    7. 15.7. Message Queues
    8. 15.8. Semaphores
      1. Semaphore Adjustment on exit
    9. 15.9. Shared Memory
    10. 15.10. Client–Server Properties
    11. 15.11. Summary
    12. Exercises
  23. 16. Network IPC: Sockets
    1. 16.1. Introduction
    2. 16.2. Socket Descriptors
    3. 16.3. Addressing
      1. 16.3.1. Byte Ordering
      2. 16.3.2. Address Formats
      3. 16.3.3. Address Lookup
      4. 16.3.4. Associating Addresses with Sockets
    4. 16.4. Connection Establishment
    5. 16.5. Data Transfer
    6. 16.6. Socket Options
    7. 16.7. Out-of-Band Data
    8. 16.8. Nonblocking and Asynchronous I/O
    9. 16.9. Summary
    10. Exercises
  24. 17. Advanced IPC
    1. 17.1. Introduction
    2. 17.2. STREAMS-Based Pipes
      1. 17.2.1. Naming STREAMS Pipes
      2. 17.2.2. Unique Connections
    3. 17.3. UNIX Domain Sockets
      1. 17.3.1. Naming UNIX Domain Sockets
      2. 17.3.2. Unique Connections
    4. 17.4. Passing File Descriptors
      1. 17.4.1. Passing File Descriptors over STREAMS-Based Pipes
      2. 17.4.2. Passing File Descriptors over UNIX Domain Sockets
    5. 17.5. An Open Server, Version 1
    6. 17.6. An Open Server, Version 2
    7. 17.7. Summary
    8. Exercises
  25. 18. Terminal I/O
    1. 18.1. Introduction
    2. 18.2. Overview
    3. 18.3. Special Input Characters
    4. 18.4. Getting and Setting Terminal Attributes
    5. 18.5. Terminal Option Flags
    6. 18.6. stty Command
    7. 18.7. Baud Rate Functions
    8. 18.8. Line Control Functions
    9. 18.9. Terminal Identification
    10. 18.10. Canonical Mode
    11. 18.11. Noncanonical Mode
    12. 18.12. Terminal Window Size
    13. 18.13. termcap, terminfo, and curses
    14. 18.14. Summary
    15. Exercises
  26. 19. Pseudo Terminals
    1. 19.1. Introduction
    2. 19.2. Overview
      1. Network Login Servers
      2. script Program
      3. expect Program
      4. Running Coprocesses
      5. Watching the Output of Long-Running Programs
    3. 19.3. Opening Pseudo-Terminal Devices
      1. 19.3.1. STREAMS-Based Pseudo Terminals
      2. 19.3.2. BSD-Based Pseudo Terminals
      3. 19.3.3. Linux-Based Pseudo Terminals
    4. 19.4. pty_fork Function
    5. 19.5. pty Program
    6. 19.6. Using the pty Program
      1. utmp File
      2. Job Control Interaction
      3. Watching the Output of Long-Running Programs
      4. script Program
      5. Running Coprocesses
      6. Driving Interactive Programs Noninteractively
    7. 19.7. Advanced Features
      1. Packet Mode
      2. Remote Mode
      3. Window Size Changes
      4. Signal Generation
    8. 19.8. Summary
    9. Exercises
  27. 20. A Database Library
    1. 20.1. Introduction
    2. 20.2. History
    3. 20.3. The Library
    4. 20.4. Implementation Overview
    5. 20.5. Centralized or Decentralized?
    6. 20.6. Concurrency
      1. Coarse-Grained Locking
      2. Fine-Grained Locking
    7. 20.7. Building the Library
    8. 20.8. Source Code
    9. 20.9. Performance
      1. Single-Process Results
      2. Multiple-Process Results
    10. 20.10. Summary
    11. Exercises
  28. 21. Communicating with a Network Printer
    1. 21.1. Introduction
    2. 21.2. The Internet Printing Protocol
    3. 21.3. The Hypertext Transfer Protocol
    4. 21.4. Printer Spooling
      1. Security
    5. 21.5. Source Code
    6. 21.6. Summary
    7. Exercises
  29. A. Function Prototypes
  30. B. Miscellaneous Source Code
    1. B.1. Our Header File
    2. B.2. Standard Error Routines
  31. C. Solutions to Selected Exercises
    1. Chapter 1
    2. Chapter 2
    3. Chapter 3
    4. Chapter 4
    5. Chapter 5
    6. Chapter 6
    7. Chapter 7
    8. Chapter 8
    9. Chapter 9
    10. Chapter 10
    11. Chapter 11
    12. Chapter 12
    13. Chapter 13
    14. Chapter 14
    15. Chapter 15
    16. Chapter 16
    17. Chapter 17
    18. Chapter 18
    19. Chapter 19
    20. Chapter 20
    21. Chapter 21
  32. Bibliography

Product information

  • Title: Advanced Programming in the UNIX® Environment: Second Edition
  • Author(s):
  • Release date: June 2005
  • Publisher(s): Addison-Wesley Professional
  • ISBN: 9780201433074