Cover image for Exim: The Mail Transfer Agent

Book description

Exim delivers electronic mail, both local and remote. It has all the virtues of a good postman: it's easy to talk to, reliable, efficient, and eager to accommodate even the most complex special requests. It's the default mail transport agent installed on some Linux systems, runs on many versions of Unix, and is suitable for any TCP/IP network with any combination of hosts and end-user mail software. Exim is growing in popularity because it is open source, scalable, and rich in features such as the following:

  • Compatibility with the calling interfaces and options of Sendmail (for which Exim is usually a drop-in replacement)

  • Lookups in LDAP servers, MySQL and PostgreSQL databases, and NIS or NIS+ services

  • Support for many kinds of address parsing, including regular expressions that are compatible with Perl 5

  • Sophisticated error handling

  • Innumerable tuning parameters for improving performance and handling enormous volumes of mail

Best of all, Exim is easy to configure. You never have to deal with ruleset 3 or worry that a misplaced asterisk will cause an inadvertent mail bomb. While a basic configuration is easy to read and can be created quickly, Exim's syntax and behavior do get more subtle as you enter complicated areas like virtual hosting, filtering, and automatic replies. This book is a comprehensive survey that provides quick information for people in a hurry as well as thorough coverage of more advanced material.

Table of Contents

  1. Special Upgrade Offer
  2. Preface
    1. Organization of the Book
    2. Conventions Used in This Book
    3. Comments and Questions
    4. Acknowledgments
  3. 1. Introduction
  4. 2. How Internet Mail Works
    1. Different Types of MTA
    2. Internet Message Standards
    3. RFC 822 Message Format
    4. The Message “On the Wire”
    5. Summary of the SMTP Protocol
    6. Forgery
    7. Authentication and Encryption
    8. Routing a Message
    9. Checking Incoming Mail
    10. Overview of the DNS
    11. DNS Records Used for Mail Routing
    12. Related DNS Records
    13. Common DNS Errors
    14. Role of the Postmaster
  5. 3. Exim Overview
    1. Exim Philosophy
    2. Exim’s Queue
    3. Receiving and Delivering Messages
    4. Exim Processes
    5. Coordination Between Processes
    6. How Exim Is Configured
    7. How Exim Delivers Messages
    8. Local and Remote Addresses
    9. Processing an Address
    10. A Simple Example
    11. Complications While Directing and Routing
      1. Duplicate Addresses
      2. Missing Data
      3. Directing Loops
      4. Remote Address Becoming Local
      5. Remote Address Routing to the Local Host
    12. Complications During Delivery
    13. Complications After Delivery
    14. Use of Transports by Directors and Routers
  6. 4. Exim Operations Overview
    1. How Exim Identifies Messages
    2. Watching Exim at Work
    3. The Runtime Configuration File
      1. Layout of the Configuration File
      2. A Minimal Usable Configuration File
      3. Option Setting Syntax
      4. Macros in the Configuration File
      5. Hiding Configuration Data
      6. String Expansions
      7. File and Database Lookups
      8. Domain, Host, and Address Lists
    4. The Default Qualification Domain
    5. Handling Frozen Bounce Messages
    6. Reducing Activity at High Load
      1. Delaying or Suspending Delivery When the Load Is High
      2. Suspending Incoming Mail When the Load Is High
      3. Controlling the Number of Incoming SMTP Connections
      4. Checking for Free Disk Space
    7. Limiting Message Sizes
    8. Parallel Remote Delivery
    9. Controlling the Number of Delivery Processes
    10. Large Message Queues
    11. Large Installations
      1. Linear Password Files
      2. Mailbox Directories
      3. Simultaneous Message Deliveries
      4. Minimizing Name Server Delays
      5. Storing Messages for Dial-up Hosts
      6. Hardware Configuration
  7. 5. Extending the Delivery Configuration
    1. Multiple Local Domains
      1. Differentiating Between Multiple Domains
    2. Virtual Domains
      1. Defaults in Virtual Domains
      2. Postmasters in Virtual Domains
    3. Mailing Lists
      1. Syntax Errors in Mailing Lists
      2. NFS-Mounted Mailing Lists
      3. Reexpansion of Mailing Lists
      4. Closed Mailing Lists
      5. External Mailing List Software
    4. Using an External Local Delivery Agent
    5. Multiple User Addresses
    6. Mixed Local/Remote Domains
    7. Delivering to UUCP
    8. Ignoring the Local Part in Local Deliveries
    9. Handling Local Parts in a Case-Sensitive Manner
    10. Scanning Messages for Viruses
      1. Virus Checking on the Local Host
      2. Virus Checking on an External Host
    11. Modifying Message Bodies
  8. 6. Options Common to Directors and Routers
    1. Conditional Running of Routers and Directors
      1. Restricting Drivers to Specific Domains
      2. Stopping an Address from Being Passed On
      3. Restricting Drivers to Specific Local Parts
      4. Restricting Drivers to Specific Senders
      5. Restricting Drivers by Other Conditions
      6. Restricting a Driver to Verification Only
      7. Restricting Drivers by File Existence
      8. Interaction of Conditions
    2. Changing a Driver’s Successful Outcome
    3. Adding Data for Use by Transports
      1. Adding or Removing Header Lines
      2. Changing the Return Path
      3. Controlling the Environment for Local Deliveries
      4. Specifying fallback_hosts
    4. Debugging Directors and Routers
    5. Summary of Director/Router Generic Options
  9. 7. The Directors
    1. Conditional Running of Directors
      1. Local Part Prefixes and Suffixes
      2. Control of EXPN
    2. Optimizing Single-Level Aliasing
    3. Adding Data for Use by Transports
    4. The aliasfile and forwardfile Directors
      1. Items in Alias and Forward Lists
        1. Duplicate addresses
        2. Including the incoming address in a list
        3. A bad interaction between aliases and forwarding
        4. Nonaddress alias and forward items
        5. Nonaddress alias-only items
      2. Options Common to aliasfile and forwardfile
        1. Checks on file attributes
        2. Ancestor checking
        3. Transports for pipes and files
        4. Disabling pipes and files
        5. Unqualified addresses
        6. Rewriting generated addresses
        7. One-time aliasing and forwarding
        8. Missing include files
        9. Syntax errors in alias or forward lists
        10. Telling users about broken .forward files
      3. Summary of Options Common to aliasfile and forwardfile
    5. The aliasfile Director
      1. Specifying the Lookup
      2. Expanding a List of Aliases
      3. Specifying a Transport for aliasfile
      4. Summary of aliasfile Options
    6. The forwardfile Director
      1. Contents of forwardfile Lists
      2. Specifying a .forward File
      3. Specifying an Inline Forwarding List
      4. Checking Local Users
      5. Special Error Handling
      6. Disabling Certain Features
      7. Enabling Certain System Actions
      8. The $home Variable
      9. Current and Home Directories
      10. Summary of forwardfile Options
    7. The localuser Director
      1. Transports for localuser
      2. Checking the Home Directory
    8. The smartuser Director
      1. Using smartuser to Generate New Addresses
      2. Using smartuser to Direct to a Transport
      3. Summary of smartuser Options
  10. 8. The Routers
    1. Timeouts While Routing
    2. Domains That Route to the Local Host
      1. Treating Domains Routed to self as Local
      2. Passing Domains Routed to self to the Next Router
      3. Rerouting Domains Routed to self
      4. Failing Domains Routed to Self
      5. Transporting Domains Routed to self
      6. Deferring Domains Routed to self
    3. The lookuphost Router
      1. Controlling DNS lookups
      2. Conditions for MX Records
      3. Using the System’s Host lookup Instead of the DNS
      4. Explicit lookup Widening
      5. Header Rewriting
      6. Summary of lookuphost Options
    4. The domainlist Router
      1. Inline Routing Rules
      2. Looked-up Routing Rules
      3. Preprocessing the Host List
      4. Routing to a Local Transport
      5. Using domainlist on a Mail Hub
      6. Varying the Transport
      7. Using domainlist to Change the Routing Domain
      8. Other domainlist Options
      9. Summary of domainlist Options
    5. The ipliteral Router
    6. The queryprogram Router
      1. The queryprogram Command
      2. Running the queryprogram Command
      3. The Result of the queryprogram Command
      4. Summary of queryprogram Options
  11. 9. The Transports
    1. Options Common to All Transports
      1. Debugging Transports
      2. Transporting Only Part of a Message
      3. Controlling Message Size
      4. Adding and Removing Header Lines
      5. Rewriting Addresses in Header Lines
      6. Changing the Return Path
      7. Transport Filters
      8. Shadow Transports
      9. Summary of Generic Transport Options
    2. The smtp Transport
      1. Control of Multiple Addresses
      2. Control of Outgoing Calls
      3. Control of the TCP/IP Connection
      4. Use of the SIZE Option in SMTP
      5. Use of the AUTH Command in SMTP
      6. Use of the LMTP Protocol
      7. Specifying Hosts
        1. Specifying a primary host list
        2. Overriding a router’s host list
        3. Randomizing a host list
        4. Specifying a fallback host list
        5. Looking up IP addresses
        6. Handling the local host
      8. Control of Retrying
      9. Summary of smtp Options
    3. Environment for Local Transports
      1. Uids and Gids
      2. Current and Home Directories
      3. Expansion Variables Derived from the Address
    4. Options Common to the appendfile and pipe Transports
      1. Controlling the Delivery Environment
      2. Controlling the Format of the Message
        1. Separating messages in a single file
        2. Escaping lines in the message
        3. Control of line terminators
      3. Batched Delivery and BSMTP
        1. A non-BSMTP batching example
        2. A BSMTP batching example
        3. Use of multiple files for batched messages
      4. Control of Retrying
      5. Summary of Options Common to appendfile and pipe
    5. The appendfile Transport
      1. Setting Up a Multimessage File for Appending
        1. Mailbox location
        2. Symbolic links for mailbox files
        3. Delivering to named pipes (FIFOs)
        4. Creating a nonexistent file
        5. The owner of an existing file
        6. The mode of the file
      2. Format of Appended Messages
        1. MBX format mailboxes
        2. Checking an existing file’s format
      3. Locking a File for Appending
        1. Locking using a lock file
        2. Using a locking function
        3. Why use lock files?
        4. Locking options for non-MBX mailboxes
        5. Locking options for MBX mailboxes
      4. Delivering Each Message into a Separate File
      5. Maildir Format
      6. Mailbox Quotas
      7. Inclusive and Exclusive Quotas
      8. Quota Warnings
      9. Notifying comsat
      10. Summary of appendfile Options
    6. The pipe Transport
      1. Defining the Command to Run
      2. The Uid and Gid for the Command
      3. Running the Command
        1. Parsing the command line
        2. Using a shell
      4. The Command Environment
      5. Timing the Command
      6. Restricting Which Commands Can Be Run
      7. Handling Command Errors
      8. Handling Output from the Command
      9. Summary of pipe Options
    7. The lmtp Transport
    8. The autoreply Transport
      1. The Parameters of the Message
      2. Once-Only Messages
      3. Keeping a Log of Messages Sent
      4. Summary of autoreply Options
  12. 10. Message Filtering
    1. Examples of Filter Commands
    2. Filtering Compared with an External Delivery Agent
    3. Setting Up a User Filter
    4. Setting Up a System Filter
      1. Summary of System Filter Options
    5. Testing Filter Files
      1. Testing a System Filter File
      2. Testing an Installed Filter File
    6. Format of Filter Files
    7. Significant Actions
    8. Filter Commands
    9. The add Command
    10. Delivery Commands
      1. The deliver Command
      2. The save Command
      3. The pipe Command
      4. Ignoring Delivery Errors
    11. Mail Commands
    12. Logging Commands
    13. The testprint Command
    14. The finish Command
    15. Obeying Filter Commands Conditionally
      1. String Testing Conditions
      2. Numeric Testing Conditions
      3. Testing for Personal Mail
      4. Testing for Significant Actions
      5. Testing for Error Messages
      6. Testing Delivery Status
      7. Testing a List of Addresses
    16. Additional Features for System Filters
      1. The fail Command
      2. The freeze Command
      3. The headers add Command
      4. The headers remove Command
  13. 11. Shared Data and Exim Processes
    1. Message Files
    2. Locking Message Files
    3. Hints Files
    4. Log Files
    5. User and Group IDs for Exim Processes
    6. Process Relationships
    7. The Daemon Process
      1. Summary of Options for the Daemon
    8. Reception Processes
    9. Queue Runner Processes
      1. Special Kinds of Queue Run
    10. Delivery Processes
      1. Variations on Delivery
    11. Summary of Message Handling Process Types
    12. Other Types of Process
  14. 12. Delivery Errors and Retrying
    1. Retrying After Errors
    2. Remote Delivery Errors
      1. Host Errors
      2. Message Errors
      3. Recipient Errors
      4. Problems of Error Classification
      5. Delivery to Multiple Hosts
    3. Local Delivery Errors
    4. Routing and Directing Errors
    5. Retry Rules
      1. Retry Rule Patterns
      2. Retry Rule Error Names
      3. Retry Rule Parameter Sets
    6. Computing Retry Times
    7. Using Retry Times
    8. Retry Rule Examples
    9. Timeout of Retry Data
    10. Long-Term Failures
    11. Ultimate Address Timeout
    12. Intermittently Connected Hosts
      1. Incoming Mail for an Intermittently Connected Host
      2. Exim on an Intermittently Connected Host
        1. The daemon on an intermittently connected host
        2. Incoming mail on an intermittently connected host
  15. 13. Message Reception and Policy Controls
    1. Message Sources
    2. Message Size Control
    3. Messages from Local Processes
      1. Addresses in Header Lines
      2. Specifying Recipient Addresses
      3. Local Sender Addresses
    4. Unqualified Addresses from Remote Hosts
    5. Checking a Remote Host
      1. Verifying a Host’s Name
      2. Verifying EHLO or HELO
      3. Using a DNS Blocking List
        1. Configuring Exim to use an RBL
        2. RBL warnings
        3. RBL rejection
        4. Using RBL data values
        5. Remaining RBL options
      4. Explicit Host Blocking
    6. Checking Remote Sender Addresses
      1. Verifying SMTP Sender Addresses
        1. Verify callbacks
        2. Configuring routers and directors for verification
      2. Exceptions to Sender Verification
      3. Temporary Sender Verification Failures
      4. Permanent Sender Verification Failures
      5. Fixing Bad Envelope Senders
      6. Testing Sender Verification
      7. Checking Senders in Header Lines
      8. Explicitly Rejecting Senders
      9. Summary of Sender Checking Options
    7. Checking Recipient Addresses
      1. Verifying Recipient Addresses
      2. Conditional Recipient Verification
      3. Testing Recipient Verification
      4. Explicitly Rejecting Recipients
      5. Summary of Recipient Rejection Options
    8. Checking Header Line Syntax
    9. Relay Control
      1. Incoming and Outgoing Relaying
      2. Relay Checking
        1. Local parts containing % or @
        2. Incomplete domains
      3. Incoming Relaying
        1. Automatic relaying for MX backups
      4. Outgoing Relaying
        1. Relaying from authenticated hosts
        2. Relaying using encryption
        3. Relaying from specific senders
        4. Permitting relaying by host or sender
      5. Summary of Relay Control Options
    10. Customizing Prohibition Messages
    11. Incoming Message Processing
      1. The UUCP “From” Line
      2. The From: Header Line
      3. The Sender: Header Line
      4. The Bcc:, Cc:, and To: Header Lines
      5. The Return-path:, Envelope-to:, and Delivery-date: Header Lines
      6. The Date: Header Line
      7. The Message-id: Header Line
      8. The Received: Header Line
  16. 14. Rewriting Addresses
    1. Automatic Rewriting
    2. Configured Rewriting
      1. General Rewriting
      2. Per-Transport Rewriting
    3. Rewriting Rules
      1. Format of Rewriting Rules
      2. Applying Rewriting Rules
      3. Conditional Rewriting
      4. Lookup-Driven Rewriting
    4. Rewriting Patterns
    5. Rewriting Flags
      1. Flags Specifying What to Rewrite
      2. Flags Controlling the Rewriting Process
      3. The SMTP-Time Rewriting Flag
    6. A Further Rewriting Example
    7. Testing Rewriting Rules
  17. 15. Authentication, Encryption, and Other SMTP Processing
    1. SMTP Authentication
      1. Authentication Mechanisms
      2. PLAIN Authentication
      3. LOGIN Authentication
      4. CRAM-MD5 Authentication
      5. Choice of Authentication Mechanism
      6. Exim Authenticators
      7. Authentication on an Exim Server
      8. Advertising Authentication
      9. Testing Server Authentication
      10. Authenticated Senders
      11. Authentication by an Exim Client
      12. Options Common to All Authenticators
      13. Using the plaintext Authenticator in a Server
      14. Using plaintext in a Client
      15. Using cram_md5 in a Server
      16. Using cram_md5 in a Client
    2. Encrypted SMTP Connections
      1. Configuring Exim to Use TLS as a Server
      2. Setting Conditions on TLS Connections
      3. Forcing Clients to Use TLS
      4. Allowing Relaying over TLS Sessions
      5. Variables That Are Set for a TLS Connection
      6. Configuring Exim to Use TLS as a Client
    3. SMTP over TCP/IP
      1. Outgoing SMTP over TCP/IP
      2. Incoming SMTP Messages over TCP/IP
      3. The VRFY and EXPN Commands
      4. The ETRN Command
    4. Local SMTP
    5. Batched SMTP
  18. 16. File and Database Lookups
    1. Single-Key Lookup Types
    2. Query-Style Lookup Types
    3. Quoting Lookup Data
    4. NIS+
    5. LDAP
      1. Data Returned by an LDAP Lookup
    6. MySQL and PostgreSQL
    7. DNS Lookups
    8. Implicit Keys in Query-Style Lookups
    9. Temporary Errors in Lookups
    10. Default Values in Single-Key Lookups
    11. Partial Matching in Single-Key Lookups
    12. Lookup Caching
  19. 17. String Expansion
    1. Variable Substitution
    2. Header Insertion
    3. Operations on Substrings
      1. Extracting the Initial Part of a Substring
      2. Extracting an Arbitrary Part of a Substring
      3. Hashing Operators
      4. Forcing the Case of Letters
    4. Character Translation
    5. Text Substitution
    6. Conditional Expansion
      1. Testing for a Specific String
      2. Negated Conditions
      3. Regular Expression Matching
      4. Encrypted String Comparison
      5. PAM Authentication
      6. Numeric Comparisons
      7. Empty Variables and Nonexistent Header Lines
      8. File Existence
      9. The State of a Message’s Delivery
      10. Combining Expansion Conditions
      11. Forcing Expansion Failure
    7. Lookups in Expansion Strings
      1. Single-Key Lookups in Expansion Strings
      2. Partial Lookups in Expansion Strings
      3. Single-Key Lookup Failures
      4. Query-Style Lookups in Expansion Strings
      5. Reducing the Number of Database Queries
      6. Defaults for Lookups in Expansion Strings
      7. A Shorthand for One Common Case
    8. Extracting Fields from Substrings
      1. Splitting Up Addresses
      2. Extracting Named Fields from a Line of Data
      3. Extracting Unnamed Fields from a Line of Data
    9. IP Address Masking
    10. Quoting
      1. Quoting Addresses
      2. Quoting Data for Regular Expressions
      3. Quoting Data in Lookup Queries
      4. Quoting Printing Data
    11. Reexpansion
    12. Running Embedded Perl
    13. Testing String Expansions
  20. 18. Domain, Host, and Address Lists
    1. Negative Items in Lists
    2. List Items in Files
    3. Lookup Items in Lists
    4. Domain Lists
    5. Host Lists
      1. Host Checks by IP Address
      2. Host Checking Using Forward Lookup
      3. Host Checking Using Reverse Lookup
      4. Use of RFC 1413 Identification in Host Lists
    6. Address Lists
      1. Case of Letters in Address Lists
  21. 19. Miscellany
    1. Security Issues
      1. Use of Root Privilege
        1. How Unix uses uids to control privilege
        2. Why does Exim need root privilege?
        3. Relinquishing root privilege
        4. The security option
      2. Running Local Deliveries as root
      3. Running an Unprivileged Exim
    2. Privileged Users
      1. Trusted Users
        1. Setting the sender of a locally submitted message
        2. Setting other information in a locally submitted message
      2. Admin Users
    3. RFC Conformance
      1. 8-Bit Characters
      2. Address Syntax
        1. Built-in address syntax extensions
        2. Configurable address syntax extensions
        3. Domain literal addresses
        4. Source routed addresses
      3. Canonicizing Addresses
      4. Coping with Broken MX Records
      5. Line Terminators in SMTP
      6. Syntax of HELO and EHLO
    4. Timestamps
    5. Checking Spool Space
    6. Control of DNS Lookups
    7. Bounce Message Handling
      1. Replying to Bounce Messages
      2. Taking Copies of Bounce Messages
      3. Messages to Postmaster
      4. Delay Warning Messages
      5. Customizing Bounce Messages
      6. Customizing Warning Messages
    8. Miscellaneous Controls
  22. 20. Command-Line Interface to Exim
    1. Input Mode Control
      1. Starting a Daemon Process
      2. Interactive SMTP Reception
      3. Batch SMTP Reception
      4. Non-SMTP reception
      5. Summary of Reception Options
    2. Additional Message Data
      1. Sender Address
      2. Sender Name
      3. Remote Host Information
    3. Immediate Delivery Control
    4. Error Routing
    5. Queue Runner Processes
      1. Overriding Retry Times and Freezing
      2. Local Addresses Only
      3. Two-Pass Processing for Remote Addresses
      4. Periodic Queue Runs
      5. Processing Specific Messages
      6. Processing Specific Addresses
      7. Summary of Queue Runner Options
    6. Configuration Overrides
    7. Watching Exim’s Queue
    8. Message Control
      1. Operations on a List of Messages
      2. Inspecting a Queued Message
      3. Modifying a Queued Message
      4. Summary of Message Control Options
    9. Testing Options
      1. Testing the Configuration Settings
      2. Testing Address Handling
      3. Testing Incoming Connections
      4. Testing Retry Rules
      5. Testing Rewriting Rules
      6. Testing Filter Files
      7. Testing String Expansion
    10. Options for Debugging
      1. Suppressing Delivery
    11. Terminating the Options
    12. Embedded Perl Options
    13. Compatibility with Sendmail
    14. Calling Exim by Different Names
  23. 21. Administering Exim
    1. Log Files
    2. Log Destination Control
      1. Logging to syslog
      2. Log Level
      3. Other Options Affecting Log Content
    3. Format of Main Log Entries
      1. Logging Message Reception
      2. Logging Deliveries
      3. Deferred Deliveries
      4. Delivery Failures
      5. Message Completion
      6. Other Log Entries
    4. Cycling Log Files
    5. Extracting Information from Log Files
      1. The exigrep Utility
      2. The eximstats Utility
    6. Watching What Exim is Doing
      1. The exiqsumm Utility
      2. The exinext Utility
      3. Querying Exim Processes
    7. The Exim Monitor
      1. Running the Monitor
      2. The Stripcharts
      3. Main Action Buttons
      4. The Log Display
      5. The Queue Display
      6. The Queue Menu
    8. Maintaining Alias and Other Datafiles
      1. Maintaining DBM Files
    9. Hints Database Maintenance
    10. Mailbox Maintenance
  24. 22. Building and Installing Exim
    1. Prerequisites
    2. Fetching and Unpacking the Source
    3. Configuration for Building
      1. The Contents of Local/Makefile
      2. Mandatory Makefile Settings
      3. Driver Choices in the Makefile
      4. Module Choices in the Makefile
      5. Recommended Makefile Settings
      6. A Plausible Minimal Makefile
      7. System-Related Makefile Settings
      8. Optional Settings in the Makefile
      9. Configuration for Building the Exim Monitor
      10. Building Exim for Multiple Systems
    4. The Building Process
    5. Installing Exim
    6. Testing Before Turning On
    7. Turning Exim On
    8. Installing Documentation in Info Format
    9. Upgrading to a New Release
  25. A. Summary of String Expansion
    1. Expansion Items
    2. Expansion Conditions
      1. Combining Conditions
    3. Expansion Variables
  26. B. Regular Expressions
    1. Testing Regular Expressions
    2. Metacharacters
    3. Backslash
    4. Changing Matching Options
    5. Circumflex and Dollar
    6. Dot (Period, Full Stop)
    7. Square Brackets
    8. POSIX Character Classes
    9. Vertical Bar
    10. Subpatterns
    11. Repetition
    12. Back References
    13. Assertions
    14. Once-Only Subpatterns
    15. Conditional Subpatterns
    16. Comments
    17. Recursive Patterns
    18. Performance
  27. Index
  28. About the Author
  29. Colophon
  30. Special Upgrade Offer
  31. Copyright