You are previewing Building and Integrating Virtual Private Networks with Openswan.
O'Reilly logo
Building and Integrating Virtual Private Networks with Openswan

Book Description

Building and Integrating Virtual Private Networks with Openswan is written by the Openswan development team.



This book is a comprehensive guide to using Openswan for building both basic and industry size, military strength VPNs for medium to very large organizations. Written by the core developers, this practical book is all you need to use Openswan to build any VPN infrastructure you may need. The authors have covered the latest developments and upcoming issues. This book will not only help you build the VPN you need, but also save you a lot of time.

Table of Contents

  1. Copyright
  2. Credits
  3. About the Authors
  4. Acknowledgements
  5. About the Reviewers
  6. Preface
    1. What This Book Covers
    2. What You Need for This Book
    3. Conventions
    4. Reader Feedback
    5. Customer Support
      1. Errata
      2. Questions
  7. 1. Introduction
    1. The Need for Cryptography
      1. Privacy
      2. Security
    2. A History of the Internet
      1. Holding the Internet Together
      2. The Creation of ICANN
      3. ICANN Bypassed
      4. The Root Name Servers
      5. Running the Top-Level Domains
    3. History of Internet Engineering
      1. The Internet Engineering Task Force (IETF)
      2. RFCs—Requests For Comments
      3. IETF and Crypto
    4. The War on Crypto
      1. Dual Use
      2. Public Cryptography
      3. The Escrowed Encryption Standard
      4. Export Laws
      5. The Summer of '97
      6. The EFF DES Cracker
      7. Echelon
      8. The End of the Export Restrictions
    5. Free Software
      1. The GPL
      2. Free as in Verifiable
      3. The Open Source Movement
    6. The History of Openswan
      1. IETF Troubles over DNS
      2. Super FreeS/WAN
      3. The Arrival of Openswan
      4. NETKEY
      5. Further Reading
    7. Using Openswan
      1. Copyright and License Conditions
      2. Writing and Contributing Code
      3. Legality of Using Openswan
      4. International Agreements
      5. International Law and Hosting Openswan
      6. Unrecognized International Claims
      7. Patent Law
        1. Expired and Bogus Patents
      8. Useful Legal Links
    8. Summary
  8. 2. Practical Overview of the IPsec Protocol
    1. A Very Brief Overview of Cryptography
      1. Valid Packet Rewriting
      2. Ciphers
        1. DES, 3DES, and AES
      3. Algorithms
      4. Uniqueness
      5. Public-Key Algorithms
      6. Exchanging Public Keys
      7. Digital Signatures
      8. Diffie-Hellman Key Exchange
      9. Avoiding the Man in the Middle
      10. Session Keys
      11. Crypto Requirements for IPsec
    2. IPsec: A Suite of Protocols
    3. Kernel Mode: Packet Handling
      1. Authentication Header (AH)
      2. Encapsulated Security Payload (ESP)
      3. Transport and Tunnel Mode
      4. Choosing the IPsec Mode and Type
      5. The Kernel State
      6. Encryption Details
      7. Manual Keying
      8. Final Note on Protocols and Ports
    4. Usermode: Handling the Trust Relationships
      1. The IKE Protocol
        1. Phase 1: Creating the ISAKMP SA
          1. Main Mode
          2. Aggressive Mode
          3. Main Mode versus Aggressive Mode
        2. Phase 2: Quick Mode
          1. Perfect Forward Secrecy (PFS)
          2. VendorIDs
          3. Dead Peer Detection (DPD)
          4. ModeConfig
          5. XAUTH
          6. X.509 Certificates
        3. The NAT Problem
          1. NATworks
          2. Passing Clients Through
          3. AT-Traversal
          4. NAT-T and IPsecPassthrough
          5. NAT-T Intricacies
          6. NAT-T's Hard Limit
    5. Summary
  9. 3. Building and Installing Openswan
    1. Linux Distributions
      1. Red Hat
      2. Debian
      3. SuSE
      4. Slackware
      5. Gentoo
      6. Linux 'Router' Distributions
    2. Deciding on the Userland
      1. Pluto
      2. Racoon
      3. Isakmpd
      4. More Reasons to Pick Pluto
    3. Choosing the Kernel IPsec Stack
      1. KLIPS, the Openswan Stack
        1. ipsecX Interfaces
        2. First Packet Caching
        3. Path MTU Discovery
          1. Firewalls and Path MTU
        4. KLIPS' Downside
      2. NETKEY, the 2.6 IPsec Stack
      3. The USAGI / SuSE IPsec Stack
      4. Making the Choice
      5. GPL Compliance and KLIPS
    4. Binary Installation of the Openswan Userland
      1. Checking for Old Versions
      2. Installing the Binary Package for Openswan
    5. Building from Source
      1. Using RPM-based Distributions
      2. Rebuilding the Openswan Userland
      3. Building src.rpm from Scratch
      4. Openswan Options
    6. Building the Openswan Userland from Source
      1. Downloading the Source Code
      2. Configuring the Userland Tools
        1. Optional Features
        2. Compile Flags
        3. File Path Options
        4. Obscure Pluto Options
      3. Compiling and Installing
    7. Binary Installation of KLIPS
    8. Building KLIPS from Source
      1. Kernel Prerequisites
      2. Identifying your Kernel's Abilities
      3. Using Both KLIPS and NETKEY
      4. The Kernel Build Options
        1. Required Kernel Options
        2. Desired Options
        3. NETKEY Stack Options
        4. KLIPS Stack Options
        5. L2TP Options
      5. Patching the Kernel
        1. NAT-Traversal Patch
        2. KLIPS Compile Shortcut
        3. Activating KLIPS
        4. Determining the Stack in Use
    9. Building KLIPS into the Linux Kernel Source Tree
      1. Building a Standard Kernel
      2. NAT Traversal
      3. Patching KLIPS into the Linux Kernel
    10. Verifying the Installation
    11. Summary
  10. 4. Configuring IPsec
    1. Manual versus Automatic
    2. PSK versus RSA
    3. Pitfalls of Debugging IPsec
    4. Pre-Flight Check
      1. The ipsec verify Command
      2. NAT and Masquerading
      3. Checking External Commands
      4. Opportunistic Encryption
    5. The ipsec livetest Command
    6. Configuration of Openswan
      1. The ipsec.conf File
    7. Host-to-Host Tunnel
      1. Left and Right
      2. The type Options
      3. The auto Option
      4. The rsasigkey Options
      5. Bringing Up the IPsec Tunnels
      6. Listing IPsec Connections
      7. Testing the IPsec Tunnel
    8. Connecting Subnets Through an IPsec Connection
      1. Testing Subnet Connections
      2. Testing Properly
      3. Encrypting the Host and the Network Behind It
        1. Employing Advanced Routing
        2. Creating More Tunnels
    9. Avoiding Duplication
      1. The Also Keyword
    10. KLIPS and the ipsecX Interfaces
    11. Pre-Shared Keys (PSKs)
      1. Proper Secrets
    12. Dynamic IP Addresses
      1. Hostnames
      2. Roadwarriors
      3. Multiple Roadwarrior Connections
      4. Dynamic IP and PSKs
      5. PSK and NAT
      6. Mixing PSK and RSA
    13. Connection Management
    14. Subnet Extrusion
    15. NAT Traversal
      1. Deprecated Syntax
      2. Confirming a Functional NAT-T
    16. Dead Peer Detection
      1. DPD Works Both Ways
      2. Configuring DPD
      3. Buggy Cisco Routers
    17. Ciphers and Algorithms
      1. Using ike= to Specify Phase 1 Parameters
      2. Using esp= to Specify Phase 2 Parameters
      3. Defaults and Strictness
      4. Unsupported Ciphers and Algorithms
    18. Aggressive Mode
    19. XAUTH
      1. XAUTH Gateway (Server Side)
      2. XAUTH Client (Supplicant Side)
    20. Fine Tuning
      1. Perfect Forward Secrecy
      2. Rekeying
      3. Key Rollover
    21. Summary
  11. 5. X.509 Certificates
    1. X.509 Certificates Explained
      1. X.509 Objects
      2. X.509 Packing
      3. Types of Certificates
      4. Passphrases, PIN Codes, and Interactivity
      5. IKE and Certificates
      6. Using the Certificate DN as ID for Openswan
    2. Generating Certificates with OpenSSL
      1. Setting the Time
      2. Configuring OpenSSL
        1. Be Consistent with All Certificates
      3. OpenSSL Commands for Common Certificate Actions
      4. Configuring Apache for IPsec X.509 Files
    3. Creating X.509-based Connections
    4. Using a Certificate Authority
      1. Using Multiple CAs
      2. Sending and Receiving Certificate Information
      3. Creating your own CA using OpenSSL
      4. Creating Host Certificates with Your Own CA
        1. Host Certificates for Microsoft Windows (PKCS#12)
      5. Certificate Revocation
        1. Dynamic CRL Fetching
        2. Configuring CRL
        3. Online Certificate Status Protocol (OCSP)
    5. Summary
  12. 6. Opportunistic Encryption
    1. History of Opportunistic Encryption
    2. Trusting Third Parties
      1. Trusting the DNS?
    3. OE in a Nutshell
      1. An OE Security Gateway
    4. DNS Key Records
      1. Forward and Reverse Zones
      2. The OE DNS Records
      3. Different Types of OE
    5. Policy Groups
    6. Internal States
    7. Configuring OE
      1. Configuring Policies
      2. Full OE or Initiate-Only
      3. Generating Correct DNS Records
      4. Name Server Updates
      5. Verifying Your OE Setup
    8. Testing Your OE Setup
      1. The trap eroute
      2. The pass eroute
      3. The hold eroute
    9. Manipulating OE Connections Manually
    10. Advanced OE Setups
    11. Caveats
    12. Summary
  13. 7. Dealing with Firewalls
    1. Where to Firewall?
    2. Allowing IPsec Traffic
      1. NAT and IPsec Passthrough
    3. Configuring the Firewall on the Openswan Host
      1. Firewalling and KLIPS
      2. Firewalling and NETKEY
      3. Packet Size
    4. Summary
  14. 8. Interoperating with Microsoft Windows and Apple Mac OS X
      1. Microsoft Windows
      2. Mac OS X
    1. Layer 2 Tunneling Protocol (L2TP)
      1. Assigning an IP for VPN Access
      2. L2TP Properties
      3. Pure IPsec versus L2TP/IPsec
      4. L2TP: PSK or X.509
    2. Client and Server Configurations for L2TP/IPsec
      1. The L2TP Openswan Server
        1. Configuring Openswan for L2TP/IPsec
        2. Linux Kernel Runtime Parameters for L2TP/IPsec
        3. Protecting the L2TP Daemon with IPsec using iptables
      2. Choosing an L2TP Daemon
        1. Configuring L2TPD
        2. Configuring User Authentication for pppd
    3. Microsoft Windows XP L2TP Configuration
    4. Microsoft Windows 2000 L2TP Configuration
    5. Apple Mac OS X L2TP Configuration
    6. Server Configuration for X.509 IPsec without L2TP
      1. Openswan Configuration for X.509 without L2TP
    7. Client Configuration for X.509 IPsec without L2TP
      1. Microsoft's IKE Daemon
      2. Microsoft's Certificate Store
      3. Clients using Microsoft Native IPsec Implementation
        1. The ipsec.exe Wrapper
        2. The Linsys IPsec Tool (lsipsectool)
        3. Securepoint IPsec Client
        4. TauVPN (iVPN)
        5. The WaveSEC Client
      4. Third-Party Replacement Clients for Windows
        1. The GreenBow VPN Client
        2. Astaro Secure Client
        3. Mac OS X IPSecuritas
        4. VPNtracker
        5. Manual Racoon Configuration
    8. Importing X.509 Certificates into Windows
    9. Importing X.509 Certificates on Mac OS X (Tiger)
    10. Summary
  15. 9. Interoperating with Other Vendors
    1. Openswan as a Client to an Appliance
    2. Preparing the Interop
      1. The Human Factor
      2. Terminology
      3. Preparation
        1. IPsec Passthrough
        2. Tunnel Limitations
        3. Anticipate Known Problems
        4. Update the Firmware
      4. GUI Issues
      5. Keepalives
      6. ISP Filtering
    3. Frequently used VPN Gateways
      1. Webmin with Openswan
      2. Cisco VPN 3000
      3. Cisco PIX Concentrator
      4. Nortel Contivity
      5. Checkpoint
      6. WatchGuard Firebox
      7. Symantec
    4. Frequently used VPN Client Appliances
      1. ZyXEL
      2. DrayTek Vigor
        1. The Vigor Web Interface
        2. Windows Logon Issues
        3. Other Vigorisms
        4. Unresolved Issues
          1. IPsec SA Limitation
          2. Rekeying Issue
          3. Phase 1 Resets
          4. Monitor
          5. Logs
          6. Spurious 1DES Requests
          7. CIDR Writing Bug
          8. UDP Fragmenting Option
          9. Enabling the IPsec VPN Service
      3. NetScreen
        1. Known Issues
      4. SonicWALL
      5. BinTec
      6. LANCOM
      7. Linksys
      8. Lucent Brick
      9. NETGEAR
      10. KAME/Racoon
    5. Aftercare
    6. Summary
  16. 10. Encrypting the Local Network
    1. Methods of Encryption
      1. Host-to-Host Mesh
      2. Host-to-Gateway Setup
      3. Single IP Extrusiautomation or L2TP
      4. Opportunistic Encryption in the LAN
      5. Non-OE-Capable Machines
    2. Designing a Solution for Encrypting the LAN
      1. Design Goals
        1. Separation of WiFi and Crypto
        2. Link Layer Protection
        3. The Logical Choice: IPsec
        4. Hotspot
    3. WaveSEC
      1. Full WaveSEC
      2. Catch 22 Traffic
      3. Building a WaveSEC Server
        1. DHCP Server Setup
        2. DNS Server Setup
        3. Openswan Server Setup
        4. Catch 22 Traffic Setup
      4. Building a WaveSEC Client
        1. DH Client Setup
        2. Openswan Setup
        3. Testing the WaveSEC
        4. Starting the WaveSEC Connection
      5. Known Issues with WaveSEC
    4. WaveSEC for Windows
      1. Design Limitations
      2. Building a WaveSEC for Windows Server
        1. Obtaining the Certificate and Client Software
      3. Our Prototype Experiences
        1. Openswan Issues
          1. Windows Client Issues
        2. Windows Kernel Issues
    5. Summary
  17. 11. Enterprise Implementation
    1. Cipher Performance
    2. Handling Thousands of Tunnels
    3. Managing Large Configuration Files
      1. Standard Naming Convention
      2. The also= Parameter
      3. The include Parameter
    4. Openswan Startup Time
    5. Limitations of the Random Device
    6. Other Performance-Enhancing Factors
      1. Logging to Disk
      2. Disable Dead Peer Detection
      3. Reducing the Number of Tunnels
        1. OSPF Setup
        2. BGPv4 Setup
      4. High Availability
      5. Heartbeat
      6. Xen Migration
    7. Using Anycast
    8. Summary
  18. 12. Debugging and Troubleshooting
    1. Do Not Lock Yourself Out!
    2. Narrowing Down the Problem
      1. Host Issues
    3. Configuration Problems
      1. Connection Names
      2. Interoperability
      3. Hunting Ghosts
      4. Rekey Problems (After an Hour)
    4. Openswan Error Messages
        1. IKE: Unknown VendorIDs
    5. Network Issues
        1. Firewalls
        2. MTU and Fragmentation Issues
    6. Debugging IPsec on Apple Mac OS X
    7. Debugging IPsec on Microsoft Windows
      1. Oakley Debugging
      2. Debugging ipsec.exe
      3. Microsoft L2TP Errors
        1. You Suddenly Cannot Log in Anymore over the VPN
    8. Software Bugs
        1. Userland Issues: Assertion Failed or Segmentation Faults
        2. Kernel Issues: Crashes and Oopses
        3. Memory Issues
    9. Common IKE Error Messages
      1. Common Kernel-Related Error Messages
      2. Common Errors when Upgrading
    10. Using tcpdump to Debug IPsec
        1. Situation A: No Communication on Port 500
          1. A1: All Packets Arrive on sg2
          2. A2: No Packets Arrive on sg2
        2. Situation B: Failure at Third Exchange
          1. Fragmentation Problem
          2. Port 4500 is Closed
        3. Situation C: QUICK Mode Initiates, but Never Completes
        4. Situation D: All IKE Messages Occur, but no Traffic Flows
        5. A Final tcpdump Example
    11. User Mode Linux Testing
      1. Preparing the Openswan for the UML Build Process
      2. Running the UMLs
      3. Writing a UML Test Case
      4. Debugging the Kernel with GDB
    12. Asking the Openswan Community for Help
      1. Internet Relay Chat (IRC)
      2. The Openswan Mailing Lists
        1. Posting to the Lists
        2. Research First, Ask Later
        3. Free, as in Beer
        4. Do not Anonymize
    13. Summary
  19. A. Unresolved and Upcoming Issues
      1. Linux Kernel Developments
      2. Kernel API Changes between 2.6.12 and 2.6.14
      3. Red Hat Kernel Developments
      4. Fedora Kernel Source/Headers Packaging Change
      5. MD5 Insecurities
      6. Discontinuation of Openswan 1 by the End of 2005
      7. Update on UML Testing Suite Installation
      8. Openswan GIT Repositories
      9. Openswan on Windows and Mac OS X Updates
      10. Known Outstanding Bugs
      11. Vulnerability Fixes in Openswan 2.4.4
  20. B. Networking 101
      1. The OSI Model and the IP Model
      2. No Layers, Just Packets
      3. The Protocol
      4. IP Network Overview
      5. IP Address Management
      6. The Old IP Classes
      7. Classless IP Networks
      8. The Definition of a Subnet
      9. Calculating with Subnets: The Subnet Mask
      10. The Rest of the Network
      11. Linux Networking Commands
      12. Routing
      13. Routing Decisions
      14. Peering
      15. Network Address Translation
      16. Port Forwarding
  21. C. Openswan Resources on the Internet
        1. Openswan Links
        2. Community Documentation
        3. Generic Linux Distributions Containing Openswan
        4. Specialized Linux Distributions Containing Openswan
  22. D. IPsec-Related Requests For Comments (RFCs)
    1. Overview RFCs
    2. Basic Protocols
    3. Key Management
    4. Procedural and Operational RFCs
    5. Detailed RFCs on Specific Cryptographic Algorithms and Ciphers
    6. Dead Peer Detection RFCs
    7. NAT-Traversal and UDP Encapsulation RFCs
    8. RFCs for Secure DNS Service, which IPSEC May Use
    9. RFCs Related to L2TP, Often Used in Combination with IPsec
    10. RFCs on IPsec in Relation to Other Protocols
    11. RFCs Not in Use or Implemented across Multiple Vendors