You are previewing Snort Intrusion Detection and Prevention Toolkit.
O'Reilly logo
Snort Intrusion Detection and Prevention Toolkit

Book Description

This all new book covering the brand new Snort version 2.6 from members of the Snort developers team.

This fully integrated book and Web toolkit covers everything from packet inspection to optimizing Snort for speed to using the most advanced features of Snort to defend even the largest and most congested enterprise networks. Leading Snort experts Brian Caswell, Andrew Baker, and Jay Beale analyze traffic from real attacks to demonstrate the best practices for implementing the most powerful Snort features.

The companion material contains examples from real attacks allowing readers test their new skills. The book will begin with a discussion of packet inspection and the progression from intrusion detection to intrusion prevention. The authors provide examples of packet inspection methods including: protocol standards compliance, protocol anomaly detection, application control, and signature matching. In addition, application-level vulnerabilities including Binary Code in HTTP headers, HTTP/HTTPS Tunneling, URL Directory Traversal, Cross-Site Scripting, and SQL Injection will also be analyzed. Next, a brief chapter on installing and configuring Snort will highlight various methods for fine tuning your installation to optimize Snort performance including hardware/OS selection, finding and eliminating bottlenecks, and benchmarking and testing your deployment. A special chapter also details how to use Barnyard to improve the overall performance of Snort. Next, best practices will be presented allowing readers to enhance the performance of Snort for even the largest and most complex networks. The next chapter reveals the inner workings of Snort by analyzing the source code. The next several chapters will detail how to write, modify, and fine-tune basic to advanced rules and pre-processors. Detailed analysis of real packet captures will be provided both in the book and the companion material. Several examples for optimizing output plugins will then be discussed including a comparison of MySQL and PostrgreSQL. Best practices for monitoring Snort sensors and analyzing intrusion data follow with examples of real world attacks using: ACID, BASE, SGUIL, SnortSnarf, Snort_stat.pl, Swatch, and more.

The last part of the book contains several chapters on active response, intrusion prevention, and using Snort’s most advanced capabilities for everything from forensics and incident handling to building and analyzing honey pots. Data from real world attacks will be presented throughout this part as well as on the companion website, http://booksite.elsevier.com/9781597490993/

  • This fully integrated book and Web toolkit covers everything all in one convenient package
  • It is authored by members of the Snort team and it is packed full of their experience and expertise
  • Includes full coverage of the brand new Snort version 2.6, packed full of all the latest information
  • Companion website at http://booksite.elsevier.com/9781597490993/ contains all companion material

Table of Contents

  1. Copyright
  2. Visit Us At
  3. Acknowledgments
  4. Technical Editor
  5. Contributing Authors
  6. Foreword
  7. Series Editor
  8. Foreword
  9. 1. Intrusion Detection Systems
    1. Introduction
    2. What Is Intrusion Detection?
      1. Network IDS
      2. Host-Based IDS
      3. Distributed IDS
    3. How an IDS Works
      1. Where Snort Fits
      2. Intrusion Detection and Network Vulnerabilities
      3. Identifying Worm Infections with IDS
      4. Identifying Server Exploit Attempts with IDS
      5. Decisions and Cautions with IDS
    4. Why Are Intrusion Detection Systems Important?
      1. Why Are Attackers Interested in Me?
      2. What Will an IDS Do for Me?
      3. What Won’t an IDS Do for Me?
      4. Where Does an IDS Fit with the Rest of My Security Plan?
      5. Doesn’t My Firewall Serve As an IDS?
      6. Where Else Should I Be Looking for Intrusions?
        1. Backdoors and Trojans
        2. Physical Security
        3. Application and Data Integrity
    5. What Else Can You Do with Intrusion Detection Systems?
      1. Monitoring Database Access
      2. Monitoring DNS Functions
      3. E-Mail Server Protection
      4. Using an IDS to Monitor My Company Policy
    6. What About Intrusion Prevention?
    7. Summary
    8. Solutions Fast Track
      1. What Is Intrusion Detection?
      2. A Trilogy of Vulnerabilities
      3. Why Are Intrusion Detection Systems Important?
      4. What Else Can You Do with Intrusion Detection?
    9. Frequently Asked Questions
  10. 2. Introducing Snort 2.6
    1. Introduction
    2. What Is Snort?
    3. What’s New in Snort 2.6
      1. Engine Improvements
      2. Preprocessor Improvements
      3. Rules Improvements
    4. Snort System Requirements
      1. Hardware
        1. Operating System
        2. Other Software
    5. Exploring Snort’s Features
      1. Packet Sniffer
      2. Preprocessor
      3. Detection Engine
      4. Alerting/Logging Component
    6. Using Snort on Your Network
      1. Snort’s Uses
        1. Using Snort as a Packet Sniffer and Logger
        2. Using Snort as an NIDS
      2. Snort and Your Network Architecture
        1. Snort and Switched Networks
      3. Pitfalls When Running Snort
        1. False Alerts
        2. Upgrading Snort
    7. Security Considerations with Snort
      1. Snort Is Susceptible to Attacks
      2. Securing Your Snort System
    8. Summary
    9. Solutions Fast Track
      1. What Is Snort?
      2. Exploring Snort’s Features
      3. Using Snort on Your Network
      4. Security Considerations with Snort
    10. Frequently Asked Questions
  11. 3. Installing Snort 2.6
    1. Introduction
    2. Choosing the Right OS
      1. Performance
        1. The Operating System and the CPU
        2. The Operating System and the NIC
      2. Stability
      3. Security
      4. Support
      5. Cost
      6. Stripping It Down
        1. Removing Nonessential Items
      7. Debian Linux
      8. CentOS
      9. Gentoo
      10. The BSDs
        1. OpenBSD
          1. Installing OpenBSD and Snort
      11. Windows
      12. Bootable Snort Distros
        1. The Network Security Toolkit As a Snort Sensor
          1. Booting the System
          2. Configuring NST’s Web User Interface
          3. Configuring Snort
    3. Hardware Platform Considerations
      1. The CPU
      2. Memory
        1. Memory’s Influence on System Performance
        2. Virtual Memory
      3. The System Bus
        1. PCI
        2. PCI-X
        3. PCI-Express
        4. Theoretical Peak Bandwidth
        5. Dual vs. Single Bus
      4. The NIC
      5. Disk Drives
    4. Installing Snort
      1. Prework
        1. Installing pcap
        2. Installing/Preparing Databases
        3. Time Synchronization (NTP)
      2. Installing from Source
        1. Benefits and Costs
        2. Compile-Time Options
      3. Installing Binaries
        1. Apt-get
        2. RPM
        3. Windows
      4. Hardening
        1. General Principles
          1. Bastille Linux
          2. AppArmor
          3. Sys Trace
          4. SELinux
          5. LIDS
    5. Configuring Snort
      1. The snort.conf File
      2. Variables
        1. Using Variables in snort.conf and in Rules
      3. Command-Line Switches
      4. Configuration Directives
        1. Snort.conf –dynamic–* Options
        2. Ruletype
      5. Plug-In Configuration
        1. Preprocessors
          1. Flow
          2. Frag3
          3. Stream4
          4. sfPortscan
        2. Output Plug-Ins
      6. Included Files
        1. Rules Files
        2. sid-msg.map
        3. threshold.conf
        4. gen-msg.map
        5. classification.config
      7. Thresholding and Suppression
    6. Testing Snort
      1. Testing within Organizations
        1. Small Organizations
          1. Using a Single Box or Nonproduction Test Lab
        2. Large Organizations
    7. Maintaining Snort
      1. Updating Rules
      2. How Can Updating Be Easy?
    8. Updating Snort
      1. Upgrading Snort
      2. Monitoring Your Snort Sensor
    9. Summary
    10. Solutions Fast Track
      1. Choosing the Right OS
      2. Hardware Platform Considerations
      3. Installing Snort
      4. Configuring Snort
      5. Testing Snort
      6. Maintaining and Updating Snort
    11. Frequently Asked Questions
  12. 4. Configuring Snort and Add-Ons
    1. Placing Your NIDS
    2. Configuring Snort on a Windows System
      1. Installing Snort
      2. Configuring Snort Options
      3. Using a Snort GUI Front End
        1. Configuring IDS Policy Manager
    3. Configuring Snort on a Linux System
      1. Configuring Snort Options
      2. Using a GUI Front-End for Snort
        1. Basic Analysis and Security Engine
    4. Other Snort Add-Ons
      1. Using Oinkmaster
      2. Additional Research
    5. Demonstrating Effectiveness
    6. Summary
    7. Solutions Fast Track
      1. Configuring an Intrusion Detection System
      2. Configuring Snort on a Windows System
      3. Configuring Snort on a Linux System
      4. Other Snort Add-Ons
      5. Demonstrating Effectiveness
    8. Frequently Asked Questions
  13. 5. Inner Workings
    1. Introduction
    2. Snort Initialization
      1. The Command Line
      2. Parsing the Config File
        1. Parsing Rules
      3. Housekeeping (i.e., Signal Handling)
    3. Snort Packet Processing
      1. Packet Acquisition
      2. Decoding
      3. Analyzing in the Preprocessors
      4. Evaluating against the Detection Engine
      5. Logging and Alerting
        1. The Event Queue
        2. Thresholds
        3. Suppression
        4. Tagging
    4. Inside the Detection Engine
      1. Rule Options
        1. The Content Option
        2. The bytejump and bytetest Options
        3. The PCRE Option
        4. The flowbits Option
      2. The Pattern-Matching Engine
        1. Building the Pattern Matcher
        2. Performance of the Different Algorithms
    5. The Dynamic Detection Engine
      1. Using the Engine
        1. Configuring the Engine
        2. Stub Rules
      2. The Dynamic Detection API
        1. The Rule Structure
        2. The Rule Options
          1. The Preprocessor Option
          2. The Content Option
          3. The PCRE Option
          4. The Flowbit Option
          5. The Flowflags Option
          6. The ASN.1 Option
          7. The Check Cursor Option
          8. The Header Check Option
          9. The Byte Test Option
          10. The Byte Jump Option
          11. The Byte Extract Option
          12. The Set Cursor Option
          13. The Loop Option
        3. Dynamic Detection Functions
      3. Writing a Shared Object Rule
        1. Creating the Module Framework
        2. A Simple Shared Object Rule
        3. The Rule Evaluation Function
    6. Summary
    7. Solutions Fast Track
      1. Snort Initialization
      2. Snort Packet Processing
      3. Inside the Detection Engine
      4. The Dynamic Detection Engine
    8. Frequently Asked Questions
  14. 6. Preprocessors
    1. Introduction
    2. What Is a Preprocessor?
    3. Preprocessor Options for Reassembling Packets
      1. The frag2 Preprocessor
        1. Configuring frag2
        2. frag2 Output
      2. The frag3 Preprocessor
        1. Configuring frag3
          1. Examples
        2. frag3 Output
      3. The flow Preprocessor
        1. Configuring flow
      4. The stream4 Preprocessor
        1. TCP Statefulness
        2. Configuring stream4 for Stateful Inspection
        3. Session Reassembly
          1. Configuring stream4 for Session Reassembly
          2. stream4’s Output
      5. A Summary of the State Preprocessors
    4. Preprocessor Options for Decoding and Normalizing Protocols
      1. The Application Preprocessors
      2. Telnet Negotiation
        1. Configuring the telnet_decode Preprocessor
        2. telnet_decode Output
      3. HTTP Inspect
        1. Hex Encoding (IIS and Apache)
        2. Double Percent Hex Encoding
        3. First Nibble Hex Encoding
        4. Second Nibble Hex Encoding
        5. Double Nibble Hex Encoding
        6. UTF-8 Encoding
        7. UTF-8 Barebyte Encoding
        8. Microsoft %U Encoding
        9. Mismatch Encoding
        10. Request Pipelining
        11. Parameter Evasion Using POST and Content-Encoding
        12. Base 36 Encoding
        13. Multislash Obfuscation
        14. IIS Backslash Obfuscation
        15. Directory Traversal
        16. Tab Obfuscation
        17. Invalid RFC Delimiters
        18. Non-RFC Characters
        19. Webroot Directory Transversal
      4. HTTP-Specific IDS Evasion Tools
        1. Using the http_inspect Preprocessor
        2. Configuring the http_inspect Preprocessor
          1. Configuring the http_inspect Global Line
          2. Configuring the http_inspect_server Lines
        3. http_ Inspect Output
      5. rpc_decode
        1. Configuring rpc_decode
        2. rpc_decode Output
    5. Preprocessor Options for Nonrule or Anomaly-Based Detection
      1. sfPortscan
        1. sfPortscan Configuration
        2. sfPortscan Tuning
      2. Back Orifice
        1. Configuring the Back Orifice Preprocessor
      3. Performance Monitoring
        1. Configuring the Performance Monitoring Preprocessor
        2. Configuring the Rule Performance Monitor
        3. Rule Profiling
        4. Preprocessor profiling
    6. Dynamic Preprocessors
      1. SMTP Dynamic Preprocessor
      2. Examples
      3. SMTP Output
      4. FTP_Telnet Dynamic Preprocessor
        1. telnet Preprocessor
        2. ftp Preprocessor
        3. Server Options
        4. Client Commands
        5. DNS Client RData Overflow
        6. Obsolete Record Types
        7. Experimental Record Types
      5. DNS Preprocessor Configuration
    7. Experimental Preprocessors
      1. arpspoof
    8. Summary
    9. Solutions Fast Track
      1. What Is a Preprocessor?
    10. Frequently Asked Questions
  15. 7. Playing by the Rules
    1. Introduction
    2. What Is a Rule?
      1. Where Can I Get Rules?
      2. What Can I Do with Rules?
      3. What Can’t I Do with Rules?
    3. Understanding Rules
      1. Parts of a Rule: Headers
        1. Actions
        2. Protocols
        3. Variables
        4. Ports
      2. Parts of a Rule: Options
        1. Rule Title
        2. Flow
        3. Content
          1. Depth
          2. Offset
          3. Within
          4. Distance
          5. Rawbytes
      3. Parts of a Rule: Metadata
        1. Reference
        2. Classtype
        3. Sid
        4. Rev
    4. Other Advanced Options
      1. Flowbits
      2. Bytetest and Bytejump
      3. PCRE
    5. Ordering for Performance
      1. Anchors
    6. Thresholding
    7. Suppression
    8. Packet Analysis
    9. Rules for Vulnerabilities, Not Exploits
    10. A Rule: Start to Finish
    11. Rules of Note
    12. Stupid Rule Tricks
    13. Keeping Rules Up to Date
      1. Updating Rules
      2. Managing Rules the ‘Hard’ Way
        1. Why Do I Need to Keep My Rules up to Date?
          1. Documentation
          2. The local.rules file
          3. Testing your Rulesets
          4. Knowing When to Update
    14. Summary
    15. Solutions Fast Track
      1. Understanding Rules
      2. You MUST Understand Rule Syntax to Analyze Events!
      3. Always use Flow Where Possible
      4. Controlling the Noise
    16. Frequently Asked Questions
  16. 8. Snort Output Plug-Ins
    1. Introduction
    2. What Is an Output Plug-In?
      1. Key Components of an Output Plug-In
    3. Exploring Snort’s Output Plug-In Options
      1. Default Logging
      2. SNMP Traps
      3. XML Logging
      4. Syslog
      5. SMB Alerting
      6. pcap Logging
      7. Snortdb
      8. Unified Logs
        1. Why Should I Use Unified Logs?
        2. What Do I Do with These Unified Files?
    4. Writing Your Own Output Plug-In
      1. Why Should I Write an Output Plug-In?
      2. Setting Up Your Output Plug-In
      3. Creating Snort’s W3C Output Plug-In
        1. Minimum Functions Required
          1. myPluginSetup (AlertW3CSetup)
          2. myPluginlnit (AlertW3CInit)
          3. myPluginAlert (AlertW3C)
          4. myPluginCleanExit (AlertW3CCleanExit)
          5. myPluginRestart (AlertW3CRestart)
        2. Creating the Plug-In
        3. Running and Testing the Snort W3C Output Plug-In
      4. Dealing with Snort Output
    5. Troubleshooting Output Plug-In Problems
    6. Add-On Tools
      1. Barnyard
      2. Cerebus
      3. Mudpit
    7. Summary
    8. Solutions Fast Track
      1. What Is an Output Plug-In?
      2. Exploring Snort’s Output Plug-In Options
      3. Writing Your Own Output Plug-In
      4. Add-On Tools
    9. Frequently Asked Questions
  17. 9. Exploring IDS Event Analysis, Snort Style
    1. Introduction
    2. What Is Data Analysis?
      1. Data Sources
      2. Events of Interest
      3. Evidence Gathering
    3. Data Analysis Tools
      1. Database Front Ends
        1. BASE
          1. Installing BASE
          2. Prerequisites for Installing BASE
          3. Operating System on BASE Host
          4. The Web Server
          5. PHP
          6. Support Libraries
          7. MySQL or PostgreSQL
          8. Activating BASE
          9. Configuring BASE
          10. Using BASE
          11. Querying the Database
          12. Alert Groups
          13. Graphical Features of BASE
          14. Managing Alert Databases
      2. SGUIL
        1. Installing SGUIL
        2. Step 1: Create the SGUIL Database
        3. Step 2: Installing Sguild, the Server
        4. Step 3: Install a SGUIL Client
        5. Step 4: Install SANCP
        6. Step 5: Install the Sensor Scripts
      3. Using SGUIL
      4. Data Processing Scripts
        1. Snort_stat.pl
        2. SnortSnarf
          1. Installing SnortSnarf
          2. Configuring Snort to Work with SnortSnarf
          3. Basic Usage of SnortSnarf
        3. SnortALog
      5. Visualization Tools
        1. EtherApe
        2. Shoki–Packet Hustler
        3. AfterGlow
          1. Property File
      6. Real-Time Monitoring Tools
        1. Swatch
        2. Tenshi
        3. Pig Sentry
    4. Analyzing Snort Events
      1. Finding Events of Interest
        1. Visualization
      2. Correlating Snort Events
        1. Web Server Correlation
        2. Simple Event Correlator
        3. Free Security Information Management Tools
        4. Commercial Correlation Solutions
    5. Reporting Snort Events
    6. Summary
    7. Solutions Fast Track
      1. What Is Data Analysis?
      2. Data Analysis Tools
      3. Analyzing Snort Events
      4. Reporting Snort Events
    8. Frequently Asked Questions
  18. 10. Optimizing Snort
    1. Introduction
    2. How Do I Choose the Hardware to Use?
      1. What Constitutes “Good” Hardware?
        1. Processors
        2. RAM Requirements
        3. Storage Medium
        4. The Network Interface Card
      2. Location: Tap vs. Span Ports
      3. How Do I Test My Hardware?
    3. How Do I Choose the Operating System to Use?
      1. What Makes a “Good” OS for an NIDS?
      2. What OS Should I Use?
      3. How Do I Test My OS Choice?
    4. Speeding Up Snort
      1. The Initial Decision
      2. Deciding Which Rules to Enable
      3. Notes on Pattern Matching
      4. Configuring Preprocessors for Speed
      5. Choosing an Output Plug-In
    5. Cranking Up the Database
      1. MySQL vs. PostgreSQL
    6. Benchmarking and Testing the Deployment
      1. Benchmark Characteristics
        1. Attributes of a Good Benchmark
        2. Attributes of a Poor Benchmark
      2. What Options Are Available for Benchmarking?
        1. IDS Informer
        2. IDS Wakeup
        3. Sneeze
        4. TCPReplay
        5. Binary Code
        6. THC’s Netdude
        7. Other Packet-Generation Tools
        8. Additional Options
      3. Stress Testing the Pig!
      4. Stress Tests
      5. Individual Snort Rule Tests
      6. Berkeley Packet Filter Tests
      7. Tuning Your Rules
    7. Summary
    8. Solutions Fast Track
      1. How Do I Choose the Hardware to Use?
      2. How Do I Choose the Operating System to Use?
      3. Speeding Up Snort
      4. Cranking Up the Database
      5. Benchmarking and Testing the Deployment
    9. Frequently Asked Questions
  19. 11. Active Response
    1. Introduction
    2. Active Response versus Intrusion Prevention
      1. Response Methods Based on Layers
      2. Attack Response Based on IDS Alerts
        1. SnortSam
        2. Fwsnort
        3. snort_inline
        4. Attack and Response
          1. Web Server WWWBoard passwd.txt Access
          2. NFS mountd Exploit
    3. SnortSam
      1. Installation
      2. Architecture
        1. Snort Output Plug-In
        2. Blocking Agent
      3. SnortSam Configuration Options
      4. SnortSam in Action
        1. WWWBoard passwd.txt Access Attack
        2. NFS mountd Overflow Attack
    4. Fwsnort
      1. Installation
      2. Configuration
      3. Execution
      4. WWWBoard passwd.txt Access Attack (Revisited)
      5. NFS mountd Overflow Attack (Revisited)
    5. snort_Inline
      1. Installation
      2. Compilation Steps for Bridging Linux Kernel
      3. Configuration
      4. Architecture
      5. Web Server Attack
      6. NFS mountd Overflow Attack
    6. Summary
    7. Solutions Fast Track
      1. Active Response versus Intrusion Prevention
      2. SnortSam
      3. Fwsnort
      4. snort_inline
    8. Frequently Asked Questions
  20. 12. Advanced Snort
    1. Introduction
    2. Monitoring the Network
      1. VLAN
    3. Configuring Channel Bonding for Linux
    4. Snort Rulesets
    5. Plug-Ins
    6. Preprocessor Plug-Ins
    7. Detection Plug-Ins
    8. Output Plug-Ins
    9. Snort Inline
    10. Solving Specific Security Requirements
      1. Policy Enforcement
        1. Catching Internal Policy Violators
        2. Banned IP Address Watchlists
      2. Network Operations Support
      3. Forensics and Incident Handling
    11. Summary
    12. Solutions Fast Track
      1. Monitoring the Network
      2. Configuring Channel Bonding for Linux
      3. Snort Rulesets
      4. Preprocessor Plug-Ins
      5. Detection Plug-Ins
      6. Output Plug-Ins
      7. Solving Specific Security Requirements
    13. Frequently Asked Questions
  21. 13. Mucking Around with Barnyard
    1. Introduction
    2. What Is Barnyard?
    3. Understanding the Snort Unified Files
      1. Unified Alert Records
      2. Unified Log Records
      3. Unified Stream-Stat Records
    4. Installing Barnyard
      1. Downloading
      2. Building and Installing
    5. Configuring Barnyard
      1. The Barnyard Command-Line Options
      2. The Configuration File
        1. Configuration Directives
          1. localtime
          2. daemon
          3. Sid-msg-map, gen-msg-map, and class-file
          4. hostname, interface, and filter
        2. Output Plug-In Directives
    6. Understanding the Output Plug-Ins
      1. alert_fast
      2. alert_csv
      3. alert_syslog
      4. alert_syslog2
      5. log_dump
      6. log_pcap
      7. acid_db
      8. sguil
    7. Running Barnyard in Batch-Processing Mode
      1. Processing a Single File
      2. Using the Dry Run Option
      3. Processing Multiple Files
    8. Using the Continual-Processing Mode
      1. The Basics of Continual-Processing Mode
      2. Running in the Background
      3. Enabling Bookmark Support
      4. Only Processing New Events
      5. Archiving Processed Files
      6. Running Multiple Barnyard Processes
      7. Signal Handling
    9. Deploying Barnyard
      1. Remote Syslog Alerting
      2. Database Logging
      3. Extracting Data
      4. Real-Time Console Alerting
    10. Writing a New Output Plug-In
      1. Implementing the Plug-In
        1. Setting Up the Source Files
          1. The Header File
          2. The C File
        2. Writing the Functions
          1. The Init Function
          2. The Setup Function
          3. The Exit Function
          4. The Start Function
          5. The Stop Function
          6. The LogConfig Function
          7. The Output Function
        3. Adding the Plug-In to op_plugbase.c
      2. Finishing Up
        1. Updating Makefile.am
        2. Building Barnyard
      3. Real-Time Console Alerting Redux
    11. Secret Capabilities of Barnyard
    12. Summary
    13. Solutions Fast Track
      1. What Is Barnyard?
      2. Understanding the Snort Unified Files
      3. Installing Barnyard
      4. Configuring Barnyard
      5. Understanding the Output Plug-Ins
      6. Running Barnyard in Batch-Processing Mode
      7. Using the Continual-Processing Mode
      8. Deploying Barnyard
      9. Writing a New Output Plug-In
      10. Secret Capabilities of Barnyard
    14. Frequently Asked Questions
  22. Appendix