You are previewing Applied Security Visualization.
O'Reilly logo
Applied Security Visualization

Book Description

APPLIED SECURITY VISUALIZATION

“Collecting log data is one thing, having relevant information is something else. The art to transform all kinds of log data into meaningful security information is the core of this book. Raffy illustrates in a straight forward way, and with hands-on examples, how such a challenge can be mastered. Let's get inspired.”

–Andreas Wuchner, Head of Global IT Security, Novartis

Use Visualization to Secure Your Network Against the Toughest, Best-Hidden Threats

As networks become ever more complex, securing them becomes more and more difficult. The solution is visualization. Using today’s state-of-the-art data visualization techniques, you can gain a far deeper understanding of what’s happening on your network right now. You can uncover hidden patterns of data, identify emerging vulnerabilities and attacks, and respond decisively with countermeasures that are far more likely to succeed than conventional methods.

In Applied Security Visualization, leading network security visualization expert Raffael Marty introduces all the concepts, techniques, and tools you need to use visualization on your network. You’ll learn how to identify and utilize the right data sources, then transform your data into visuals that reveal what you really need to know. Next, Marty shows how to use visualization to perform broad network security analyses, assess specific threats, and even improve business compliance.

He concludes with an introduction to a broad set of visualization tools. The book’s CD also includes DAVIX, a compilation of freely available tools for security visualization.

You'll learn how to:

• Intimately understand the data sources that are essential for effective visualization

• Choose the most appropriate graphs and techniques for your IT data

• Transform complex data into crystal-clear visual representations

• Iterate your graphs to deliver even better insight for taking action

• Assess threats to your network perimeter, as well as threats imposed by insiders

• Use visualization to manage risks and compliance mandates more successfully

• Visually audit both the technical and organizational aspects of information and network security

• Compare and master today’s most useful tools for security visualization

Contains the live CD Data Analysis and Visualization Linux (DAVIX). DAVIX is a compilation of powerful tools for visualizing networks and assessing their security. DAVIX runs directly from the CD-ROM, without installation.

Raffael Marty is chief security strategist and senior product manager for Splunk, the leading provider of large-scale, high-speed indexing and search technology for IT infrastructures. As customer advocate and guardian, he focuses on using his skills in data visualization, log management, intrusion detection, and compliance. An active participant on industry standards committees such as CEE (Common Event Expression) and OVAL (Open Vulnerability and Assessment Language), Marty created the Thor and AfterGlow automation tools, and founded the security visualization portal secviz.org. Before joining Splunk, he managed the solutions team at ArcSight, served as IT security consultant for PriceWaterhouseCoopers, and was a member of the IBM Research Global Security Analysis Lab.

Table of Contents

  1. Copyright
  2. Praise for Applied Security Visualization
  3. Preface
    1. What This Book Covers
    2. Audience
    3. Structure and Content
    4. Color
  4. Acknowledgments
  5. About the Author
  6. 1. Visualization
    1. What Is Visualization?
    2. Why Visualization?
    3. Visualization Benefits
    4. Security Visualization
    5. Security Visualization’s Dichotomy
    6. Visualization Theory
      1. Perception
      2. Expressive and Effective Graphs
        1. Expressiveness
        2. Effectiveness
      3. Graph Design Principles
        1. Reduce Nondata Ink
        2. Distinct Attributes
        3. Gestalt Principles
        4. Emphasize Exceptions
        5. Show Comparisons
        6. Annotate Data
        7. Show Causality
    7. Information Seeking Mantra
    8. Summary
  7. 2. Data Sources
    1. Terminology
    2. Security Data
    3. Common Problems
      1. Incomplete Information
      2. Source/Destination Confusion
    4. Packet Captures
    5. Traffic Flows
      1. Collecting Traffic Flows
      2. Aggregating Traffic Flows
      3. Clustering Traffic Flows
      4. Anonymizing Traffic Flows
    6. Firewalls
    7. Intrusion Detection and Prevention Systems
    8. Passive Network Analysis
    9. Operating Systems
      1. Real-Time Operating System Information
      2. Operating System State Information
      3. Operating System Log Problems
    10. Applications
      1. Web Proxy
      2. Mail
      3. Databases
    11. Configurations
    12. Summary
  8. 3. Visually Representing Data
    1. Graph Properties
      1. Data Types
      2. Color
      3. Size, Shape, and Orientation
      4. Chart Axes
    2. Simple Charts
      1. Pie Chart
      2. Bar Chart
      3. Line Chart
      4. 3D Bar Charts
    3. Stacked Charts
      1. Stacked Pie Chart
      2. Stacked Bar Chart
      3. Stacked Line Chart
    4. Histograms
    5. Box Plots
    6. Scatter Plots
    7. Parallel Coordinates
      1. Selected target
    8. Link Graphs
    9. Maps
    10. Treemaps
    11. Three-Dimensional Views
      1. Three-Dimensional Scatter Plots
      2. Three-Dimensional Link Graphs
    12. Interaction and Animation
      1. Interaction
      2. Animation
    13. Choosing the Right Graph
    14. Challenges
    15. Summary
  9. 4. From Data to Graphs
    1. Information Visualization Process
    2. Step 1: Define the Problem
    3. Step 2: Assess Available Data
    4. Step 3: Process Information
      1. Adding Additional Data
      2. Filtering Log Entries
      3. Aggregation
      4. Data Processing Challenges
    5. Step 4: Visual Transformation
      1. Data Mapping
      2. Size and Shape
      3. Color
    6. Step 5: View Transformation
      1. Aggregation
    7. Step 6: Interpret and Decide
    8. Tools for Data Processing
      1. Excel, OpenOffice, and Text Editors
      2. Regular Expressions
      3. UNIX tools
        1. grep
        2. awk
        3. sed
      4. Perl
      5. Parsers
      6. Other Tools
    9. Summary
  10. 5. Visual Security Analysis
    1. Reporting
      1. Reporting Tools
      2. Issues and Problems
      3. Reporting Machine Access—An Example
    2. Historical Analysis
      1. Time-Series Visualization
        1. Time Tables
        2. Multiple-Graph Snapshots
        3. Trend Lines
          1. Trend Line Graphing Example
        4. Moving-Average Charts
          1. Simple Moving Average
          2. Advanced Moving Averages
          3. Applying Moving Averages
        5. Sector Graphs
      2. Correlation Graphs
      3. Interactive Analysis
      4. Forensic Analysis
        1. Finding Attacks
          1. Network Flow Data
          2. Intrusion Detection Data
          3. Operating System Log
          4. Application Logs
          5. Additional Data Sources
        2. Assessing an Attack
        3. Documenting an Incident
    3. Real-Time Monitoring and Analysis
      1. Dashboards
        1. The CISO Dashboard
        2. Dashboard Design Principles
      2. Situational Awareness
    4. Summary
  11. 6. Perimeter Threat
    1. Traffic-Flow Monitoring and Analysis
      1. Service Characteristics
      2. Service Anomalies
      3. Worm Detection
      4. Denial of Service
      5. Botnets
      6. Policy-Based Traffic-Flow Analysis
    2. Firewall Log Analysis
      1. Firewall Visualization Process
        1. 1. Determine the Problem
        2. 2. Assess Available Data
        3. 3. Process Information
        4. 4. Visual Transformation
        5. 5. View Transformation
      2. Firewall Ruleset Analysis
    3. Intrusion Detection System Signature Tuning
      1. IDS Signature Tuning Example
    4. Wireless Sniffing
    5. Email Data Analysis
      1. Email Server Analysis
        1. Email Attacks
        2. Open Relay
        3. Large Email Delays
        4. Large Emails
      2. Social Network Analysis
    6. Vulnerability Data Visualization
      1. Risk-Posture Visualization
      2. Vulnerability-Posture Changes
    7. Summary
  12. 7. Compliance
    1. Policies, Objectives, and Controls
    2. Regulations and Industry Mandates
    3. IT Control Frameworks
    4. Logging Requirements
    5. Audit
      1. Audit Data Visualization
    6. Business Process Monitoring
    7. Compliance Monitoring
    8. Risk Management
      1. Control Objective Prioritization
      2. Risk Visualization
    9. Separation of Duties
      1. An Example of Applying Visualization to an SoD Audit
      2. Generating SoD Graphs
    10. Database Monitoring
    11. Summary
  13. 8. Insider Threat
    1. Insider Threat Visualization
    2. What Is a Malicious Insider?
    3. Three Types of Insider Crimes
      1. Information Theft
        1. Data-Leak Prevention Tools
        2. Logging In Data-Leak Prevention Tools
        3. Information-Leak Examples
      2. Fraud
        1. The Fraud Triangle
        2. Fraud-Detection Solutions
      3. Sabotage
    4. Who Are the Malicious Insiders?
      1. Information Theft
      2. Fraudster
      3. Saboteur
    5. A Detection Framework for Malicious Insiders
      1. Precursors
      2. Assigning Scores to Precursors
      3. Insider-Detection Process
        1. 1. Apply Precursors
        2. 2. Visualize the Insider Candidate List
          1. Identifying Groups of Users with Similar Behavior
          2. Augmenting the Analysis with User Roles
          3. Identifying Users with High Scores
        3. 3. Apply the Threshold Filter
        4. 4. Tune the Precursor List
      4. Summary of Insider-Detection Process
      5. Insider-Detection Process at Work
    6. Improved Insider-Detection Process
      1. Watch Lists
      2. Adding Watch Lists to the Insider-Detection Process
      3. Grouping Precursors into Buckets
      4. Candidate Graph Based on Precursor Buckets
      5. Improved Insider-Detection Process Summary
      6. Extended Insider-Detection Process at Work
        1. Insider Candidate Link Graph
        2. Insider Candidate Treemap
    7. Challenges
    8. Proactive Mitigation
    9. Sample Precursors
    10. Summary
  14. 9. Data Visualization Tools
    1. Data Inputs
      1. Comma Separated Values
      2. TM3
      3. DOT
      4. GML
    2. Freely Available Visualization Tools
      1. Static Data Graphs
        1. AfterGlow
        2. GraphViz
        3. Large Graph Layout
        4. Gnuplot
        5. Ploticus
        6. R
      2. Stand-Alone Applications
        1. GGobi
        2. Mondrian
        3. Tulip
        4. Cytoscape
        5. GUESS
        6. Real Time 3D Graph Visualizer
        7. Walrus
        8. Dotty and lneato
        9. Treemap
        10. glTail
        11. Parvis
        12. Shoki (Packet Hustler)
        13. InetVis
        14. TimeSearcher
        15. TNV
        16. NVisionIP
        17. Rumint
        18. Multi Router Traffic Grapher and Round Robin Database
        19. EtherApe
    3. Open Source Visualization Libraries
      1. Java Libraries
      2. Non-Java Libraries
      3. Charting Libraries
    4. Libraries Summary
    5. Online Tools
      1. Swivel
      2. Many Eyes
      3. Google Maps and Google Earth
      4. Google Chart API
    6. Commercial Visualization Tools
      1. Advizor
      2. Other Commercial Visualization Tools
    7. Summary
  15. Applied Security Visualization: Color Gallery