IT Security Metrics: A Practical Framework for Measuring Security & Protecting Data

IT Security Metrics: A Practical Framework for Measuring Security & Protecting Data

by Lance Hayden
Released August 2010
Publisher(s): McGraw-Hill
ISBN: 9780071713412

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Implement an Effective Security Metrics Project or Program

IT Security Metrics provides a comprehensive approach to measuring risks, threats, operational activities, and the effectiveness of data protection in your organization. The book explains how to choose and design effective measurement strategies and addresses the data requirements of those strategies. The Security Process Management Framework is introduced and analytical strategies for security metrics data are discussed. You'll learn how to take a security metrics program and adapt it to a variety of organizational contexts to achieve continuous security improvement over time. Real-world examples of security measurement projects are included in this definitive guide.

  • Define security metrics as a manageable amount of usable data
  • Design effective security metrics
  • Understand quantitative and qualitative data, data sources, and collection and normalization methods
  • Implement a programmable approach to security using the Security Process Management Framework
  • Analyze security metrics data using quantitative and qualitative methods
  • Design a security measurement project for operational analysis of security metrics
  • Measure security operations, compliance, cost and value, and people, organizations, and culture
  • Manage groups of security measurement projects using the Security Improvement Program
  • Apply organizational learning methods to security metrics

Table of contents

  1. Cover Page
  2. It Security Metrics
  3. Copyright Page
  4. Contents
  5. Foreword
  6. Acknowledgments
  7. Introduction
  8. Part I Introducing Security Metrics
    1. 1 What Is a Security Metric?
      1. Metrics and Measurement
        1. Metrics Are a Result
        2. Measurement Is an Activity
      2. Security Metrics Today
        1. Risk
        2. Security Vulnerability and Incident Statistics
        3. Annualized Loss Expectancy
        4. Return on Investment
        5. Total Cost of Ownership
      3. The Dissatisfying State of Security Metrics: Lessons from Other Industries
        1. Insurance
        2. Manufacturing
        3. Design
      4. Reassessing Our Ideas About Security Metrics
        1. Thinking Locally
        2. Thinking Analytically
        3. Thinking Ahead
      5. Summary
      6. Further Reading
    2. 2 Designing Effective Security Metrics
      1. Choosing Good Metrics
        1. Defining Metrics and Measurement
        2. Nothing Either Good or Bad, but Thinking Makes It So
        3. What Do You Want to Know?
        4. Observe!
      2. GQM for Better Security Metrics
        1. What Is GQM?
        2. Setting Goals
        3. Asking Questions
        4. Assigning Metrics
        5. Putting It All Together
        6. The Metrics Catalog
      3. More Security Uses for GQM
        1. Measuring Security Operations
        2. Measuring Compliance to a Regulation or Standard
        3. Measuring People and Culture
        4. Applying GQM to Your Own Security Measurements
      4. Summary
      5. Further Reading
    3. 3 Understanding Data
      1. What Are Data?
        1. Definitions of Data
        2. Data Types
      2. Data Sources for Security Metrics
        1. System Data
        2. Process Data
        3. Documentary Data
        4. People Data
      3. We Have Metrics and Data—Now What?
      4. Summary
      5. Further Reading
    4. Case Study 1: In Search of Enterprise Metrics
      1. Scenario One: Our New Vulnerability Management Program
      2. Scenario Two: Who’s on First?
      3. Scenario Three: The Value of a Slide
      4. Scenario Four: The Monitoring Program
      5. Scenario Five: What Cost, the Truth?
      6. Summary
  9. Part II Implementing Security Metrics
    1. 4 The Security Process Management Framework
      1. Managing Security as a Business Process
        1. Defining a Business Process
        2. Security Processes
        3. Process Management over Time
      2. The SPM Framework
        1. Security Metrics
        2. Security Measurement Projects
        3. The Security Improvement Program
        4. Security Process Management
      3. Before You Begin SPM
        1. Getting Buy-in: Where’s the Forest?
        2. The Security Research Program
      4. Summary
      5. Further Reading
    2. 5 Analyzing Security Metrics Data
      1. The Most Important Step
        1. Reasons for Analysis
        2. What Do You Want to Accomplish?
        3. Preparing for Data Analysis
      2. Analysis Tools and Techniques
        1. Descriptive Statistics
        2. Inferential Statistics
        3. Other Statistical Techniques
        4. Qualitative and Mixed Method Analysis
      3. Summary
      4. Further Reading
    3. 6 Designing the Security Measurement Project
      1. Before the Project Begins
        1. Project Prerequisites
        2. Deciding on a Project Type
        3. Tying Projects Together
        4. Getting Buy-in and Resources
      2. Phase One: Build a Project Plan and Assemble the Team
        1. The Project Plan
        2. The Project Team
      3. Phase Two: Gather the Metrics Data
        1. Collecting Metrics Data
        2. Storing and Protecting Metrics Data
      4. Phase Three: Analyze the Metrics Data and Build Conclusions
      5. Phase Four: Present the Results
        1. Textual Presentations
        2. Visual Presentations
        3. Disseminating the Results
      6. Phase Five: Reuse the Results
      7. Project Management Tools
      8. Summary
      9. Further Reading
    4. Case Study 2: Normalizing Tool Data in a Security Posture Assessment
      1. Background: Overview of the SPA Service
        1. SPA Tools
        2. Data Structures
      2. Objectives of the Case Study
        1. Methodology
        2. Challenges
      3. Summary
  10. PART III Exploring Security Measurement Projects
    1. 7 Measuring Security Operations
      1. Sample Metrics for Security Operations
      2. Sample Measurement Projects for Security Operations
        1. SMP: General Risk Assessment
        2. SMP: Internal Vulnerability Assessment
        3. SMP: Inferential Analysis
      3. Summary
      4. Further Reading
    2. 8 Measuring Compliance and Conformance
      1. The Challenges of Measuring Compliance
        1. Confusion Among Related Standards
        2. Auditing or Measuring?
        3. Confusion Across Multiple Frameworks
      2. Sample Measurement Projects for Compliance and Conformance
        1. Creating a Rationalized Common Control Framework
        2. Mapping Assessments to Compliance Frameworks
        3. Analyzing the Readability of Security Policy Documents
      3. Summary
      4. Further Reading
    3. 9 Measuring Security Cost and Value
      1. Sample Measurement Projects for Compliance and Conformance
        1. Measuring the Likelihood of Reported Personally Identifiable Information (PII) Disclosures
        2. Measuring the Cost Benefits of Outsourcing a Security Incident Monitoring Process
        3. Measuring the Cost of Security Processes
      2. The Importance of Data to Measuring Cost and Value
      3. Summary
      4. Further Reading
    4. 10 Measuring People, Organizations, and Culture
      1. Sample Measurement Projects for People, Organizations, and Culture
        1. Measuring the Security Orientation of Company Stakeholders
        2. An Ethnography of Physical Security Practices
      2. Summary
      3. Further Reading
    5. Case Study 3: Web Application Vulnerabilities
      1. Source Data and Normalization
      2. Outcomes, Timelines, Resources
      3. Initial Reporting with “Dirty Data”
        1. Ambiguous Data
        2. Determining Which Source to Use
      4. Working with Stakeholders to Perform Data Cleansing
      5. Follow-up with Reports and Discussions with Stakeholders
      6. Lesson Learned: Fix the Process, and Then Automate
      7. Lesson Learned: Don’t Wait for Perfect Data Before Reporting
      8. Summary
  11. PART IV Beyond Security Metrics
    1. 11 The Security Improvement Program
      1. Moving from Projects to Programs
      2. Managing Security Measurement with a Security Improvement Program
        1. Governance of Security Measurement
        2. The SIP: It’s Still about the Data
      3. Requirements for a SIP
        1. Before You Begin
        2. Documenting Your Security Measurement Projects
        3. Sharing Your Security Measurement Results
        4. Collaborating Across Projects and Over Time
      4. Measuring the SIP
        1. Security Improvement Is Habit Forming
        2. Is the SIP Working?
        3. Is Security Improving?
      5. Case Study: A SIP for Insider Threat Measurement
      6. Summary
      7. Further Reading
    2. 12 Learning Security: Different Contexts for Security Process Management
      1. Organizational Learning
      2. Three Learning Styles for IT Security Metrics
        1. Standardized Testing: Measurement in ISO/IEC 27004
        2. The School of Life: Basili’s Experience Factory
        3. Mindfulness: Karl Weick and the High-Reliability Organization
      3. Final Thoughts
      4. Summary
      5. Further Reading
    3. Case Study 4: Getting Management Buy-in for the Security Metrics Program
      1. The CISO Hacked My Computer
      2. What Is Buy-in?
      3. Corporations vs. Higher Ed: Who’s Crazier?
      4. Higher Education Case Study
        1. Project Overview
        2. Themes
        3. Findings
        4. Key Points
        5. Influence and Organizational Change
      5. Conclusion
  12. Index

Product information

  • Title: IT Security Metrics: A Practical Framework for Measuring Security & Protecting Data
  • Author(s): Lance Hayden
  • Release date: August 2010
  • Publisher(s): McGraw-Hill
  • ISBN: 9780071713412