You are previewing Splunk Operational Intelligence Cookbook.
O'Reilly logo
Splunk Operational Intelligence Cookbook

Book Description

Over 70 practical recipes to gain operational data intelligence with Splunk Enterprise

In Detail

This book contains over 70 practical, task-oriented recipes to build up your knowledge of Splunk's many features which you can apply to real-world operational intelligence scenarios.

Right from the first chapter, you will follow recipes that progressively build upon one another. The recipes provided will demonstrate methods to expedite delivery of intelligence reports and empower you to present data in a meaningful way through dashboards and by applying visualizations available in Splunk Enterprise. You will also delve deeply into your data with transactions, subsearching, concurrency, and more advanced search commands.

What You Will Learn

  • Search, report on, and visualize operational intelligence data
  • Enrich operational data with lookups and workflows
  • Model and accelerate data and perform pivot-based reporting
  • Build real-time, scripted, and other intelligence-driven alerts
  • Summarize data for longer term trending, reporting, and analysis
  • Build a fully featured Splunk operational intelligence application
  • Integrate advanced JavaScript charts and leverage Splunk's API
  • Downloading the example code for this book. You can download the example code files for all Packt books you have purchased from your account at http://www.PacktPub.com. If you purchased this book elsewhere, you can visit http://www.PacktPub.com/support and register to have the files e-mailed directly to you.

    Table of Contents

    1. Splunk Operational Intelligence Cookbook
      1. Table of Contents
      2. Splunk Operational Intelligence Cookbook
      3. Credits
      4. About the Authors
      5. About the Reviewers
      6. www.PacktPub.com
        1. Support files, eBooks, discount offers, and more
          1. Why subscribe?
          2. Free access for Packt account holders
      7. Preface
        1. What this book covers
        2. What you need for this book
        3. Who this book is for
        4. Conventions
        5. Reader feedback
        6. Customer support
          1. Downloading the example code
          2. Errata
          3. Piracy
          4. Questions
      8. 1. Play Time – Getting Data In
        1. Introduction
        2. Indexing files and directories
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Adding a file or directory data input via the CLI
            2. Adding a file or directory input via inputs.conf
            3. One-time indexing of data files via the Splunk CLI
            4. Indexing the Windows event logs
          5. See also
        3. Getting data through network ports
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Adding a network input via the CLI
            2. Adding a network input via inputs.conf
          5. See also
        4. Using scripted inputs
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. See also
        5. Using modular inputs
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
          5. See also
        6. Using the Universal Forwarder to gather data
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Add the receiving indexer via outputs.conf
        7. Loading the sample data for this book
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. See also
        8. Defining field extractions
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. See also
        9. Defining event types and tags
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Adding event types and tags via eventtypes.conf and tags.conf
          5. See also
        10. Summary
      9. 2. Diving into Data – Search and Report
        1. Introduction
        2. Making raw event data readable
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Tabulating every field
            2. Removing fields, then tabulating everything else
        3. Finding the most accessed web pages
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Searching for the top 10 accessed web pages
            2. Searching for the most accessed pages by user
          5. See also
        4. Finding the most used web browsers
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more…
            1. Searching the web browser data for the most used OS types
          5. See also
        5. Identifying the top-referring websites
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more…
            1. Searching for the top 10 referring websites using stats instead of top
          5. See also
        6. Charting web page response codes
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Totaling success and error web page response codes
          5. See also
        7. Displaying web page response time statistics
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Displaying web page response time by action
          5. See also
        8. Listing the top viewed products
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Searching for the percentage of cart additions from product views
          5. See also
        9. Charting the application's functional performance
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
          5. See also
        10. Charting the application's memory usage
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. See also
        11. Counting the total number of database connections
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. See also
        12. Summary
      10. 3. Dashboards and Visualizations – Make Data Shine
        1. Introduction
        2. Creating an Operational Intelligence dashboard
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Changing dashboard permissions
        3. Using a pie chart to show the most accessed web pages
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Searching for the top 10 accessed web pages
          5. See also
        4. Displaying the unique number of visitors
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more…
            1. Adding labels to a single value panel
            2. Coloring the value based on ranges
          5. See also
        5. Using a gauge to display the number of errors
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more…
          5. See also
        6. Charting the number of method requests by type and host
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. See also
        7. Creating a timechart of method requests, views, and response times
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Method requests, views, and response times by host
          5. See also
        8. Using a scatter chart to identify discrete requests by size and response time
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Using time series data points with a scatter chart
          5. See also
        9. Creating an area chart of the application's functional statistics
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. See also
        10. Using a bar chart to show the average amount spent by category
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. See also
        11. Creating a line chart of item views and purchases over time
          1. Getting ready
          2. How to do it…
          3. How it works...
          4. See also
        12. Summary
      11. 4. Building an Operational Intelligence Application
        1. Introduction
        2. Creating an Operational Intelligence application
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Creating an application from another application
            2. Downloading and installing a Splunk app
          5. See also
        3. Adding dashboards and reports
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more…
            1. Changing the permissions of saved reports
          5. See also
        4. Organizing the dashboards more efficiently
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more…
            1. Modifying the SimpleXML directly
          5. See also
        5. Dynamically drilling down on activity reports
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more…
            1. Disabling the drilldown feature in tables and charts
          5. See also
        6. Creating a form to search web activities
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Adding a Submit button to your form
          5. See also
        7. Linking web page activity reports to the form
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Adding an overlay to the Sessions Over Time chart
          5. See also
        8. Displaying a geographical map of visitors
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Adding a map panel using SimpleXML
            2. Mapping different distributions by area
          5. See also
        9. Scheduling the PDF delivery of a dashboard
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. See also
        10. Summary
      12. 5. Extending Intelligence – Data Models and Pivoting
        1. Introduction
        2. Creating a data model for web access logs
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Searching data models using the search interface
          5. See also
        3. Creating a data model for application logs
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. See also
        4. Accelerating data models
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Viewing data model and acceleration summary information
            2. Advanced configuration of data model acceleration
          5. See also
        5. Pivoting total sales transactions
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Pivot searching using the pivot command and search interface
          5. See also
        6. Pivoting purchases by geographical location
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. See also
        7. Pivoting slowest responding web pages
          1. Getting ready
          2. How to do it...
          3. How it works…
          4. See also
        8. Pivot charting top error codes
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. See also
        9. Summary
      13. 6. Diving Deeper – Advanced Searching
        1. Introduction
        2. Calculating the average session time on a website
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Starts with a website visit, ends with a checkout
            2. Defining maximum pause, span, and events in a transaction
          5. See also
        3. Calculating the average execution time for multi-tier web requests
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more…
            1. Calculating the average execution time without using a join
          5. See also
        4. Displaying the maximum concurrent checkouts
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. See also
        5. Analyzing the relationship of web requests
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more…
            1. Analyzing relationships of DB actions to memory utilization
          5. See also
        6. Predicting website-traffic volumes
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more…
            1. Predicting the total number of items purchased
            2. Predicting the average response time of function calls
          5. See also
        7. Finding abnormally sized web requests
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. The anomalies command
            2. The anomalousvalues command
            3. The cluster command
          5. See also
        8. Identifying potential session spoofing
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Creating logic for urgency
          5. See also
        9. Summary
      14. 7. Enriching Data – Lookups and Workflows
        1. Introduction
          1. Lookups
        2. Looking up product code descriptions
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Manually adding the lookup to Splunk
          5. See also
        3. Flagging suspicious IP addresses
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Modifying an existing saved search to populate a lookup table
          5. See also
        4. Creating a session state table
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. See also
        5. Adding hostnames to IP addresses
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more…
            1. Enabling automatic external field lookups
          5. See also
        6. Searching ARIN for a given IP address
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Limiting workflow actions by event types
          5. See also
        7. Triggering a Google search for a given error
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Triggering a Google search from the chart drilldown options
          5. See also
        8. Creating a ticket for application errors
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Adding a workflow action manually in Splunk
          5. See also
        9. Looking up inventory from an external database
          1. Getting ready
          2. How to do it…
          3. How it works...
          4. There's more...
            1. Use DB Connect for direct external DB lookups
          5. See also
        10. Summary
      15. 8. Being Proactive – Creating Alerts
        1. Introduction
        2. Alerting on abnormal web page response times
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Viewing triggered alerts in Splunk's Alert manager
          5. See also
        3. Alerting on errors during checkout in real time
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Building alerts via a configuration file
            2. Identify the real-time searches that are running
          5. See also
        4. Alerting on abnormal user behavior
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Alerting on abnormal user purchases without checkouts
          5. See also
        5. Alerting on failure and triggering a scripted response
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more…
          5. See also
        6. Alerting when predicted sales exceed inventory
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more…
            1. Adding an RSS feed notification action to an alert
          5. See also
        7. Summary
      16. 9. Speed Up Intelligence – Data Summarization
        1. Introduction
        2. Calculating an hourly count of sessions versus completed transactions
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Generating the summary more frequently
            2. Avoiding summary index overlaps and gaps
          5. See also
        3. Backfilling the number of purchases by city
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Backfilling a summary index from within a search directly
          5. See also
        4. Displaying the maximum number of concurrent sessions over time
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Viewing the status of an accelerated report
          5. See also
        5. Summary
      17. 10. Above and Beyond – Customization, Web Framework, REST API, and SDKs
        1. Introduction
        2. Customizing the application's navigation
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more…
        3. Adding a force-directed graph of web hits
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more…
            1. Changing the time range on the search manager
          5. See also
        4. Adding a calendar heatmap of product purchases
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. See also
        5. Remotely querying Splunk's REST API for unique page views
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more…
            1. Authenticating with a session token
          5. See also
        6. Creating a Python application to return unique IP addresses
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Paginating the results of your search
          5. See also
        7. Creating a custom search command to format product names
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. See also
        8. Summary
      18. Index