You are previewing Troubleshooting OpenStack.
O'Reilly logo
Troubleshooting OpenStack

Book Description

Get unstuck and start stacking!

About This Book

  • Easily fix the nagging problems that commonly plague OpenStack and become the go-to person in your organization

  • Get better equipped to troubleshoot and solve common problems in performance, availability, and automation that confront production-ready OpenStack environments

  • Save time and decrease frustration by solving significant issues that arise from OpenStack deployments pertaining to storage and networking

  • Who This Book Is For

    You will need a basic understanding of OpenStack, Linux, and Cloud computing. If you have an understanding of Linux, this book will help you leverage that knowledge in the world of OpenStack, giving you confidence to tackle most issues that may arise.

    What You Will Learn

  • Diagnose and remediate authentication and authorization problems in Keystone

  • Fix common issues with images served through Glance

  • Master the art of troubleshooting Neutron networking

  • Navigate and overcome problems with Nova

  • Troubleshoot and resolve Cinder block storage issues

  • Identify and correct Swift object storage problems

  • Isolate and fix issues caused by Heat orchestration

  • Leverage Ceilometer and other metering and monitoring tools for effective troubleshooting

  • In Detail

    OpenStack is a collection of software projects that work together to provide a cloud fabric. OpenStack is one of the fastest growing open source projects in history that unlocks cloud computing for everyone. With OpenStack, you are able to create public or private clouds on your own hardware. The flexibility and control afforded by OpenStack puts the cloud within reach of anyone willing to learn this technology.

    Starting with an introduction to OpenStack troubleshooting tools, we’ll walk through each OpenStack service and how you can quickly diagnose, troubleshoot, and correct problems in your OpenStack. Understanding the various projects and how they interact is essential for anyone attempting to troubleshoot an OpenStack cloud. We will start by explaining each of the major components and the dependencies between them, and move on to show you how to identify and utilize an effective set of OpenStack troubleshooting tools and fix common Keystone problems. Next, we will expose you to common errors and problems you may encounter when using the OpenStack Block Storage service (Cinder). We will then examine Heat, the OpenStack Orchestration Service, where you will learn how to trace errors, determine their root cause, and effectively correct the issue.

    Finally, you will get to know the best practices to architect your OpenStack cloud in order to achieve optimal performance, availability, and reliability.

    Style and approach

    This is straight-to-the point guide to fixing your OpenStack cluster. Common problems are identified and suggestions to resolve these problems are presented in a simple, easy-to-understand manner.

    Downloading the example code for this book. You can download the example code files for all Packt books you have purchased from your account at If you purchased this book elsewhere, you can visit and register to have the code file.

    Table of Contents

    1. Troubleshooting OpenStack
      1. Table of Contents
      2. Troubleshooting OpenStack
      3. Credits
      4. About the Author
      5. About the Reviewer
        1. eBooks, discount offers, and more
          1. Why subscribe?
      7. Preface
        1. What this book covers
        2. What you need for this book
        3. Who this book is for
        4. Conventions
        5. Reader feedback
        6. Customer support
          1. Errata
          2. Piracy
          3. Questions
      8. 1. The Troubleshooting Toolkit
        1. The project overview of OpenStack
          1. Keystone
          2. Glance
          3. Neutron
          4. Nova
          5. Cinder
          6. Swift
          7. Heat
          8. Ceilometer
          9. Horizon
          10. Oslo
          11. Documentation
          12. Ironic
          13. Magnum
          14. Trove
          15. Barbican
          16. Congress
          17. Designate
        2. The supporting technologies
          1. Linux
          2. Databases
          3. Message queue
          4. The Apache web server
        3. Basic troubleshooting methodology and tools
          1. General Linux tools
            1. Linux processes
            2. ps
            3. pgrep
            4. pkill
            5. top and htop
            6. Hard drives
            7. df
            8. fdisk
            9. parted
            10. cat /proc/partitions
        4. Installed packages
          1. General tools
            1. The watch command
            2. File tools
            3. Message broker tools
              1. RabbitMQ
        5. Summary
      9. 2. Troubleshooting OpenStack Identity
        1. Know your version
        2. Running Keystone under Eventlet
          1. Checking the Keystone service
          2. Checking the Keystone client
          3. Checking the OpenStack Client
            1. The client debug mode
          4. Checking the API
          5. Keystone process not starting
          6. Database stopped
          7. The service catalog endpoint
          8. Running under WSGI
            1. mod_wsgi
            2. wsgi-keystone.conf
            3. Stopping the Eventlet process
            4. Checking WSGI files
          9. Checking the Keystone service
        3. Summary
      10. 3. Troubleshooting the OpenStack Image Service
        1. Glance services
          1. Confirming the Glance database
          2. Confirming the Glance authentication
            1. Keystone up
            2. Service User Set Up
              1. The service user setup
            3. Service endpoints correct
          3. Confirming the Glance API setup
          4. Checking the command-line interface client
        2. Glance logging and configuration
          1. The log level
          2. Where to look
          3. Searching logs
        3. Common errors
          1. Unable to establish connection
          2. Internal server errors (HTTP 500)
          3. Unable to validate token
        4. Summary
      11. 4. Troubleshooting OpenStack Networking
        1. Identifying Neutron issues
        2. Neutron services and agents
          1. Neutron logs
        3. Common problems
          1. When you can't ping an instance
            1. Security groups
          2. Network namespaces
          3. No IP address
        4. Troubleshooting tools
          1. ovs-vsctl
        5. The Neutron client
        6. Summary
      12. 5. Troubleshooting OpenStack Compute
        1. Checking the services
          1. nova-api
            1. Address already in use
            2. The permission error
          2. nova-scheduler
          3. nova-compute
          4. nova-conductor
        2. Supporting services
          1. The Nova database
          2. Nova authentication
            1. Keystone up
            2. Setting up the service user
            3. Service endpoints correct
          3. Nova and Glance
          4. Nova and Neutron
        3. Summary
      13. 6. Troubleshooting OpenStack Block Storage
        1. Cinder processes
          1. Logging
        2. Cinder dependencies
          1. Keystone authentication problems
          2. RabbitMQ problems
        3. Cinder errors
          1. Missing the cinder-volumes volume group
          2. The volume stuck in the creating state
          3. Insufficient free space
          4. Not sending heartbeat
        4. Summary
      14. 7. Troubleshooting OpenStack Object Storage
        1. Swift processes
          1. The proxy server
        2. Swift authentication
          1. Troubleshooting TempAuth
            1. TempAuth configuration
              1. The account and username
              2. The password
              3. 400 Bad Request
          2. Troubleshooting Swauth
            1. Swauth initialization
          3. Swift with Keystone
            1. Swift users
        3. Summary
      15. 8. Troubleshooting the OpenStack the Orchestration Service
        1. Heat services
          1. Running heat-api
          2. Running heat-engine
        2. Heat authentication
          1. The Keystone service
          2. Auth credentials
        3. Heat template errors
        4. Summary
      16. 9. Troubleshooting the OpenStack Telemetry Service
        1. Ceilometer processes
        2. Ceilometer authentication
        3. Ceilometer dependencies
          1. The message broker
          2. Databases
        4. The command-line client
          1. The meter list command
            1. Glance meters
            2. Nova meters
          2. The sample list
        5. Summary
      17. 10. OpenStack Performance, Availability, and Reliability
        1. Databases
          1. Availability
            1. MySQL with Galera Cluster
            2. Postgres
          2. Performance
            1. MySQL
              1. Show status
              2. Mytop
                1. The Mytop header section
                2. The Mytop thread section
              3. Percona Toolkit
            2. Postgres
              1. The PostgreSQL statistics collector
          3. Database backups
          4. Monitoring
            1. Resource monitoring
            2. OpenStack quotas
        2. RabbitMQ
          1. Clustering
          2. Mirrored queues
        3. Services
          1. Monitoring service processes
          2. Backing up services
        4. Community resources
          1. Testing
          2. Bugs
        5. Summary
      18. Index