You are previewing DevOps Automation Cookbook.
O'Reilly logo
DevOps Automation Cookbook

Book Description

Over 120 recipes covering key automation techniques through code management and virtualization offered by modern Infrastructure as a Service

About This Book

  • Use some of the powerful tools that have emerged to enable systems administrators and developers to take control and automate the management, monitoring, and creation of complex infrastructures

  • Covers some of the most exciting technologies available to DevOps engineers, and demonstrates multiple techniques for using them

  • A hands-on guide filled with clear examples and multiple methodologies to demonstrate the various DevOps tools available

  • Who This Book Is For

    If you are a systems administrator or developer who is keen to employ DevOps techniques to help with the day-to-day complications of managing complex infrastructures, then this book is for you. No prior knowledge of DevOps tools is required.

    What You Will Learn

  • Manage, use, and work with code in the Git version management system

  • Manage the life cycle of hosts, from creation to on-going management, using Puppet Razor

  • Create hosts automatically using a simple combination of TFTP, DHCP, and pre-seeds

  • Implement virtual hosts using the ubiquitous VMware ESXi hypervisor

  • Control configuration using the powerful and popular Ansible configuration management system

  • Develop powerful, consistent, and portable containers using Docker

  • Track trends, discover data, and monitor key systems using InfluxDB, syslog, and Sensu

  • Deal efficiently with powerful cloud infrastructures using the Amazon AWS Infrastructure as a Service and the Heroku Platform as a Service

  • In Detail

    There has been a recent explosion in tools that allow you to redefine the delivery of infrastructure and applications, using a combination of automation and testing to deliver continuous deployment. DevOps has garnered interest from every quarter, and is rapidly being recognized as a radical shift, as large as the Agile movement for the delivery of software.

    This book takes a collection of some of the coolest software available today and shows you how to use it to create impressive changes to the way you deliver applications and software. It tackles the plethora of tools that are now available to enable organizations to take advantage of the automation, monitoring, and configuration management techniques that define a DevOps-driven infrastructure.

    Starting off with the fundamental command-line tools that every DevOps enthusiast must know, this book will guide you through the implementation of the Ansible tool to help you facilitate automation and perform diverse tasks. You will explore how to build hosts automatically with the creation of Apt mirrors and interactive pre-seeds, which are of the utmost importance for Ubuntu automation. You will also delve into the concept of virtualization and creating and manipulating guests with ESXi. Following this, you will venture into the application of Docker; learn how to install, run, network, and restore Docker containers; and also learn how to build containers in Jenkins and deploy apps using a combination of Ansible, Docker, and Jenkins. You will also discover how to filter data with Grafana and the usage of InfluxDB along with unconventional log management. Finally, you will get acquainted with cloud infrastructure, employing the Heroku and Amazon AWS platforms.

    By tackling real-world issues, this book will guide you through a huge variety of tools, giving new users the ability to get up and running and offering advanced users some interesting recipes that may help with existing issues.

    Style and approach

    This book has an easy-to-follow approach to both introducing the wide range of available DevOps tools and working with them. Each recipe is lavishly sprinkled with example code and uses clear instructions to easily allow you to see how you can both implement the solutions, and more importantly, tailor them to your unique needs.

    Downloading the example code for this book. You can download the example code files for all Packt books you have purchased from your account at http://www.PacktPub.com. If you purchased this book elsewhere, you can visit http://www.PacktPub.com/support and register to have the code file.

    Table of Contents

    1. DevOps Automation Cookbook
      1. Table of Contents
      2. DevOps Automation Cookbook
      3. Credits
      4. About the Author
      5. About the Reviewers
      6. www.PacktPub.com
        1. Support files, eBooks, discount offers, and more
          1. Why Subscribe?
          2. Free Access for Packt account holders
      7. Preface
        1. What this book covers
        2. What you need for this book
        3. Who this book is for
        4. Sections
          1. Getting ready
          2. How to do it…
          3. How it works…
          4. There's more…
          5. See also
        5. Conventions
        6. Reader feedback
        7. Customer support
          1. Downloading the example code
          2. Downloading the color images of this book
          3. Errata
          4. Piracy
          5. Questions
      8. 1. Basic Command Line Tools
        1. Introduction
        2. Controlling network interfaces
          1. Getting ready
          2. How to do it…
          3. See also
        3. Monitoring network details with the IP command
          1. Getting ready
          2. How to do it…
        4. Monitoring connections using the ss command
          1. Getting ready
          2. How to do it…
        5. Gathering basic OS statistics
          1. Getting ready
          2. How to do it…
        6. Viewing historical resource usage with SAR
          1. Getting ready
          2. How to do it…
        7. Installing and configuring a Git client
          1. Getting ready
          2. How to do it…
        8. Creating an SSH key for Git
          1. Getting ready
          2. How to do it
          3. How it works…
        9. Using ssh-copy-id to copy keys
          1. Getting ready
          2. How to do it…
          3. How it works…
          4. See also
        10. Creating a new Git repository
          1. Getting ready
          2. How to do it…
          3. How it works…
          4. See also
        11. Cloning an existing Git repository
          1. Getting ready
          2. How to do it…
          3. How it works…
          4. See also
        12. Checking changes into a Git repository
          1. Getting ready
          2. How to do it…
          3. How it works…
          4. See also
        13. Pushing changes to a Git remote
          1. Getting ready
          2. How to do it…
          3. How it works…
          4. See also
        14. Creating a Git branch
          1. Getting ready
          2. How to do it…
          3. How it works…
          4. See also
      9. 2. Ad Hoc Tasks with Ansible
        1. Introduction
        2. Installing an Ansible control node on Ubuntu
          1. Getting ready
          2. How to do it…
          3. See also
        3. Installing an Ansible control node on CentOS
          1. Getting ready
          2. How to do it…
          3. See also
        4. Creating an Ansible inventory
          1. Getting ready
          2. How to do it…
          3. See also
        5. Using the raw module to install python-simplejson
          1. Getting ready
          2. How to do it…
          3. See also
        6. Installing packages with Ansible
          1. Getting ready
          2. How to do it...
          3. See also
        7. Restarting services using Ansible
          1. Getting ready
          2. How to do it…
          3. See also
        8. Executing freeform commands with Ansible
          1. Getting ready
          2. How to do it…
        9. Managing users with Ansible
          1. Getting ready
          2. How to do it…
          3. See also
        10. Managing SSH keys with Ansible
          1. Getting ready
          2. How to do it...
          3. See also
      10. 3. Automatic Host builds
        1. Introduction
        2. Creating an Apt mirror using aptly
          1. Getting ready
          2. How to do it…
          3. See also
        3. Automated installation using PXE boot and a Preseed file
          1. Getting ready
          2. How to do it…
          3. See also
        4. Automating post-installation tasks
          1. Getting ready
          2. How to do it…
      11. 4. Virtualization with VMware ESXi
        1. Introduction
        2. Installing ESXi
          1. Getting ready
          2. How to do it…
        3. Installing and using the vSphere Client
          1. Getting ready
          2. How to do it….
        4. Allowing SSH access to ESXi
          1. Getting ready
          2. How to do it…
        5. Creating a new guest
          1. Getting ready
          2. How to do it...
        6. Allocating resources to a guest
          1. Getting ready
          2. How to do it…
        7. Using the ESXi command line to start, stop, and destroy guests
          1. Getting ready
          2. How to do it…
        8. Managing command-line snapshots
          1. Getting ready
          2. How to do it…
        9. Tuning the host for guest performance
          1. Getting ready
          2. How to do it…
          3. See also
      12. 5. Automation with Ansible
        1. Introduction
        2. Installing Ansible
          1. Getting ready
          2. How to do it…
          3. See also
        3. Creating a scaffold Playbook
          1. Getting ready
          2. How to do it
        4. Creating a common role
          1. Getting ready
          2. How to do it…
          3. See also
        5. Creating a webserver using Ansible and Nginx
          1. Getting ready
          2. How to do it…
          3. See also
        6. Creating an application server role using Tomcat and Ansible
          1. Getting ready
          2. How to do it…
          3. See also
        7. Installing MySQL using Ansible
          1. Getting ready
          2. How to do it…
          3. See also
        8. Installing and managing HAProxy with Ansible
          1. Getting ready
          2. How to do it…
          3. See also
        9. Using ServerSpec to test your Playbook
          1. Getting ready
          2. How to do it…
          3. See also
      13. 6. Containerization with Docker
        1. Introduction
        2. Installing Docker
          1. Getting ready
          2. How to do it…
          3. See also
        3. Pulling an image from the public Docker registry
          1. Getting ready
          2. How to do it…
          3. See also
        4. Performing basic Docker operations
          1. Getting ready
          2. How to do it…
          3. See also
        5. Running a container interactively
          1. Getting ready
          2. How to do it…
          3. See also
        6. Creating a Dockerfile
          1. Getting ready
          2. How to do it…
          3. See also
        7. Running a container in detached mode
          1. Getting ready
          2. How to do it…
          3. See also
        8. Saving and restoring a container
          1. Getting ready
          2. How to do it…
          3. See also
        9. Using the host only network
          1. Getting ready
          2. How to do it…
          3. See also
        10. Running a private Docker registry
          1. Getting ready
          2. How to do it
          3. See also
        11. Managing images with a private registry
          1. Getting ready
          2. How to do it…
            1. Pushing images
            2. Pulling images
          3. See also
      14. 7. Using Jenkins for Continuous Deployment
        1. Introduction
        2. Installing Jenkins
          1. Getting ready
          2. How to do it…
          3. See also…
        3. Installing the Git plugin
          1. Getting ready
          2. How to do it…
          3. See also
        4. Installing a Jenkins slave
          1. Getting ready
          2. How to do it…
          3. See also
        5. Creating your first Jenkins job
          1. Getting ready
          2. How to do it…
          3. See also
        6. Building Docker containers using Jenkins
          1. Getting ready
          2. How to do it…
        7. Deploying a Java application to Tomcat with zero downtime using Ansible
          1. Getting ready
          2. How to do it…
          3. See also
      15. 8. Metric Collection with InfluxDB
        1. Introduction
        2. Installing InfluxDB
          1. Getting ready
          2. How to do it…
          3. See also
        3. Creating a new InfluxDB database
          1. Getting ready
          2. How to do it…
          3. See also
        4. Logging events with the InfluxDB REST API
          1. Getting ready
          2. How to do it…
          3. See also
        5. Gathering host statistics with Telegraf
          1. Getting ready
          2. How to do it…
          3. See also
        6. Exploring data with the InfluxDB data explorer
          1. Getting ready
          2. How to do it…
          3. See also
        7. Installing Grafana
          1. Getting ready…
          2. How to do it…
          3. See also
        8. Creating dashboards with Grafana
          1. Getting ready
          2. How to do it…
          3. See also
      16. 9. Log Management
        1. Introduction
        2. Centralizing logs with Syslog
          1. Getting ready
          2. How to do it…
          3. See also
        3. Using syslog templates
          1. Getting ready
          2. How to do it…
          3. See also
        4. Managing log rotation with the Logrotate utility
          1. Getting ready
          2. How to do it…
          3. See also
        5. Installing ElasticSearch, Logstash, and Kibana
          1. Getting ready
          2. How to do it…
          3. See also
        6. Importing logs into Elasticsearch with Logstash
          1. Getting ready
          2. How to do it…
          3. See also
        7. Using Kibana queries to explore data
          1. Getting ready
          2. How to do it…
          3. See also
        8. Using Kibana queries to examine data
          1. Getting ready
          2. How to do it…
          3. See also
      17. 10. Monitoring with Sensu
        1. Introduction
        2. Installing a Sensu server
          1. Getting ready
          2. How to do it…
          3. See also
        3. Installing a Sensu client
          1. Getting ready
          2. How to do it…
          3. See also
        4. Installing check prerequisites
          1. Getting ready
          2. How to do it…
        5. Finding community checks
          1. Getting ready
          2. How to do it…
          3. See also
        6. Adding a DNS check
          1. Getting ready
          2. How to do it…
          3. See also
        7. Adding a disk check
          1. Getting ready
          2. How to do it…
          3. See also
        8. Adding a RAM check
          1. Getting ready
          2. How to do it…
          3. See also
        9. Adding a process check
          1. Getting ready…
          2. How to do it…
          3. See also
        10. Adding a CPU check
          1. Getting ready
          2. How to do it…
          3. See also
        11. Creating e-mail alerts
          1. Getting ready
          2. How to do it…
          3. See also
        12. Creating SMS alerts
          1. Getting ready
          2. How to do it…
          3. See also
        13. Using Ansible to install Sensu
          1. Getting ready
          2. How to do it…
          3. See also
      18. 11. IAAS with Amazon AWS
        1. Introduction
        2. Signing up for AWS
          1. Getting ready
          2. How to do it…
          3. See also
        3. Setting up IAM
          1. Getting ready
          2. How to do it…
          3. See also
        4. Creating your first security group
          1. Getting ready
          2. How to do it…
          3. See also
        5. Creating your first EC2 host
          1. Getting ready
          2. How to do it…
          3. See also
        6. Using Elastic Load Balancers
          1. Getting ready
          2. How to do it…
          3. See also
        7. Managing DNS with route53
          1. Getting ready…
          2. How to do it…
          3. See also
        8. Using Ansible to create EC2 hosts
          1. Getting ready
          2. How to do it…
          3. See also
      19. 12. Application Performance Monitoring with New Relic
        1. Introduction
        2. Signing up for a New Relic account
          1. Getting ready
          2. How to do it…
          3. See also
        3. Installing the New Relic Java agent
          1. Getting ready
          2. How to do it…
          3. See also
        4. Using the performance overview
          1. Getting ready
          2. How to do it…
          3. See also
        5. Locating performance bottlenecks with Transaction Traces
          1. Getting ready
          2. How to do it…
          3. See also
        6. Observing database performance with New Relic
          1. Getting ready
          2. How to do it…
          3. See also
        7. Release performance monitoring with New Relic
          1. Getting ready
          2. How to do it…
          3. See also
        8. Server Monitoring with New Relic
          1. Getting ready
          2. How to do it…
          3. See also
      20. Index