You are previewing Learning DevOps: Continuously Deliver Better Software.
O'Reilly logo
Learning DevOps: Continuously Deliver Better Software

Book Description

Learn to use some of the most exciting and powerful tools to deliver world-class quality software with continuous delivery and DevOps

About This Book

  • Get to know the background of DevOps so you understand the collaboration between different aspects of an IT organization and a software developer

  • Deploy top-quality software and ensure software maintenance and release management with this practical guide

  • This course covers some of the most exciting technology available to DevOps engineers, and demonstrates multiple techniques for using them

  • Real-world and realistic examples are provided to help you as you go about the implementation and adoption of continuous delivery and DevOps

  • Who This Book Is For

    This course is for developers who want to understand how the infrastructure that builds today's enterprises works, and how to painlessly and regularly ship quality software.

    What You Will Learn

  • Set up and familiarize yourself with all the tools you need to be efficient with DevOps

  • Design an application that is suitable for continuous deployment systems with DevOps in mind

  • Test the code using automated regression testing with Jenkins Selenium

  • Managing the lifecycle of hosts, from creation to ongoing management using Puppet Razor

  • Find out how to manage, use, and work with Code in the Git version management system

  • See what traps, pitfalls, and hurdles to look out for as you implement continuous delivery and DevOps

  • In Detail

    Harness the power of DevOps to boost your skill set and make your IT organization perform better. If you’re keen to employ DevOps techniques to better your software development, this course contains all you need to overcome the day-to-day complications of managing complex infrastructures the DevOps way.

    Start with your first module – Practical DevOps - that encompasses the entire flow from code from testing to production. Get a solid ground-level knowledge of how to monitor code for any anomalies, perform code testing, and make sure the code is running smoothly through a series of real-world exercise, and develop practical skills by creating a sample enterprise Java application.

    In the second module, run through a series of tailored mini-tutorials designed to give you a complete understanding of every DevOps automation technique. Create real change in the way you deliver your projects by utilizing some of the most commendable software available today. Go from your first steps of managing code in Git to configuration management in Puppet, monitoring using Sensu, and more.

    In the final module, get to grips with the continuous delivery techniques that will help you reduce the time and effort that goes into the delivery and support of software.

    This Learning Path combines some of the best that Packt has to offer in one complete, curated package. It includes content from the following Packt products:

  • Practical DevOps by Joakim Verona

  • DevOps Automation Cookbook by Michael Duffy

  • Continuous Delivery and DevOps : A Quickstart Guide - Second Edition by Paul Swartout

  • Style and approach

    This course is an easy to follow project based guide for all those with a keen interest in deploying world-class software using some of the most effective and remarkable technologies available.

    Downloading the example code for this book You can download the example code files for all Packt books you have purchased from your account at http://www.PacktPub.com. If you purchased this book elsewhere, you can visit http://www.PacktPub.com/support and register to have the files e-mailed directly to you.

    Table of Contents

    1. Learning DevOps: Continuously Deliver Better Software
      1. Table of Contents
      2. Learning DevOps: Continuously Deliver Better Software
      3. Learning DevOps: Continuously Deliver Better Software
      4. Credits
      5. Preface
        1. What this learning path covers
        2. What you need for this learning path
        3. Who this learning path is for
        4. Reader feedback
        5. Customer support
          1. Downloading the example code
          2. Errata
          3. Piracy
          4. Questions
      6. I. Module 1
        1. 1. Introduction to DevOps and Continuous Delivery
          1. Introducing DevOps
          2. How fast is fast?
          3. The Agile wheel of wheels
          4. Beware the cargo cult Agile fallacy
          5. DevOps and ITIL
          6. Summary
        2. 2. A View from Orbit
          1. The DevOps process and Continuous Delivery – an overview
            1. The developers
            2. The revision control system
            3. The build server
            4. The artifact repository
            5. Package managers
            6. Test environments
            7. Staging/production
          2. Release management
          3. Scrum, Kanban, and the delivery pipeline
          4. Wrapping up – a complete example
          5. Identifying bottlenecks
          6. Summary
        3. 3. How DevOps Affects Architecture
          1. Introducing software architecture
          2. The monolithic scenario
          3. Architecture rules of thumb
          4. The separation of concerns
          5. The principle of cohesion
          6. Coupling
          7. Back to the monolithic scenario
          8. A practical example
          9. Three-tier systems
          10. The presentation tier
          11. The logic tier
          12. The data tier
          13. Handling database migrations
          14. Rolling upgrades
          15. Hello world in Liquibase
          16. The changelog file
          17. The pom.xml file
          18. Manual installation
          19. Microservices
          20. Interlude – Conway's Law
          21. How to keep service interfaces forward compatible
          22. Microservices and the data tier
          23. DevOps, architecture, and resilience
          24. Summary
        4. 4. Everything is Code
          1. The need for source code control
          2. The history of source code management
          3. Roles and code
          4. Which source code management system?
          5. A word about source code management system migrations
          6. Choosing a branching strategy
          7. Branching problem areas
          8. Artifact version naming
          9. Choosing a client
          10. Setting up a basic Git server
          11. Shared authentication
          12. Hosted Git servers
          13. Large binary files
          14. Trying out different Git server implementations
          15. Docker intermission
          16. Gerrit
            1. Installing the git-review package
            2. The value of history revisionism
          17. The pull request model
          18. GitLab
          19. Summary
        5. 5. Building the Code
          1. Why do we build code?
          2. The many faces of build systems
          3. The Jenkins build server
          4. Managing build dependencies
          5. The final artifact
          6. Cheating with FPM
          7. Continuous Integration
          8. Continuous Delivery
          9. Jenkins plugins
          10. The host server
          11. Build slaves
          12. Software on the host
          13. Triggers
          14. Job chaining and build pipelines
          15. A look at the Jenkins filesystem layout
          16. Build servers and infrastructure as code
            1. Building by dependency order
          17. Build phases
          18. Alternative build servers
          19. Collating quality measures
          20. About build status visualization
          21. Taking build errors seriously
          22. Robustness
          23. Summary
        6. 6. Testing the Code
          1. Manual testing
          2. Pros and cons with test automation
          3. Unit testing
          4. JUnit in general and JUnit in particular
            1. A JUnit example
          5. Mocking
          6. Test Coverage
          7. Automated integration testing
            1. Docker in automated testing
            2. Arquillian
          8. Performance testing
          9. Automated acceptance testing
          10. Automated GUI testing
          11. Integrating Selenium tests in Jenkins
          12. JavaScript testing
          13. Testing backend integration points
          14. Test-driven development
          15. REPL-driven development
          16. A complete test automation scenario
            1. Manually testing our web application
            2. Running the automated test
            3. Finding a bug
            4. Test walkthrough
            5. Handling tricky dependencies with Docker
          17. Summary
        7. 7. Deploying the Code
          1. Why are there so many deployment systems?
            1. Configuring the base OS
            2. Describing clusters
            3. Delivering packages to a system
          2. Virtualization stacks
          3. Executing code on the client
            1. A note about the exercises
          4. The Puppet master and Puppet agents
          5. Ansible
          6. PalletOps
          7. Deploying with Chef
          8. Deploying with SaltStack
          9. Salt versus Ansible versus Puppet versus PalletOps execution models
          10. Vagrant
          11. Deploying with Docker
          12. Comparison tables
          13. Cloud solutions
          14. AWS
          15. Azure
          16. Summary
        8. 8. Monitoring the Code
          1. Nagios
          2. Munin
          3. Ganglia
          4. Graphite
          5. Log handling
            1. Client-side logging libraries
            2. The ELK stack
          6. Summary
        9. 9. Issue Tracking
          1. What are issue trackers used for?
          2. Some examples of workflows and issues
          3. What do we need from an issue tracker?
          4. Problems with issue tracker proliferation
          5. All the trackers
            1. Bugzilla
            2. Trac
            3. Redmine
            4. The GitLab issue tracker
            5. Jira
          6. Summary
        10. 10. The Internet of Things and DevOps
          1. Introducing the IoT and DevOps
          2. The future of the IoT according to the market
          3. Machine-to-machine communication
          4. IoT deployment affects software architecture
          5. IoT deployment security
          6. Okay, but what about DevOps and the IoT again?
          7. A hands-on lab with an IoT device for DevOps
          8. Summary
      7. II. Module 2
        1. 1. Basic Command Line Tools
          1. Introduction
          2. Controlling network interfaces
            1. Getting ready
            2. How to do it…
            3. See also
          3. Monitoring network details with the IP command
            1. Getting ready
            2. How to do it…
          4. Monitoring connections using the ss command
            1. Getting ready
            2. How to do it…
          5. Gathering basic OS statistics
            1. Getting ready
            2. How to do it…
          6. Viewing historical resource usage with SAR
            1. Getting ready
            2. How to do it…
          7. Installing and configuring a Git client
            1. Getting ready
            2. How to do it…
          8. Creating an SSH key for Git
            1. Getting ready
            2. How to do it
            3. How it works…
          9. Using ssh-copy-id to copy keys
            1. Getting ready
            2. How to do it…
            3. How it works…
            4. See also
          10. Creating a new Git repository
            1. Getting ready
            2. How to do it…
            3. How it works…
            4. See also
          11. Cloning an existing Git repository
            1. Getting ready
            2. How to do it…
            3. How it works…
            4. See also
          12. Checking changes into a Git repository
            1. Getting ready
            2. How to do it…
            3. How it works…
            4. See also
          13. Pushing changes to a Git remote
            1. Getting ready
            2. How to do it…
            3. How it works…
            4. See also
          14. Creating a Git branch
            1. Getting ready
            2. How to do it…
            3. How it works…
            4. See also
        2. 2. Ad Hoc Tasks with Ansible
          1. Introduction
          2. Installing an Ansible control node on Ubuntu
            1. Getting ready
            2. How to do it…
            3. See also
          3. Installing an Ansible control node on CentOS
            1. Getting ready
            2. How to do it…
            3. See also
          4. Creating an Ansible inventory
            1. Getting ready
            2. How to do it…
            3. See also
          5. Using the raw module to install python-simplejson
            1. Getting ready
            2. How to do it…
            3. See also
          6. Installing packages with Ansible
            1. Getting ready
            2. How to do it...
            3. See also
          7. Restarting services using Ansible
            1. Getting ready
            2. How to do it…
            3. See also
          8. Executing freeform commands with Ansible
            1. Getting ready
            2. How to do it…
          9. Managing users with Ansible
            1. Getting ready
            2. How to do it…
            3. See also
          10. Managing SSH keys with Ansible
            1. Getting ready
            2. How to do it...
            3. See also
        3. 3. Automatic Host builds
          1. Introduction
          2. Creating an Apt mirror using aptly
            1. Getting ready
            2. How to do it…
            3. See also
          3. Automated installation using PXE boot and a Preseed file
            1. Getting ready
            2. How to do it…
            3. See also
          4. Automating post-installation tasks
            1. Getting ready
            2. How to do it…
        4. 4. Virtualization with VMware ESXi
          1. Introduction
          2. Installing ESXi
            1. Getting ready
            2. How to do it…
          3. Installing and using the vSphere Client
            1. Getting ready
            2. How to do it….
          4. Allowing SSH access to ESXi
            1. Getting ready
            2. How to do it…
          5. Creating a new guest
            1. Getting ready
            2. How to do it...
          6. Allocating resources to a guest
            1. Getting ready
            2. How to do it…
          7. Using the ESXi command line to start, stop, and destroy guests
            1. Getting ready
            2. How to do it…
          8. Managing command-line snapshots
            1. Getting ready
            2. How to do it…
          9. Tuning the host for guest performance
            1. Getting ready
            2. How to do it…
            3. See also
        5. 5. Automation with Ansible
          1. Introduction
          2. Installing Ansible
            1. Getting ready
            2. How to do it…
            3. See also
          3. Creating a scaffold Playbook
            1. Getting ready
            2. How to do it
          4. Creating a common role
            1. Getting ready
            2. How to do it…
            3. See also
          5. Creating a webserver using Ansible and Nginx
            1. Getting ready
            2. How to do it…
            3. See also
          6. Creating an application server role using Tomcat and Ansible
            1. Getting ready
            2. How to do it…
            3. See also
          7. Installing MySQL using Ansible
            1. Getting ready
            2. How to do it…
            3. See also
          8. Installing and managing HAProxy with Ansible
            1. Getting ready
            2. How to do it…
            3. See also
          9. Using ServerSpec to test your Playbook
            1. Getting ready
            2. How to do it…
            3. See also
        6. 6. Containerization with Docker
          1. Introduction
          2. Installing Docker
            1. Getting ready
            2. How to do it…
            3. See also
          3. Pulling an image from the public Docker registry
            1. Getting ready
            2. How to do it…
            3. See also
          4. Performing basic Docker operations
            1. Getting ready
            2. How to do it…
            3. See also
          5. Running a container interactively
            1. Getting ready
            2. How to do it…
            3. See also
          6. Creating a Dockerfile
            1. Getting ready
            2. How to do it…
            3. See also
          7. Running a container in detached mode
            1. Getting ready
            2. How to do it…
            3. See also
          8. Saving and restoring a container
            1. Getting ready
            2. How to do it…
            3. See also
          9. Using the host only network
            1. Getting ready
            2. How to do it…
            3. See also
          10. Running a private Docker registry
            1. Getting ready
            2. How to do it
            3. See also
          11. Managing images with a private registry
            1. Getting ready
            2. How to do it…
              1. Pushing images
              2. Pulling images
            3. See also
        7. 7. Using Jenkins for Continuous Deployment
          1. Introduction
          2. Installing Jenkins
            1. Getting ready
            2. How to do it…
            3. See also…
          3. Installing the Git plugin
            1. Getting ready
            2. How to do it…
            3. See also
          4. Installing a Jenkins slave
            1. Getting ready
            2. How to do it…
            3. See also
          5. Creating your first Jenkins job
            1. Getting ready
            2. How to do it…
            3. See also
          6. Building Docker containers using Jenkins
            1. Getting ready
            2. How to do it…
          7. Deploying a Java application to Tomcat with zero downtime using Ansible
            1. Getting ready
            2. How to do it…
            3. See also
        8. 8. Metric Collection with InfluxDB
          1. Introduction
          2. Installing InfluxDB
            1. Getting ready
            2. How to do it…
            3. See also
          3. Creating a new InfluxDB database
            1. Getting ready
            2. How to do it…
            3. See also
          4. Logging events with the InfluxDB REST API
            1. Getting ready
            2. How to do it…
            3. See also
          5. Gathering host statistics with Telegraf
            1. Getting ready
            2. How to do it…
            3. See also
          6. Exploring data with the InfluxDB data explorer
            1. Getting ready
            2. How to do it…
            3. See also
          7. Installing Grafana
            1. Getting ready…
            2. How to do it…
            3. See also
          8. Creating dashboards with Grafana
            1. Getting ready
            2. How to do it…
            3. See also
        9. 9. Log Management
          1. Introduction
          2. Centralizing logs with Syslog
            1. Getting ready
            2. How to do it…
            3. See also
          3. Using syslog templates
            1. Getting ready
            2. How to do it…
            3. See also
          4. Managing log rotation with the Logrotate utility
            1. Getting ready
            2. How to do it…
            3. See also
          5. Installing ElasticSearch, Logstash, and Kibana
            1. Getting ready
            2. How to do it…
            3. See also
          6. Importing logs into Elasticsearch with Logstash
            1. Getting ready
            2. How to do it…
            3. See also
          7. Using Kibana queries to explore data
            1. Getting ready
            2. How to do it…
            3. See also
          8. Using Kibana queries to examine data
            1. Getting ready
            2. How to do it…
            3. See also
        10. 10. Monitoring with Sensu
          1. Introduction
          2. Installing a Sensu server
            1. Getting ready
            2. How to do it…
            3. See also
          3. Installing a Sensu client
            1. Getting ready
            2. How to do it…
            3. See also
          4. Installing check prerequisites
            1. Getting ready
            2. How to do it…
          5. Finding community checks
            1. Getting ready
            2. How to do it…
            3. See also
          6. Adding a DNS check
            1. Getting ready
            2. How to do it…
            3. See also
          7. Adding a disk check
            1. Getting ready
            2. How to do it…
            3. See also
          8. Adding a RAM check
            1. Getting ready
            2. How to do it…
            3. See also
          9. Adding a process check
            1. Getting ready…
            2. How to do it…
            3. See also
          10. Adding a CPU check
            1. Getting ready
            2. How to do it…
            3. See also
          11. Creating e-mail alerts
            1. Getting ready
            2. How to do it…
            3. See also
          12. Creating SMS alerts
            1. Getting ready
            2. How to do it…
            3. See also
          13. Using Ansible to install Sensu
            1. Getting ready
            2. How to do it…
            3. See also
        11. 11. IAAS with Amazon AWS
          1. Introduction
          2. Signing up for AWS
            1. Getting ready
            2. How to do it…
            3. See also
          3. Setting up IAM
            1. Getting ready
            2. How to do it…
            3. See also
          4. Creating your first security group
            1. Getting ready
            2. How to do it…
            3. See also
          5. Creating your first EC2 host
            1. Getting ready
            2. How to do it…
            3. See also
          6. Using Elastic Load Balancers
            1. Getting ready
            2. How to do it…
            3. See also
          7. Managing DNS with route53
            1. Getting ready…
            2. How to do it…
            3. See also
          8. Using Ansible to create EC2 hosts
            1. Getting ready
            2. How to do it…
            3. See also
        12. 12. Application Performance Monitoring with New Relic
          1. Introduction
          2. Signing up for a New Relic account
            1. Getting ready
            2. How to do it…
            3. See also
          3. Installing the New Relic Java agent
            1. Getting ready
            2. How to do it…
            3. See also
          4. Using the performance overview
            1. Getting ready
            2. How to do it…
            3. See also
          5. Locating performance bottlenecks with Transaction Traces
            1. Getting ready
            2. How to do it…
            3. See also
          6. Observing database performance with New Relic
            1. Getting ready
            2. How to do it…
            3. See also
          7. Release performance monitoring with New Relic
            1. Getting ready
            2. How to do it…
            3. See also
          8. Server Monitoring with New Relic
            1. Getting ready
            2. How to do it…
            3. See also
      8. III. Module 3
        1. 1. Evolution of a Software House
          1. A brief history of ACME systems
          2. ACME systems version 1.0
            1. Software delivery process flow version 1.0
          3. ACME systems version 2.0
            1. Software delivery process flow version 2.0
            2. A few brave men and women
          4. ACME systems version 3.0
            1. Software delivery process flow version 3.0
          5. ACME systems version 4.0
          6. The evolution in a nutshell
          7. Summary
        2. 2. No Pain, No Gain
          1. Elephant in the room
          2. Defining the rules
          3. Including (almost) everyone
            1. Identifying the key people
            2. Too many cooks
          4. Openness, transparency, and honesty
            1. Location, location, location
            2. It's all happy-clappy management waffle – isn't it?
          5. The great elephant disclosure
            1. Value stream mapping
          6. Summary
        3. 3. Plan of Attack
          1. Setting and communicating the goal and vision
          2. Standardizing vocabulary and language
          3. A business change project in its own right
          4. The merits of a dedicated team
            1. Who to include
          5. The importance of evangelism
          6. Courage and determination
          7. Understanding the cost
          8. Seeking advice from others
          9. Summary
        4. 4. Culture and Behaviors
          1. All roads lead to culture
          2. An open, honest, and safe environment
            1. Openness and honesty
            2. Courageous dialogue
            3. The physical environment
          3. Encouraging and embracing collaboration
          4. Fostering innovation and accountability at grass roots
          5. The blame culture
            1. Blame slow, learn quickly
          6. Building trust-based relationships across organizational boundaries
          7. Rewarding good behaviors and success
            1. The odd few
          8. Recognizing dev and ops teams are incentivized can have an impact
          9. Embracing change and reducing risk
            1. Changing people's perceptions with pudding
          10. Being transparent
          11. Summary
        5. 5. Approaches, Tools, and Techniques
          1. Engineering best practice
            1. Source control
            2. Small, frequent, and simple changes
            3. Never break your consumer
            4. Open and honest peer-working practices
            5. Fail fast and often
            6. Automated builds and tests
            7. Continuous Integration
            8. Using the same binary across all environments
          2. How many environments are enough?
          3. Developing against a production-like environment
          4. CD tooling
            1. Automated provisioning
            2. No-downtime deployments
          5. The cloud
          6. Monitoring
          7. When a simple manual process is also an effective tool
          8. Summary
        6. 6. Hurdles Along the Way
          1. What are the potential issues you need to look out for?
            1. Dissenters in the ranks
              1. No news is no news
              2. The anti-agile brigade
            2. The transition curve
            3. The outsiders
            4. Corporate guidelines, red tape, and standards
            5. Geographically diverse teams
            6. Failure during evolution
            7. Processes that are not repeatable
            8. Recruitment
          2. Summary
        7. 7. Vital Measurements
          1. Measuring effective engineering best practice
            1. Simple quality metrics
            2. Code complexity
            3. Unit test coverage
            4. Commit rates
            5. Adherence to coding rules and standards
            6. Where to start and why bother?
          2. Measuring the real world
            1. Measuring the stability of the environments
              1. Incorporating automated tests
              2. Combining automated tests and system monitoring
              3. Real-time monitoring of the software itself
              4. Monitoring utopia
            2. Effectiveness of CD and DevOps
            3. Impact of CD and DevOps
            4. Measuring your culture
          3. Summary
        8. 8. Are We There Yet?
          1. Reflect on where you are now
          2. Streaming
          3. A victim of your own success
          4. [P]lan, [D]o, [C]heck, [A]djust
          5. Exit stage left
          6. Rest on your laurels (not)
          7. Summary
        9. 9. The Future is Bright
          1. Expanding your horizon
            1. Reactive performance and load testing
            2. Reducing feature flag complexity
            3. Easing A/B testing
            4. Security patching and saving your bacon
            5. Order out of chaos monkey
            6. End user self-service
            7. CD and DevOps and the mobile world
          2. Expanding beyond software delivery
          3. What about me?
          4. What have you learned?
          5. Summary
        10. A. Some Useful Information
          1. Tools
          2. People
          3. Recommended reading
        11. B. Where Am I on the Evolutionary Scale?
        12. C. Retrospective Games
          1. The timeline game
          2. StoStaKee
        13. D. Vital Measurements Expanded
          1. Code complexity – some science
          2. Code versus comments
          3. Embedding monitoring into your software
      9. A. Bibliography
      10. Index