You are previewing Mastering Proxmox - Second Edition.
O'Reilly logo
Mastering Proxmox - Second Edition

Book Description

Master the skills you need to build a rock-solid virtualization environment with the all new Proxmox 4

About This Book

  • Formulate Proxmox-based solutions and set up virtual machines of any size while gaining expertise even on the most complex multi-cluster setups

  • Master the skills needed to analyze, monitor, and troubleshoot real-world virtual environments

  • This is the most up-to-date title on mastering Proxmox, with examples based on the new Linux Kernel 4.2 and Debian Jessie (8.x)

  • Who This Book Is For

    The book targets Linux and system administrators and professionals working in IT teams who would like to design and implement an enterprise-quality virtualized environment using Proxmox.Knowledge of networking and virtualization concepts is assumed.

    What You Will Learn

  • Install the basic Proxmox VE nodes

  • Get to know the Proxmox Graphical User Interface (GUI)

  • See and understand Proxmox’s internal structure and mechanics

  • Create and manage KVM- or LXC-based virtual machines

  • Understand advanced virtual networks

  • Leverage the power of the built-in Proxmox firewall to provide protection at any level

  • Configure high availability Proxmox nodes

  • Learn different backup strategies and to restore a VM in the event of a disaster to ensure data safety

  • Monitor a Proxmox cluster and all of its components using Zabbix

  • Plan large virtual environments for mission critical virtual environments

  • Work through real-world scenarios to troubleshoot Proxmox

  • In Detail

    Proxmox is an open source server virtualization solution that has enterprise-class features to manage virtual machines, to be used for storage, and to virtualize both Linux and Windows application workloads.

    You begin with refresher on the advanced installation features and the Proxmox GUI to familiarize yourself with the Proxmox VE hypervisor. You then move on to explore Proxmox under the hood, focusing on the storage systems used with Proxmox. Moving on, you will learn to manage KVM Virtual Machines and Linux Containers and see how networking is handled in Proxmox. You will then learn how to protect a cluster or a VM with a firewall and explore the new HA features introduced in Proxmox VE 4 along with the brand new HA simulator.

    Next, you will dive deeper into the backup/restore strategy followed by learning how to properly update and upgrade a Proxmox node. Later, you will learn how to monitor a Proxmox cluster and all of its components using Zabbix. By the end of the book, you will become an expert at making Proxmox environments work in production environments with minimum downtime.

    Style and approach

    This book walks you through every aspect of virtualization using Proxmox using a practical, scenario-based approach that features best practices and all the weaponry you need to succeed when building virtual environments with Proxmox 4.

    Downloading the example code for this book. You can download the example code files for all Packt books you have purchased from your account at http://www.PacktPub.com. If you purchased this book elsewhere, you can visit http://www.PacktPub.com/support and register to have the code file.

    Table of Contents

    1. Mastering Proxmox - Second Edition
      1. Table of Contents
      2. Mastering Proxmox - Second Edition
      3. Credits
      4. About the Author
      5. About the Reviewer
      6. www.PacktPub.com
        1. eBooks, discount offers, and more
          1. Why subscribe?
      7. Preface
        1. What this book covers
        2. What you need for this book
        3. Who this book is for
        4. Conventions
        5. Reader feedback
        6. Customer support
          1. Downloading the color images of this book
          2. Errata
          3. Piracy
          4. Questions
      8. 1. Understanding Proxmox VE and Advanced Installation
        1. Understanding Proxmox features
          1. It is free!
          2. Built-in firewall
          3. Open vSwitch
          4. The graphical user interface
          5. KVM
          6. Linux containers or LXC
          7. Storage plugins
          8. Vibrant culture
        2. The basic installation of Proxmox
        3. The advanced installation option
        4. Debugging the Proxmox installation
        5. Customizing the Proxmox splash screen
        6. Summary
      9. 2. Exploring the Proxmox GUI
        1. The GUI menu system
        2. The Datacenter menu
          1. Datacenter | Search
          2. Datacenter | Summary
          3. Datacenter | Options
          4. Datacenter | Storage
          5. Datacenter | Backup
          6. Datacenter | Users
          7. Datacenter | Groups
          8. Datacenter | Pools
          9. Datacenter | Permissions
          10. Datacenter | Roles
          11. Datacenter | Authentication
          12. Datacenter | HA
          13. Datacenter | Firewall
          14. Datacenter | Support
        3. Node menu
          1. Node | Search
          2. Node | Summary
          3. Node | Services
          4. Node | Network
          5. Node | DNS
          6. Node | Time
          7. Node | Syslog
          8. Node | Task History
          9. Node | Firewall
          10. Node | Updates
          11. Node | Console
          12. Node | Ceph
          13. Node | Subscription
        4. The KVM VM menu
          1. KVM VM | Summary
          2. KVM VM | Hardware
          3. KVM VM | Options
          4. KVM VM | Task History
          5. KVM VM | Monitor
          6. KVM VM | Backup
          7. KVM VM | Snapshot
          8. KVM VM | Console
          9. KVM VM | Firewall
          10. KVM VM | Permissions
        5. The LXC Container menu
          1. The LXC Container | Summary
          2. LXC Container | Resources
          3. LXC Container | Network
          4. LXC Container | DNS
          5. LXC Container | Options
          6. LXC Container | Task History
          7. LXC Container | Backup
          8. LXC Container | Console
          9. LXC Container | Snapshots
          10. LXC Container | Firewall
          11. LXC Container | Permissions
        6. Pool menu
          1. Pool | Summary
          2. Pool | Members
          3. Pool | Permissions
        7. Cluster view mode
          1. The Server View
          2. The Folder View
          3. Storage view
          4. The Pool View
        8. Summary
      10. 3. Proxmox under the Hood
        1. The Proxmox cluster filesystem
        2. Proxmox directory structure
        3. Dissecting the configuration files
          1. The cluster configuration file
            1. logging { }
            2. nodelist { }
            3. nodeid
            4. quorum_votes
            5. ring0_addr
            6. quorum { }
            7. totem { }
            8. interface { }
          2. Storage configuration file
          3. User configuration files
          4. The password configuration file
          5. KVM virtual machine configuration file
          6. Arguments in the KVM configuration file
          7. LXC container configuration file
          8. Version configuration file
          9. Member nodes
          10. Virtual machine list file
          11. The cluster log file
          12. Ceph configuration files
          13. Firewall configuration file
        4. Summary
      11. 4. Storage Systems
        1. Local storage versus shared storage
          1. Live migration of a virtual machine
          2. Seamless expansion of multinode storage space
          3. Centralized backup

          4. Multilevel data tiering
          5. Central storage management
        2. Local and shared storage comparison
        3. A virtual disk image
          1. Supported image formats
            1. The .qcow2 image type
            2. The .raw image type
            3. The .vmdk image type
          2. Managing disk images
            1. Resizing a virtual disk image
            2. Moving a virtual disk image
            3. Throttling a virtual disk image
            4. Caching a virtual disk image
        4. Storage types in Proxmox
          1. Directory
          2. iSCSI
          3. LVM
          4. NFS
          5. ZFS
          6. Ceph RBD
            1. The Ceph components
              1. A physical node
              2. Maps
              3. A cluster map
              4. A CRUSH Map
              5. Monitor
              6. OSD
              7. OSD Journal
              8. MDS
              9. Placement Group
              10. Pools
              11. The Ceph components summary
            2. Virtual Ceph for training
            3. Ceph cluster
            4. Ceph on Proxmox
            5. Preparing the Proxmox node for Ceph
            6. Installing Ceph
            7. Creating MONs from Proxmox GUI
            8. Creating OSD from Proxmox GUI
            9. Creating a new Ceph Pool using Proxmox GUI
            10. Connecting RBD to Proxmox
            11. Ceph command list
          7. GlusterFS
        5. Noncommercial and commercial storage options
        6. Summary
      12. 5. KVM Virtual Machines
        1. Exploring a KVM
        2. Creating a KVM
          1. Creating a VM from scratch
            1. General tab
            2. OS tab
            3. CD/DVD tab
            4. Hard Disk tab
              1. Bus/Device
              2. Storage
              3. Disk size
              4. Format
              5. Cache
              6. No backup
              7. Discard
              8. Iothread
            5. CPU tab
              1. Sockets
              2. Cores
              3. Enabling NUMA
              4. Type
            6. Memory tab
            7. Network
              1. Bridged mode
              2. Firewall
              3. NAT mode
              4. No network device
              5. Model
              6. MAC address
              7. Rate limit
              8. Multiqueues
              9. Disconnect
          2. Creating a VM by cloning
            1. Mode
          3. Creating VMs from templates
        3. Advanced configuration options for VM
          1. Configuring a sound device
          2. Configuring PCI passthrough
          3. Configuring GPU Passthrough
          4. Configuring Hotplug
        4. Migrating a KVM
        5. Nested virtual cluster
        6. Summary
      13. 6. LXC Virtual Machines
        1. Exploring an LXC virtual machine
        2. Understanding container templates
        3. Creating an LXC container
          1. General tab
            1. Node
            2. VM ID
            3. Hostname
            4. Resource Pool
          2. The Template tab
          3. The Root Disk tab
          4. The CPU tab
            1. CPU limits
            2. CPU units
          5. The Memory tab
          6. The Network tab
            1. ID
            2. Name
            3. MAC address
            4. Bridge
            5. The VLAN Tag
            6. Firewall
            7. IPv4/IPv6
          7. The DNS tab
          8. The Confirm tab
        4. Managing an LXC container
          1. Adjusting resources using the GUI
          2. Adjusting resources using the CLI
          3. Adjusting resources using direct modification
        5. Migrating an LXC container
        6. Accessing an LXC container
        7. Unprivileged versus privileged containers
          1. Creating an unprivileged container as root
          2. Checking container processes
        8. Converting OpenVZ to LXC
        9. Summary
      14. 7. Network of Virtual Networks
        1. Exploring a virtual network
        2. A physical network versus a virtual network
          1. A physical network
          2. A virtual network
        3. Networking components in Proxmox
          1. Virtual Network Interface Card (vNIC)
            1. Adding vNIC
          2. A virtual bridge
            1. Adding a virtual bridge
          3. Extra bridge options
            1. bridge_stp
            2. bridge_fd
          4. Virtual LAN
            1. Adding a VLAN
          5. Network Address Translation/Translator
            1. Adding NAT/masquerading
          6. Network bonding
            1. Adding a bonding interface
              1. The layer 2 hash policy
              2. The layer 2+3 hash policy
              3. The layer 3+4 hash policy
          7. Multicast
            1. Configuring multicast on Netgear
        4. Open vSwitch
          1. Features of Open vSwitch
          2. Adding the Open vSwitch bridge
          3. Adding the Open vSwitch bond
          4. Adding Open vSwitch IntPort
          5. CLI for Open vSwitch
          6. Practicing Open vSwitch
            1. Configuration requirements
            2. Solutions
        5. Sample virtual networks
          1. Network #1 – Proxmox in its simplest form
          2. Network #2 – the multitenant environment
          3. Network #3 – academic institution
        6. A multitenant virtual environment
          1. A multitenant network diagram
        7. Summary
      15. 8. The Proxmox Firewall
        1. Exploring the Proxmox VE firewall
          1. Components of the Proxmox firewall
            1. Zones
            2. Security Groups
            3. IPSet
            4. Rules
            5. Protocols
            6. Macros
            7. The pve-firewall and pvefw-logger services
          2. Configuration files of a firewall
        2. Configuring the Datacenter-specific firewall
          1. Configuring the Datacenter firewall through the GUI
            1. Creating the Datacenter firewall rules
            2. Creating the Datacenter IPSet
            3. Creating aliases
          2. Configuring the Datacenter firewall through the CLI
            1. [OPTIONS]
            2. [ALIASES]
            3. [IPSET <name>]
            4. [RULES]
            5. [group <name>]
        3. Configuring a host-specific firewall
          1. Creating host firewall rules
            1. Options for the host zone firewall
              1. Enable a firewall
              2. The SMURFS filter
              3. The TCP flags filter
              4. nf_conntrack_max
              5. nf_conntrack_tcp_timeout_established
              6. log_level_in/out
              7. tcp_flags_log_level
              8. smurf_log_level
          2. Configuring the host firewall through the CLI
        4. Configuring a VM-specific firewall
          1. Creating VM firewall rules
          2. Creating Aliases
          3. Creating IPSet
          4. Options for a VM zone firewall
            1. Enable DHCP
            2. The MAC filter
          5. Configuring a VM-specific firewall through the CLI
        5. Integrating a Suricata IDS/IPS
          1. Installing/configuring Suricata
          2. Limitation of Suricata in Proxmox
        6. Summary
      16. 9. Proxmox High Availability
        1. Understanding High Availability
          1. High Availability in Proxmox
          2. How Proxmox HA works
        2. Requirements for HA setup
          1. Minimum three nodes
          2. Shared storage
          3. Fencing
          4. BIOS power on feature
        3. Configuring Proxmox HA
          1. Status menu
          2. Groups menu
            1. ID
            2. Node
            3. Restricted
            4. Nofailback
          3. Resources menu
          4. Fencing menu
        4. Testing Proxmox HA configuration
        5. The Proxmox HA simulator
          1. Configuring the Proxmox HA simulator
        6. Summary
      17. 10. Backup/Restore VMs
        1. Proxmox backup options
          1. A full backup
            1. Full backup modes
              1. Snapshot
              2. Suspend
              3. Stop
            2. Backup compression
              1. None
              2. LZO
              3. GZIP
          2. Snapshots
        2. Configuring backup storage
        3. Configuring full backup
          1. Creating a schedule for Backup
            1. Node
            2. Storage
            3. Day of week
            4. Start Time
            5. Selection mode
            6. Send e-mail to
            7. E-mail notification
            8. Compression
            9. Mode
            10. Enable
        4. Creating a manual backup
        5. Creating snapshots
        6. Restoring a VM
        7. Backup/restore through the CLI
          1. Backup using the CLI
          2. Restore using the CLI
          3. Unlocking a VM after a backup error
        8. Back up a configuration file
          1. #bwlimit
          2. #lockwait
          3. #stopwait
          4. #script
          5. #exclude-path
          6. #pigz
        9. Summary
      18. 11. Updating and Upgrading Proxmox
        1. Introducing Proxmox update
        2. Updating Proxmox through the GUI
        3. Updating Proxmox through the CLI
          1. Difference between upgrade and dist-upgrade
        4. Recovering from the grub2 update issue
        5. Updating after a subscription change
        6. Rebooting dilemma after Proxmox updates
        7. Summary
      19. 12. Monitoring a Proxmox Cluster
        1. An introduction to monitoring
        2. Proxmox built-in monitoring
        3. Zabbix as a monitoring solution
        4. Installing Zabbix
        5. Configuring Zabbix
          1. Configuring a host to monitor
          2. Displaying data using a graph
          3. Configuring the Disk Health notification
            1. Installing smart monitor tools
            2. Configuring the Zabbix agent
            3. Creating a Zabbix item in the GUI
            4. Creating a trigger in the GUI
            5. Creating graphs in the GUI
        6. Configuring SNMP in Proxmox
          1. OIDs
          2. MIB
        7. Adding an SNMP device in Zabbix
        8. Monitoring the Ceph cluster with the Ceph dashboard
        9. Summary
      20. 13. Proxmox Production-Level Setup
        1. Defining production level
          1. Key components
            1. Stable and scalable hardware
            2. Redundancy
              1. Node level
              2. Utility level
              3. Network level
              4. HVAC level
              5. Storage level
            3. Current load versus future growth
            4. Budget
            5. Simplicity
            6. Tracking hardware inventory
            7. Hardware selection
        2. Sizing CPU and memory
          1. Single socket versus multi-socket
          2. Hyper-threading – enable versus disable
          3. Start small with VM resources
          4. Balancing node resources
        3. Production Ceph cluster
          1. Forget about hardware RAID
          2. SSD for Ceph Journal
          3. Network bandwidth
        4. Liquid cooling
          1. Total immersion in oil
          2. Total immersion in 3M Novec
          3. Direct contact liquid cooling
        5. Real-world Proxmox scenarios
          1. Scenario 1 – an academic institution
          2. Scenario 2 – multi-tier storage cluster with a Proxmox cluster
          3. Scenario 3 – virtual infrastructure for a multi-tenant cloud service provider
          4. Scenario 4 – nested virtual environment for a software development company
          5. Scenario 5 – virtual infrastructure for a public library
          6. Scenario 6 – multi-floor office virtual infrastructure with virtual desktops
          7. Scenario 7 – virtual infrastructure for the hotel industry
          8. Scenario 8 – virtual infrastructure for a geological survey organization
        6. Summary
      21. 14. Proxmox Troubleshooting
        1. Proxmox node issues
          1. Issue – Fresh Proxmox install stuck with /dev to be fully populated error during node reboot
          2. Issue – Rejoining a node to a Proxmox node with the same old IP address
          3. Issue – Booting with a USB stick fails in Proxmox
          4. Issue – Proxmox installation completed but GRUB is in an endless loop after reboot
          5. Issue – LSI MegaRAID 9240-8i/9240-4i causes an error during booting on the Proxmox node
            1. Downloading and updating the LSI driver
            2. Updating the Supermicro BIOS
        2. Main cluster issues
          1. Issue – Proxmox virtual machines are running, but the Proxmox GUI shows that everything is offline
          2. Issue – Kernel panic when disconnecting USB devices, such as a keyboard, mouse, or UPS
          3. Issue – Virtual machines on Proxmox will not shut down if shutdown is initiated from the Proxmox GUI
          4. Issue – Kernel panic with HP NC360T (Intel 82571EB Chipset) only in Proxmox VE 3.2
          5. Issue – The Proxmox cluster is out of Quorum and cluster filesystem is in read-only mode
          6. Issue – Proxmox cannot start due to the getpwnam error
          7. Issue – Cannot log in to the GUI as root after reinstalling Proxmox on the same node
          8. Issue – The Upgrade button is disabled on the Proxmox GUI, which prevents the node upgrade
          9. Issue – VM will not respond to shutdown, restart
          10. Issue – The Proxmox GUI not showing RRD graphs
        3. Storage issues
          1. Issue – Deleting a damaged LVM from Proxmox with the error read failed from 0 to 4096
          2. Issue – Proxmox cannot mount NFS Share due to the timing out error
          3. Issue: How to delete leftover NFS shares in Proxmox or what to do when the NFS stale file handles error occurs?
          4. Issue – Proxmox issues—mode session exit code 21 errors while trying to access the iSCSI target
          5. Issue – Cannot read an iSCSI target even after it has been deleted from Proxmox storage
          6. Issue – A Ceph node is removed from the Proxmox cluster, but OSDs still show up in PVE
          7. Issue – The 'No Such Block Device' error during creation of an OSD through the Proxmox GUI
          8. Issue – The fstrim command does not trim unused blocks for the Ceph storage
          9. Issue – The RBD Couldn't Connect To Cluster (500) error when connecting Ceph with Proxmox
          10. Issue – Changing the storage type from ide to virtio after the VM has been set up and the OS has been installed
          11. Issue – The pveceph configuration not initialized (500) error when you click on the Ceph tab in the Proxmox GUI
          12. Issue – The CephFS storage disappears after a Proxmox node reboots
          13. Issue – VM Cloning does not parse in the Ceph storage
        4. Network connectivity issues
          1. Issue – No connectivity on Realtek RTL8111/8411 Rev. 06 network interfaces
          2. Issue – Network performance is slower with the e1000 virtual network interface
          3. Issue – Patch port for Openvswitch in Proxmox not working
          4. Issue – Trying to add a node to a newly created Proxmox cluster when nodes do not form quorum
          5. Issue – Implemented IPv6 but firewall rules do not get applied
        5. KVM virtual machine issues
          1. Issue – Windows 7 VM does not reboot, instead it shuts down requiring manual boot from Proxmox
          2. Issue – The qemu-img command does not convert the .vmdk image files created with the .ova template in Proxmox VE 4.1
          3. Issue – Online migration of a virtual machine fails with a 'Failed to sync data' error
          4. Issue – Adjusting RAM through the GUI and rebooting the VM does not change allocated memory
          5. Issue – No audio in Windows KVM
          6. Issue – The virtio virtual disk is not available during the Windows Server installation
        6. LXC container issues
          1. Issue – A Proxmox node hangs when trying to stop or restart an LXC container
          2. Issue – The noVNC console only shows a cursor for LXC containers
        7. Backup/restore issues
          1. Issue – A Proxmox VM is locked after backup crashes unexpectedly
          2. Issue – How can Proxmox backup only the primary OS virtual disk instead of all the virtual disks for a VM?
          3. Issue – Backup of virtual machines stops prematurely with an Operation Not Permitted error
          4. Issue – A backup task takes a very long time to complete, or it crashes when multiple nodes are backing up to the same backup storage
          5. Issue – Backup of virtual machines aborts a backup task prematurely
          6. Issue – Backup storage has a lot of .dat files and .tmp folders using the storage space
        8. VNC/SPICE console issues
          1. Issue – The mouse pointer is not shared with SPICE-VIEWER on Windows 8 VM
          2. Issue – The SPICE console is unstable or nonfunctioning after updating to Proxmox VE 4.1
          3. Issue – Remote Viewer is unable to connect to a SPICE-enabled virtual machine on the Windows OS
        9. Firewall issues
          1. Issue – Rules are created and a firewall is enabled for vNIC, but rules do not get applied
          2. Issue – A firewall is enabled for a VM and the necessary rules are created, but nothing is being filtered for that VM. All other VM firewall rules in the same node work properly
        10. Summary
      22. Index