IBM Spectrum Scale (formerly GPFS)

IBM Spectrum Scale (formerly GPFS)

Released May 2015
Publisher(s): IBM Redbooks
ISBN: 9780738440736

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

This IBM® Redbooks® publication updates and complements the previous publication: Implementing the IBM General Parallel File System in a Cross Platform Environment, SG24-7844, with additional updates since the previous publication version was released with IBM General Parallel File System (GPFS™). Since then, two releases have been made available up to the latest version of IBM Spectrum™ Scale 4.1. Topics such as what is new in Spectrum Scale, Spectrum Scale licensing updates (Express/Standard/Advanced), Spectrum Scale infrastructure support/updates, storage support (IBM and OEM), operating system and platform support, Spectrum Scale global sharing - Active File Management (AFM), and considerations for the integration of Spectrum Scale in IBM Tivoli® Storage Manager (Spectrum Protect) backup solutions are discussed in this new IBM Redbooks publication.

This publication provides additional topics such as planning, usability, best practices, monitoring, problem determination, and so on. The main concept for this publication is to bring you up to date with the latest features and capabilities of IBM Spectrum Scale as the solution has become a key component of the reference architecture for clouds, analytics, mobile, social media, and much more.

This publication targets technical professionals (consultants, technical support staff, IT Architects, and IT Specialists) responsible for delivering cost effective cloud services and big data solutions on IBM Power Systems™ helping to uncover insights among clients' data so they can take actions to optimize business results, product development, and scientific discoveries.

Table of contents

  1. Front cover
  2. Notices
    1. Trademarks
  3. IBM Redbooks promotions
  4. Preface
    1. Authors
    2. Now you can become a published author, too!
    3. Comments welcome
    4. Stay connected to IBM Redbooks
  5. Chapter 1. Introduction
    1. 1.1 Overview
    2. 1.2 IBM Spectrum Scale major components and terminology
      1. 1.2.1 IBM Spectrum Scale cluster topologies
    3. 1.3 IBM Spectrum Scale new features and enhancements
      1. 1.3.1 Enhanced security
      2. 1.3.2 Performance improvements
      3. 1.3.3 New usability features
      4. 1.3.4 Reliability, availability, and serviceability
    4. 1.4 IBM Spectrum Scale competitive strategy
    5. 1.5 IBM Spectrum Scale licensing
    6. 1.6 IBM Spectrum Scale on cross-platform environments
      1. 1.6.1 IBM Spectrum Scale for AIX
      2. 1.6.2 IBM Spectrum Scale for Windows operating systems
      3. 1.6.3 IBM Spectrum Scale for Linux
      4. 1.6.4 IBM Spectrum Scale for Linux on System z
      5. 1.6.5 IBM Spectrum Scale Server solution
    7. 1.7 IBM Spectrum Scale and virtualization
      1. 1.7.1 Virtualization on IBM Power Systems
      2. 1.7.2 Virtualization on x86 Linux operating systems
      3. 1.7.3 Virtualization on Windows operating systems
    8. 1.8 Contact information
  6. Chapter 2. Infrastructure planning and considerations
    1. 2.1 IBM Spectrum Scale cluster topologies
      1. 2.1.1 Network-based Spectrum Scale client
      2. 2.1.2 Direct attached storage
      3. 2.1.3 Mixed NSD access: server and clients
      4. 2.1.4 IBM Spectrum Scale File Placement Optimizer
    2. 2.2 Network design
      1. 2.2.1 Ethernet adapters options
      2. 2.2.2 NIC teaming configurations
      3. 2.2.3 InfiniBand networks
      4. 2.2.4 RDMA over Converged Enhanced Ethernet
      5. 2.2.5 IBM data center networking products
    3. 2.3 Storage design
      1. 2.3.1 Host bus adapter
      2. 2.3.2 Multipath driver
      3. 2.3.3 Storage subsystem considerations
      4. 2.3.4 Tape library
    4. 2.4 IBM Spectrum Scale supported platforms
      1. 2.4.1 IBM Spectrum Scale on AIX
      2. 2.4.2 IBM Spectrum Scale for Linux (x86 and POWER)
      3. 2.4.3 IBM Spectrum Scale for Linux on System z
      4. 2.4.4 IBM Spectrum Scale on Windows operating systems
      5. 2.4.5 Summary of IBM Spectrum Scale support functions on operating systems
      6. 2.4.6 GPT partition table
      7. 2.4.7 SCSI-3 Persistent Reservation
    5. 2.5 Security considerations for IBM Spectrum Scale clusters
      1. 2.5.1 Remote shell with ssh
      2. 2.5.2 Remote cluster: Subnets, firewall rules, and TCP port numbers
      3. 2.5.3 SELinux configuration with IBM Spectrum Scale
    6. 2.6 IBM Spectrum Scale configuration planning
      1. 2.6.1 High availability
      2. 2.6.2 Network Shared Disk creation considerations
      3. 2.6.3 Planning the IBM Spectrum Scale file system
      4. 2.6.4 Planning for IBM Spectrum Scale FPO
  7. Chapter 3. Scenarios
    1. 3.1 IBM Spectrum Scale advantages over Network File System
    2. 3.2 IBM Spectrum Scale in active-passive and mutual takeover clusters
    3. 3.3 IBM Spectrum Scale in active-active clusters
    4. 3.4 Two-node Linux IBM Spectrum Scale cluster
      1. 3.4.1 Installing IBM Spectrum Scale
      2. 3.4.2 Configure auxiliary tools
      3. 3.4.3 Building the IBM Spectrum Scale portability layer on Linux nodes
      4. 3.4.4 Create IBM Spectrum Scale cluster
      5. 3.4.5 Create NSD disks
      6. 3.4.6 Starting the IBM Spectrum Scale cluster
      7. 3.4.7 Quorum configuration
      8. 3.4.8 Start IBM Spectrum Scale at boot
      9. 3.4.9 Create the IBM Spectrum Scale file system
    5. 3.5 Cluster NFS
      1. 3.5.1 NFS setup
      2. 3.5.2 Configuring cNFS
    6. 3.6 Windows operating system-only cluster
      1. 3.6.1 Configuring Windows operating system
      2. 3.6.2 Static IP address
      3. 3.6.3 Active Directory domain
      4. 3.6.4 UAC
      5. 3.6.5 Disable IPv6
      6. 3.6.6 Windows operating system firewall
      7. 3.6.7 IBM Spectrum Scale traces auxiliary tools
      8. 3.6.8 Installing Cygwin
      9. 3.6.9 Install IBM Spectrum Scale
      10. 3.6.10 Configure IBM Spectrum Scale embedded remote shell
      11. 3.6.11 Create IBM Spectrum Scale cluster
      12. 3.6.12 Creating NSD disks
      13. 3.6.13 Start the IBM Spectrum Scale cluster
      14. 3.6.14 Quorum configuration
      15. 3.6.15 Start IBM Spectrum Scale at boot
      16. 3.6.16 Create the IBM Spectrum Scale file system
      17. 3.6.17 Adding a Linux node to the Windows operating system cluster
      18. 3.6.18 Adding a Windows operating system node to the cluster
    7. 3.7 Oracle Real Application Cluster with IBM Spectrum Scale
    8. 3.8 IBM Spectrum Scale integration with IBM Spectrum Protect (formerly Tivoli Storage Manager)
    9. 3.9 Sample Spectrum Scale FPO configuration
    10. 3.10 Integrating with OpenStack Swift
  8. Chapter 4. Management and maintenance
    1. 4.1 IBM Spectrum Scale and GPFS migration and update
      1. 4.1.1 GPFS and Spectrum Scale migration considerations
      2. 4.1.2 Migrating to Spectrum Scale 4.1 from GPFS 3.5 (Rolling update)
      3. 4.1.3 Migrating to Spectrum Scale 4.1 from GPFS 3.4 or GPFS 3.3
      4. 4.1.4 Migrating to Spectrum Scale 4.1 from GPFS 3.2 or earlier releases of GPFS
      5. 4.1.5 Completing the GPFS or IBM Spectrum Scale migration
      6. 4.1.6 Applying corrective fixes to IBM Spectrum Scale
    2. 4.2 Managing IBM Spectrum Scale cluster
      1. 4.2.1 Managing cluster repository
      2. 4.2.2 Changing cluster manager nodes
      3. 4.2.3 Managing IBM Spectrum Scale nodes
    3. 4.3 Managing IBM Spectrum Scale file systems
      1. 4.3.1 Listing file systems
      2. 4.3.2 Mounting a file system
      3. 4.3.3 Unmounting a file system
      4. 4.3.4 Creating an IBM Spectrum Scale file system
      5. 4.3.5 Removing a file system
      6. 4.3.6 Repairing a file system
      7. 4.3.7 Reducing file system fragmentation
      8. 4.3.8 Listing file system attributes
      9. 4.3.9 Changing file system attributes
      10. 4.3.10 Optimizing extended attributes: The fastea option
    4. 4.4 Managing IBM Spectrum Scale disks
      1. 4.4.1 Displaying disks
      2. 4.4.2 Creating NSDs
      3. 4.4.3 Adding a disk to a file system
      4. 4.4.4 Deleting disks from a file system
      5. 4.4.5 Replacing disks in an IBM Spectrum Scale file system
    5. 4.5 Managing IBM Spectrum Scale data migration
      1. 4.5.1 Migrating data using NSDs in GPFS file systems
      2. 4.5.2 Migrating data using AFM-based NFS Migration
    6. 4.6 Managing the IBM Spectrum Scale network
    7. 4.7 Managing IBM Spectrum Scale remote cluster
      1. 4.7.1 Planning and preparation
      2. 4.7.2 Adding a remote cluster
      3. 4.7.3 Enabling GPFS replication
      4. 4.7.4 Exporting or importing a file system
    8. 4.8 Configuring IBM Spectrum Scale callback
    9. 4.9 Monitoring IBM Spectrum Scale with SNMPD protocol
    10. 4.10 SSH configuration
  9. Chapter 5. Information lifecycle management
    1. 5.1 Explaining the ILM concept
    2. 5.2 Fileset
    3. 5.3 Snapshot
      1. 5.3.1 Snapshot at the file system level
      2. 5.3.2 Snapshot at the fileset level
      3. 5.3.3 Snapshot at file level
    4. 5.4 Storage pools
      1. 5.4.1 Internal storage pools
      2. 5.4.2 External storage pools
    5. 5.5 Immutability and appendOnly attributes
    6. 5.6 Quotas
      1. 5.6.1 Enabling and editing quotas
      2. 5.6.2 Creating quota reports
    7. 5.7 Policies and rules
      1. 5.7.1 Policies
      2. 5.7.2 Rules
      3. 5.7.3 Example of polices with internal pools only
      4. 5.7.4 Example of policies with one external pool only
    8. 5.8 Access control list
      1. 5.8.1 Traditional Spectrum Scale ACL administration
      2. 5.8.2 NFSv4 ACL administration
    9. 5.9 The mmfind policy sample
    10. 5.10 Differences with IBM EasyTier
  10. Chapter 6. Active File Management
    1. 6.1 Active file management fundamentals
      1. 6.1.1 AFM mode
      2. 6.1.2 Minimum requirements and basic operating system tuning
      3. 6.1.3 Using NFS for data movement
    2. 6.2 AFM single-writer
      1. 6.2.1 Creating a single-writer Cache fileset
      2. 6.2.2 Working with AFM filesets (single-writer)
      3. 6.2.3 Running single-writer fileset in disconnected mode
      4. 6.2.4 Auto recovering from failed connections
      5. 6.2.5 Manual recovering from failed connections
      6. 6.2.6 Permanent loss of HOME for single-writer
      7. 6.2.7 Permanent loss of Cache (single-writer)
      8. 6.2.8 Summary for AFM with single-writer
    3. 6.3 AFM independent-writer
      1. 6.3.1 Steps to create AFM enabled fileset in independent-writer mode
      2. 6.3.2 Working with independent-writer filesets
      3. 6.3.3 Cache eviction in independent-writer mode
      4. 6.3.4 Running independent-writer fileset in disconnected mode
      5. 6.3.5 Manual recovery from failed connections
      6. 6.3.6 Permanent loss of HOME
      7. 6.3.7 Temporary loss of Cache
      8. 6.3.8 Using AFM independent-writer for DR scenarios
      9. 6.3.9 Summary
    4. 6.4 Using AFM to migrate the content of an existing file system
    5. 6.5 Customizing and tuning AFM filesets
    6. 6.6 Building a global name space
    7. 6.7 Enhancements in IBM Spectrum Scale 4.1 TL 1
      1. 6.7.1 Gateway node changes
      2. 6.7.2 Native GPFS protocol
      3. 6.7.3 Performing parallel data transfers
  11. Chapter 7. Backup and disaster recovery using IBM Spectrum Scale
    1. 7.1 Disaster recovery solution using IBM Spectrum Scale replication
      1. 7.1.1 Configuration
      2. 7.1.2 Characteristics of this DR configuration
      3. 7.1.3 The IBM Spectrum Scale mmfsctl command
    2. 7.2 Implementing a scenario with IBM Spectrum Scale replication
      1. 7.2.1 Environment: Hardware, software, network, storage
      2. 7.2.2 IBM Spectrum Scale configuration
      3. 7.2.3 IBM Spectrum Scale configuration diagram
      4. 7.2.4 Set up and configure the IBM Spectrum Scale DR cluster
    3. 7.3 Backup and restore for IBM Spectrum Scale
      1. 7.3.1 mmbackup utility
      2. 7.3.2 Spectrum Scale advanced backup tools
      3. 7.3.3 Scale Out Backup and Restore
  12. Chapter 8. Problem determination
    1. 8.1 Problem determination process
      1. 8.1.1 Understanding the problem
      2. 8.1.2 Gathering information from the user
    2. 8.2 Gathering system information
      1. 8.2.1 Gathering operating system and IBM Spectrum Scale information
      2. 8.2.2 Testing connectivity and remote access
    3. 8.3 Operating system logs and IBM Spectrum Scale messages
      1. 8.3.1 Operating system logs
      2. 8.3.2 IBM Spectrum Scale messages
    4. 8.4 Verifying IBM Spectrum Scale cluster status
      1. 8.4.1 The mmgetstate command
      2. 8.4.2 The mmlsconfig command
      3. 8.4.3 The mmlscluster command
    5. 8.5 Collecting IBM Spectrum Scale file system and disk information
      1. 8.5.1 The mmlsfs command
      2. 8.5.2 The mmlsmount command
      3. 8.5.3 The mmlsnsd command
      4. 8.5.4 The mmwindisk command
      5. 8.5.5 The mmpmon command
    6. 8.6 Collecting IBM Spectrum Scale general information and logs
      1. 8.6.1 Considerations when gathering Spectrum Scale information
      2. 8.6.2 IBM Spectrum Scale severity tags
      3. 8.6.3 Spectrum Scale logs
      4. 8.6.4 The gpfs.snap command
      5. 8.6.5 Using the gpfs.snap command
    7. 8.7 Collecting IBM Spectrum Scale debug information
      1. 8.7.1 Collecting specific information
    8. 8.8 Working with IBM Spectrum Scale trace facility
      1. 8.8.1 Generating Spectrum Scale tracing information
      2. 8.8.2 Trace data analysis commands and scripts
    9. 8.9 Deadlock detection features
      1. 8.9.1 Automated deadlock detection
      2. 8.9.2 Automated deadlock data collection
      3. 8.9.3 Automated deadlock breakup
    10. 8.10 Additional information
    11. 8.11 IBM Spectrum Scale problem determination scenarios
      1. 8.11.1 Scenario 1: Spectrum Scale daemon not running
      2. 8.11.2 Scenario 2: Analyzing waiters
      3. 8.11.3 Scenario 3: Spectrum Scale file system hang
      4. 8.11.4 Scenario 4: Spectrum Scale file system unmounting
      5. 8.11.5 Scenario 5: Spectrum Scale file system not mounting
      6. 8.11.6 Scenario 6: Upgrading GPFS or Spectrum Scale
      7. 8.11.7 Scenario 7: NSD failures
  13. Chapter 9. Encryption
    1. 9.1 Introduction to encryption with IBM Spectrum Scale
    2. 9.2 How encryption is implemented in IBM Spectrum Scale
      1. 9.2.1 Support statement for encryption with other IBM Spectrum Scale features
    3. 9.3 Step-by-step setup procedure
      1. 9.3.1 Installing and setting up the IBM Security Key Lifecycle Manager
      2. 9.3.2 Installing IBM Security Lifecycle Manager
      3. 9.3.3 Accessing KLM
      4. 9.3.4 Updating the server
      5. 9.3.5 Creating certificates and keys for use with IBM Spectrum Scale and IBM Security Lifecycle Manager
      6. 9.3.6 Prepare IBM Security Lifecycle Manager for use with IBM Spectrum Scale
      7. 9.3.7 Configure IBM Spectrum Scale for encryption
    4. 9.4 Working with encryption
      1. 9.4.1 Verify encryption
      2. 9.4.2 Back up your keys
      3. 9.4.3 Remote key manager and RKM.conf
      4. 9.4.4 Secure deletion file data
      5. 9.4.5 Understanding read access to an encrypted file
    5. 9.5 Implementing access control on a node base
      1. 9.5.1 Changes to RKM
      2. 9.5.2 Changes to RKM.conf
      3. 9.5.3 Changes to Spectrum Scale policy
      4. 9.5.4 Activating access control
      5. 9.5.5 Rewrap FEKs to achieve node isolation from data
      6. 9.5.6 Securing your environment
      7. 9.5.7 Summary
  14. Appendix A. Recovery of the IBM Spectrum Scale file system configuration
    1. Back up the file system configuration
    2. Generating the scripts for creating the original file system configuration
    3. Restoring the file system configuration
  15. Appendix B. How to obtain an IBM Spectrum Scale trial version
  16. Related publications
    1. IBM Redbooks
    2. Other publications
    3. Online resources
    4. Help from IBM
  17. Back cover

Product information

  • Title: IBM Spectrum Scale (formerly GPFS)
  • Author(s):
  • Release date: May 2015
  • Publisher(s): IBM Redbooks
  • ISBN: 9780738440736