You are previewing Understanding and Using Q Replication for High Availability Solutions on the IBM z/OS Platform.
O'Reilly logo
Understanding and Using Q Replication for High Availability Solutions on the IBM z/OS Platform

Book Description

With ever-increasing workloads on production systems from transaction, batch, online query and reporting applications, the challenges of high availability and workload balancing are more important than ever.

This IBM® Redbooks® publication provides descriptions and scenarios for high availability solutions using the Q Replication technology of the IBM InfoSphere® Data Replication product on the IBM z/OS® platform. Also included are key considerations for designing, implementing, and managing solutions for the typical business scenarios that rely on Q Replication for their high availability solution.

This publication also includes sections on latency analysis, managing Q Replication in the IBM DB2® for z/OS environment, and recovery procedures. These are topics of particular interest to clients who implement the Q Replication solution on the z/OS platform.

Q Replication is a high-volume, low-latency replication solution that uses IBM WebSphere® MQ message queues to replicate transactions between source and target databases or subsystems. A major business benefit of the low latency and high throughput solution is timely availability of the data where the data is needed.

High availability solutions are implemented to minimize the impact of planned and unplanned disruptions of service to the applications. Disruption of service can be caused by software maintenance and upgrades or by software and hardware outages. As applications' high availability requirements evolve towards continuous availability, that is availability of the data 24 hours a day and 7 days a week, so does the Q Replication solution, to meet these challenges.

If you are interested in the Q Replication solution and how it can be used to implement some of the high availability requirements of your business scenarios, this book is for you.

Table of Contents

  1. Front cover
  2. Notices
    1. Trademarks
  3. Preface
    1. Authors
    2. Now you can become a published author, too!
    3. Comments welcome
    4. Stay connected to IBM Redbooks
  4. Chapter 1. High Availability scenarios
    1. 1.1 Introduction
      1. 1.1.1 How the book is organized
    2. 1.2 Terminology
      1. 1.2.1 High availability
      2. 1.2.2 Data replication
    3. 1.3 Business challenges and solutions
    4. 1.4 Business scenarios
      1. 1.4.1 High Availability and Q Replication
      2. 1.4.2 Q Replication at-a-glance
      3. 1.4.3 Conflict avoidance
    5. 1.5 Four client implementation scenarios
      1. 1.5.1 The two-node scenario
      2. 1.5.2 The advanced two-node scenario
      3. 1.5.3 The partially connected three-node scenario
      4. 1.5.4 The fully connected three-node scenario
      5. 1.5.5 Added Q Replication functionality
      6. 1.5.6 Q Replication scenario environment
  5. Chapter 2. IIDR Q Replication overview
    1. 2.1 Q Replication objects
      1. 2.1.1 Q Replication control tables and their architecture level
      2. 2.1.2 Q Replication queue maps and WebSphere MQ objects
      3. 2.1.3 Q subscriptions and choosing a replication topology
    2. 2.2 The Q Capture program
      1. 2.2.1 The DB2 IFI Filtering option
      2. 2.2.2 Heartbeat message and heartbeat interval
    3. 2.3 The Q Apply program
      1. 2.3.1 Conflict detection and conflict resolution
      2. 2.3.2 The Q Apply oldest committed transaction time for a queue
    4. 2.4 Administration tools
    5. 2.5 The Data Replication Dashboard
    6. 2.6 Utilities
    7. 2.7 Load processing
      1. 2.7.1 Internal and external load options
      2. 2.7.2 External load considerations
      3. 2.7.3 Subscription State Transition during load
      4. 2.7.4 Referential Integrity considerations
      5. 2.7.5 Spill queue considerations
  6. Chapter 3. The two-node scenario
    1. 3.1 Introduction to Q Replication
    2. 3.2 Scenario architecture and topology
    3. 3.3 Replicating simple tables
      1. 3.3.1 Preparing your environment
      2. 3.3.2 Replicating DB2 Add Column automatically
      3. 3.3.3 Configuring bidi for optimal performance
      4. 3.3.4 Starting the Q Replication programs
      5. 3.3.5 Checking the status of Q Replication
      6. 3.3.6 Replicating add column
      7. 3.3.7 Replicating alter column set data type
    4. 3.4 Using non-persistent queues
      1. 3.4.1 Simulating a lost message
    5. 3.5 Replicating temporal tables
      1. 3.5.1 Creating temporal tables
      2. 3.5.2 Creating the stored procedure for Q Apply
      3. 3.5.3 Creating Q Subscriptions
      4. 3.5.4 Starting Q Subscriptions
      5. 3.5.5 Checking the status of Q Replication
    6. 3.6 Using parallel send queues
      1. 3.6.1 Configuring WebSphere MQ for parallel send queues
      2. 3.6.2 Starting WebSphere MQ communications
      3. 3.6.3 Creating Q Map for parallel send queues
      4. 3.6.4 Reviewing SENDQ and RECVQ parallel send queue columns
      5. 3.6.5 Creating Q Subscriptions for parallel send queues
      6. 3.6.6 Activating new Q Maps/Q Subscriptions replication
      7. 3.6.7 Replicating data
      8. 3.6.8 Checking the status of Q Replication
  7. Chapter 4. The advanced two-node scenario
    1. 4.1 Introduction to the advanced two-node scenario
    2. 4.2 Scenario architecture and topology
    3. 4.3 Replicating four simple tables
      1. 4.3.1 Preparing your environment
      2. 4.3.2 Starting the replication programs
      3. 4.3.3 Checking the status of Q Replication
    4. 4.4 Altering a replication key
  8. Chapter 5. The three-node scenarios
    1. 5.1 Introduction to three-site replication
    2. 5.2 The three-node partially connected scenario
      1. 5.2.1 Scenario architecture and topology
      2. 5.2.2 Q Replication V10.2.1 and DB2 for z/OS V10 NFM
      3. 5.2.3 Conflict avoidance strategy using DB2 for z/OS
      4. 5.2.4 Recapturing of transactions at the primary node
      5. 5.2.5 Filtering transactions by using the IBMQREP_IGNTRAN table
      6. 5.2.6 Managing exceptions and the IBMQREP_EXCEPTIONS table
      7. 5.2.7 Q Apply program exploiting DB2 for z/OS MRI function
    3. 5.3 The three-node fully connected scenario
      1. 5.3.1 Scenario architecture and topology
      2. 5.3.2 Application key partitioning
      3. 5.3.3 DDL replication and the fully connected topology
      4. 5.3.4 Planned outages with the fully connected topology
      5. 5.3.5 Unplanned outages with the fully connected topology
      6. 5.3.6 Moving from bidi to multi-uni topologies
  9. Chapter 6. Latency analysis
    1. 6.1 Overview
      1. 6.1.1 How this chapter is organized
      2. 6.1.2 The end-to-end latency of a replicated transaction
      3. 6.1.3 The statistics and the monitor tables
    2. 6.2 Q Capture latency
      1. 6.2.1 The Q Capture latency
      2. 6.2.2 Determine if Q Capture is keeping up with the DB2 log activity
      3. 6.2.3 DB2 IFI time
      4. 6.2.4 Determine why (what) Q Capture is waiting (on)
      5. 6.2.5 WebSphere MQ PUT time and MQ COMMIT Time
      6. 6.2.6 Determine if Q Capture spilled a transaction
      7. 6.2.7 Determine if the transmit queue is becoming a bottleneck
      8. 6.2.8 Tuning the Q Capture program
    3. 6.3 Q Apply latency
      1. 6.3.1 The Q Apply latency
      2. 6.3.2 Why Q Apply waits for transaction dependencies
      3. 6.3.3 Determine why Q Apply is waiting for agents
      4. 6.3.4 Determine why Q Apply is retrying so many SQL statements
      5. 6.3.5 DB2 DBMS Time
      6. 6.3.6 Monster transactions
      7. 6.3.7 Tuning the Q Apply program
    4. 6.4 Queue latency
      1. 6.4.1 The Q Capture COMMIT_INTERVAL
      2. 6.4.2 The WebSphere MQ transport delay
      3. 6.4.3 The Q Apply MQGET_TIME
      4. 6.4.4 Tuning WebSphere MQ
    5. 6.5 Batch workloads
    6. 6.6 The Data Replication Dashboard
      1. 6.6.1 Live monitoring
      2. 6.6.2 Generating a latency report
  10. Chapter 7. Managing Q Replication in the DB2 for z/OS Environment
    1. 7.1 Introduction
      1. 7.1.1 The Q Capture versioning tables and schema evolution
    2. 7.2 DB2 utilities
      1. 7.2.1 CAPSTARTing a subscription
      2. 7.2.2 Replicating DB2 utilities and the CAPTURE_LOAD option
      3. 7.2.3 The LOAD and CHECKDATA utilities and DB2 RI constraints
      4. 7.2.4 DB2 utilities and the data compression dictionary
    3. 7.3 DB2 commands and statements
      1. 7.3.1 An alternative to the TRUNCATE command
      2. 7.3.2 Q Replication and the EXCHANGE statement
      3. 7.3.3 The START tablespace RREPL option
      4. 7.3.4 Replicating ARCHIVE tables
    4. 7.4 DB2 DDL Statements
      1. 7.4.1 ALTER ADD COLUMN
      2. 7.4.2 RENAME COLUMN
      3. 7.4.3 ALTER COLUMN SET DATA TYPE
      4. 7.4.4 DROP COLUMN
      5. 7.4.5 DROP TABLE
      6. 7.4.6 Add a column to a replication index
      7. 7.4.7 Add a unique index to a target table
      8. 7.4.8 The ROTATE PARTITION DDL statement
    5. 7.5 DB2 maintained tables and columns
  11. Chapter 8. Recovery procedures
    1. 8.1 Data that is needed to recover
      1. 8.1.1 The Q Capture start lsn and commit lsn values
      2. 8.1.2 The Q Capture restart file
    2. 8.2 Starting Q Capture from an earlier point in the DB2 log
      1. 8.2.1 Manually updating the Q Capture program restart file
      2. 8.2.2 The Recovery Advisor of the Data Replication Dashboard
    3. 8.3 Starting Q Capture from the current point of the DB2 log
    4. 8.4 Stopping Q Capture (or a send queue) at a particular point in DB2 log
    5. 8.5 Recovering from LOAD spill queue full conditions
      1. 8.5.1 Preventing spill queue full condition
      2. 8.5.2 Recuperating from spill queue full condition
  12. Appendix A. Using ASNCLP on z/OS
    1. A.1 ASNCLP on z/OS
  13. Appendix B. Additional material
    1. Locating the Web material
    2. Using the Web material
  14. Related publications
    1. IBM Redbooks
    2. Online resources
    3. Help from IBM
  15. Back cover