The Art of Application Performance Testing, 2nd Edition

The Art of Application Performance Testing, 2nd Edition

Released December 2014
Publisher(s): O'Reilly Media, Inc.
ISBN: 9781491900543

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Buy on Amazon Buy on ebooks.com
Start your free trial

Book description

Because performance is paramount today, this thoroughly updated guide shows you how to test mission-critical applications for scalability and performance before you deploy them—whether it’s to the cloud or a mobile device. You’ll learn the complete testing process lifecycle step-by-step, along with best practices to plan, coordinate, and conduct performance tests on your applications.

Table of contents

  1. Preface
    1. Audience
    2. About This Book
    3. Conventions Used in This Book
    4. Glossary
    5. Using Code Examples
    6. Safari® Books Online
    7. How to Contact Us
    8. Acknowledgments
  2. 1. Why Performance Test?
    1. What Is Performance? The End-User Perspective
      1. Performance Measurement
      2. Performance Standards
      3. The World Wide Web and Ecommerce
    2. Bad Performance: Why It’s So Common
      1. The IT Business Value Curve
      2. Performance Testing Maturity: What the Analysts Think
      3. Lack of Performance Considerations in Application Design
      4. Performance Testing Is Left to the Last Minute
      5. Scalability
      6. Underestimating Your Popularity
      7. Performance Testing Is Still an Informal Discipline
      8. Not Using Automated Testing Tools
      9. Application Technology Impact
    3. Summary
  3. 2. Choosing an Appropriate Performance Testing Tool
    1. Performance Testing Tool Architecture
    2. Choosing a Performance Testing Tool
    3. Performance Testing Toolset: Proof of Concept
      1. Proof of Concept Checklist
    4. Summary
  4. 3. The Fundamentals of Effective Application Performance Testing
    1. Making Sure Your Application Is Ready
    2. Allocating Enough Time to Performance Test
    3. Obtaining a Code Freeze
    4. Designing a Performance Test Environment
      1. Virtualization
      2. Cloud Computing
      3. Load Injection Capacity
      4. Addressing Different Network Deployment Models
      5. Environment Checklist
      6. Software Installation Constraints
    5. Setting Realistic Performance Targets
      1. Consensus
      2. Performance Target Definition
      3. Network Utilization
      4. Server Utilization
    6. Identifying and Scripting the Business-Critical Use Cases
      1. Use-Case Checklist
      2. Use-Case Replay Validation
      3. What to Measure
      4. To Log In or Not to Log In
      5. Peaceful Coexistence
    7. Providing Test Data
      1. Input Data
      2. Target Data
      3. Session Data
      4. Data Security
    8. Ensuring Accurate Performance-Test Design
      1. Principal Types of Performance Test
      2. The Load Model
      3. Think Time
      4. Pacing
    9. Identifying the KPIs
      1. Server KPIs
      2. Network KPIs
      3. Application Server KPIs
    10. Summary
  5. 4. The Process of Performance Testing
    1. Activity Duration Guidelines
    2. Performance Testing Approach
      1. Step 1: Nonfunctional Requirements Capture
      2. Step 2: Performance Test Environment Build
      3. Step 3: Use-Case Scripting
      4. Step 4: Performance Test Scenario Build
      5. Step 5: Performance Test Execution
      6. Step 6: Post-Test Analysis and Reporting
    3. Case Study 1: Online Banking
      1. Application Landscape
      2. Application Users
      3. Step 1: Pre-Engagement NFR Capture
      4. Step 2: Test Environment Build
      5. Step 3: Use-Case Scripting
      6. Step 4: Performance Test Build
      7. Step 5: Performance Test Execution
      8. Online Banking Case Study Review
    4. Case Study 2: Call Center
      1. Application Landscape
      2. Application Users
      3. Step 1: Pre-Engagement NFR Capture
      4. Step 2: Test Environment Build
      5. Step 3: Use-Case Scripting
      6. Step 4: Performance Test Scenario Build
      7. Step 5: Performance Test Execution
      8. Call Center Case Study Review
    5. Summary
  6. 5. Interpreting Results: Effective Root-Cause Analysis
    1. The Analysis Process
      1. Real-Time Analysis
      2. Post-Test Analysis
    2. Types of Output from a Performance Test
      1. Statistics Primer
      2. Response-Time Measurement
      3. Throughput and Capacity
      4. Monitoring Key Performance Indicators
      5. Server KPI Performance
      6. Network KPI Performance
      7. Load Injector Performance
    3. Root-Cause Analysis
      1. Scalability and Response Time
      2. Digging Deeper
      3. Inside the Application Server
      4. Looking for the Knee
      5. Dealing with Errors
      6. Baseline Data
    4. Analysis Checklist
      1. Pre-Test Tasks
      2. Tasks During Test Execution
      3. Post-Test Tasks
    5. Summary
  7. 6. Performance Testing and the Mobile Client
    1. What’s Different About a Mobile Client?
    2. Mobile Testing Automation
    3. Mobile Design Considerations
    4. Mobile Testing Considerations
    5. Mobile Test Design
      1. On-Device Performance Not in Scope
      2. On-Device Performance Testing Is in Scope
    6. Summary
  8. 7. End-User Experience Monitoring and Performance
    1. What Is External Monitoring?
    2. Why Monitor Externally?
    3. External Monitoring Categories
    4. Active Monitoring
      1. Output Metrics
      2. ISP Testing Best Practices
      3. Synthetic End-User Testing Best Practices
    5. Passive Monitoring
      1. How Passive Monitoring Works
    6. Pros and Cons of Active Versus Passive Monitoring
      1. Active Pros
      2. Active Cons
      3. Passive Pros
      4. Passive Cons
    7. Tooling for External Monitoring of Internet Applications
      1. Tool Selection Criteria
      2. Active Monitoring Tooling
      3. Passive Monitoring Tooling
    8. Creating an External Monitoring Testing Framework
      1. Building Blocks of an Effective Testing Framework
      2. Specific Design Aspects of Active Monitoring
      3. Specific Design Aspects of Passive Monitoring
    9. Isolating and Characterizing Issues Using External Monitoring
    10. Monitoring Native Mobile Applications
    11. Essential Considerations for CDN Monitoring
    12. Performance Results Interpretation
      1. Key Performance Indicators for Web-Based Ecommerce Applications
      2. Setting KPI Values
    13. The Application Performance Index (APDEX)
    14. Management Information
      1. Data Preparation
      2. Statistical Considerations
      3. Correlation
    15. Effective Reporting
    16. Competitive Understanding
      1. Visitor performance map
    17. Alerting
      1. Gotchas!
    18. Summary
  9. 8. Integrating External Monitoring and Performance Testing
    1. Tooling Choices
    2. Active and Passive Integration with Static Performance Testing
      1. Passive and Performance Testing
      2. RUM and APM
      3. Integration of Active Test Traffic with APM Tooling
      4. Active External Monitoring and Performance Testing
    3. Test Approach
      1. Test Scheduling
      2. Performance Testing of Multimedia Content
      3. End-User Understanding in Non-Internet Application Performance Tests
    4. Useful Source Materials
    5. Summary
  10. 9. Application Technology and Its Impact on Performance Testing
    1. Asynchronous Java and XML (AJAX)
      1. Push Versus Pull
    2. Citrix
      1. Citrix Checklist
      2. Citrix Scripting Advice
    3. Virtual Desktop Infrastructure
    4. HTTP Protocol
      1. Web Services
      2. .NET Remoting
      3. Browser Caching
      4. Secure Sockets Layer
    5. Java
    6. Oracle
      1. Oracle Two-Tier
      2. Oracle Forms Server
      3. Oracle Checklist
    7. SAP
      1. SAP Checklist
    8. Service-Oriented Architecture
    9. Web 2.0
      1. Windows Communication Foundation and Windows Presentation Foundation
    10. Oddball Application Technologies: Help, My Load Testing Tool Won’t Record It!
      1. Before Giving Up in Despair . . .
      2. Alternatives to Capture at the Middleware Level
      3. Manual Scripting
    11. Summary
  11. 10. Conclusion
  12. A. Use-Case Definition Example
  13. B. Proof of Concept and Performance Test Quick Reference
    1. The Proof of Concept
      1. POC Checklist
    2. Performance Test Execution Checklist
      1. Activity Duration Guidelines
      2. Step 1: Pre-Engagement NFR Capture
      3. Step 2: Test Environment Build
      4. Step 3: Scripting
      5. Step 4: Performance Test Build
      6. Step 5: Performance Test Execution
      7. Step 6 (Post-Test Phase): Analyze Results, Report, Retest If Required
    3. Analysis Checklist
      1. Pre-Test Analysis Tasks
      2. Tasks During Test Execution
      3. Post-Test Tasks
  14. C. Performance and Testing Tool Vendors
    1. Application Performance Management
    2. End-User Experience and Website Monitoring
    3. Functional Testing
    4. Performance Testing
      1. Open Source
    5. SaaS Performance Testing
    6. Requirements Management
      1. Open Source
  15. D. Sample Monitoring Templates: Infrastructure Key Performance Indicator Metrics
    1. Generic KPI Templates
      1. Windows OS : Generic KPI Template
      2. Linux/Unix: Generic KPI Template
    2. Application-Specific KPI Templates
      1. Windows OS: MS SQL Server KPI Template
  16. E. Sample Project Plan
  17. Index

Product information

  • Title: The Art of Application Performance Testing, 2nd Edition
  • Author(s):
  • Release date: December 2014
  • Publisher(s): O'Reilly Media, Inc.
  • ISBN: 9781491900543