You are previewing A Practical Guide to Testing Wireless Smartphone Applications.
O'Reilly logo
A Practical Guide to Testing Wireless Smartphone Applications

Book Description

Testing applications for mobile phones is difficult, time-consuming, and hard to do effectively. Many people have limited their testing efforts to hands-on testing of an application on a few physical handsets, and they have to repeat the process every time a new version of the software is ready to test. They may miss many of the permutations of real-world use, and as a consequence their users are left with the unpleasant mess of a failing application on their phone. Test automation can help to increase the range and scope of testing, while reducing the overhead of manual testing of each version of the software. However automation is not a panacea, particularly for mobile applications, so we need to pick our test automation challenges wisely. This book is intended to help software and test engineers pick appropriately to achieve more; and as a consequence deliver better quality, working software to users. This Synthesis lecture provides practical advice based on direct experience of using software test automation to help improve the testing of a wide range of mobile phone applications, including the latest AJAX applications. The focus is on applications that rely on a wireless network connection to a remote server, however the principles may apply to other related fields and applications. We start by explaining terms and some of the key challenges involved in testing smartphone applications. Subsequent chapters describe a type of application e.g. markup, AJAX, Client, followed by a related chapter on how to test each of these applications. Common test automation techniques are covered in a separate chapter, and finally there is a brief chapter on when to test manually. The book also contains numerous pointers and links to further material to help you to improve your testing using automation appropriately. Table of Contents: Introduction / Markup Languages / Testing Techniques for Markup Applications / AJAX Mobile Applications / Testing Mobile AJAX Applications / Client Applications / Testing Techniques for Client Applications / Common Techniques / When to Test Manually / Future Work / Appendix A: Links and References / Appendix B: Data Connectivity / Appendix C: Configuring Your Machine

Table of Contents

  1. Cover Page
  2. Copyright Page
  3. Title Page
  4. Preface
  5. What is Inside
  6. Conventions
  7. What you will Need
  8. Using Code Examples
  9. Alphabet Soup and the Half-Life of Links
  10. Acknowledgments
  11. Contents
  12. 1 - Introduction
    1. 1.1 - What Is a Mobile Wireless Application?
    2. 1.2 - Classifications of Mobile Wireless Applications
      1. 1.2.1 - Client Applications
      2. 1.2.2 - Messaging Applications
      3. 1.2.3 - Browser Applications
      4. 1.2.4 - The Supporting Servers
      5. 1.2.5 - Things That Do Not Quite Fit
    3. 1.3 - Currently Outside the Scope of This Book
    4. 1.4 - Scope of Mobile Wireless Test Automation
    5. 1.5 - Challenges in Testing Mobile Wireless Applications
    6. 1.6 - Problem Space
      1. 1.6.1 - Transcoding Web Content
    7. 1.7 - Our Testing Focus
    8. 1.8 - Our Goals When Testing
    9. 1.9 - Our Overall Testing Strategy
    10. 1.10 - Core Concepts
  13. 2 - Markup Languages
    1. 2.1 - Examples of Markup Languages
    2. 2.2 - Testing Strategy for Markup Applications
    3. 2.3 - Example Problems
  14. 3 - Testing Techniques for Markup Applications
    1. 3.1 - Getting Started With Test Automation
    2. 3.2 - Examples Written in Python
      1. 3.2.1 - A Test to Detect if Google Maps Is Offered to Mobile Users
      2. 3.2.2 - Using Regular Expressions in Our Test
      3. 3.2.3 - Combining XML With Regular Expressions
      4. 3.2.4 - Using XPATH in Our Tests
    3. 3.3 - Summary of the Examples in Python
    4. 3.4 - Building on Your First Automated Scripts
      1. 3.4.1 - Data-Driven Tests
      2. 3.4.2 - Obtaining Metadata to Drive Our Tests
      3. 3.4.3 - Using Metadata
      4. 3.4.4 - Test Using Carrier Networks
      5. 3.4.5 - Timing the Request/Response Pair
      6. 3.4.6 - Implementing Rule-Based Tests
      7. 3.4.7 - Probe Servers
      8. 3.4.8 - Strengths and Weaknesses of Rule-Based Testing
      9. 3.4.9 - A Complementary Tool to Rule-Based Tests
      10. 3.4.10 - Is Appropriate Content Being Served?
    5. 3.5 - Tips When Implementing Automated Test Scripts
    6. 3.6 - Test Tools for Browser-Based Applications
      1. 3.6.1 - Using Web-Testing Tools
      2. 3.6.2 - “Mobile Readiness” Tools
      3. 3.6.3 - Utilities to Help With Testing Browser Applications
  15. 4 - AJAX Mobile Applications
    1. 4.1 - Testing Challenges for AJAX Mobile Applications
    2. 4.2 - Examples of Problems With Mobile AJAX Applications
  16. 5 - Testing Mobile AJAX Applications
    1. 5.1 - Using Desktop Browser Automation Tools
      1. 5.1.1 - Selenium
      2. 5.1.2 - WebDriver
      3. 5.1.3 - Customizing Desktop Web Browsers
      4. 5.1.4 - Limitations of Using Desktop Web Browsers
    2. 5.2 - Using an Embedded Browser
      1. 5.2.1 - Using Simulators
      2. 5.2.2 - General Tips
      3. 5.2.3 - Selenium Tips for Mobile AJAX Automation
      4. 5.2.4 - WebDriver Tips for Mobile AJAX Automation
  17. 6 - Client Applications
    1. 6.1 - Portable Applications
    2. 6.2 - Native Applications
      1. 6.2.1 - Developing Native Applications
      2. 6.2.2 - Example Problems for Portable Applications
      3. 6.2.3 - Example Problems for Native Applications
    3. 6.3 - Testing Strategy for Client Applications
  18. 7 - Testing Techniques for Client Applications
    1. 7.1 - Automated Unit Tests
      1. 7.1.1 - Examples of Unit Tests
      2. 7.1.2 - Running Unit Tests in an Emulator
    2. 7.2 - System Testing for iPhone Applications
    3. 7.3 - Code Injection
      1. 7.3.1 - Code Injection for Java ME
      2. 7.3.2 - Custom Test Applications
    4. 7.4 - Prober Clients
    5. 7.5 - Signature Testing
    6. 7.6 - Test Tools for Client Applications
      1. 7.6.1 - On-Device Debugging
      2. 7.6.2 - Test Automation of the Runtime Environment
      3. 7.6.3 - Emulators
    7. 7.7 - Test Automation Using the Blackberry Emulator
      1. 7.7.1 - Summary of Testing Techniques for Client Applications
  19. 8 - Common Techniques
    1. 8.1 - GUI-Level Automation
    2. 8.2 - Image Stitching
      1. 8.2.1 - Steps in Image Stitching
      2. 8.2.2 - Challenges of Image Matching
      3. 8.2.3 - Optical Character Recognition
      4. 8.2.4 - Encoding Data in Pixels
      5. 8.2.5 - Making Image Matching Easier
      6. 8.2.6 - Using Advanced Image Matching Techniques
      7. 8.2.7 - Detecting Good and Bad Results
    3. 8.3 - Contact Sheets
    4. 8.3.1 - Using Transparency Masking
      1. 8.3.2 - Combining Automation With Human Judgment
    5. 8.4 - Model Based Testing
  20. 9 - When to Test Manually
    1. 9.1 - Examples of Effective Manual Testing
    2. 9.2 - Computer-Assisted Testing for Mobile Wireless Applications
    3. 9.3 - Testability for Automated Testing
    4. 9.4 - How to Improve Testability
      1. 9.4.1 - Browser-Based Applications
      2. 9.4.2 - Client Applications
      3. 9.4.3 - SMS Applications
      4. 9.4.4 - General
  21. 10 - Future Work
  22. Appendix A Links and References
    1. A.1 Testing Markup (Web Sites)
    2. A.2 J2ME Testing
    3. A.3 Java Byte Code Instrumentation
    4. A.4 Native Application Testing
    5. A.5 Test Automation With Emulators
    6. A.6 SMS Services
    7. A.7 Connectivity
    8. A.8 Miscellaneous Links
    9. A.9 Common Tools
    10. A.10 Other References
    11. A.11 Raw Ingredients
  23. Appendix B Data Connectivity
    1. B.1 How to Use a Mobile Phone Modem Interactively
    2. B.2 How to Use a Mobile Phone Modem for IP Traffic
    3. B.3 Possible Problems With Data Connectivity
    4. B.4 Miscellaneous Problems
  24. Appendix C Configuring your Machine
    1. C.1 Raw Ingredients
    2. C.2 Prepare the Device
    3. C.3 Connecting Your Modem or Phone
    4. C.4 Using Hyperterminal in Windows
    5. C.5 Using Minicom in Linux
    6. C.6 Configuring PPP in Windows
    7. C.7 Configuring PPP in Linux
  25. Author Biography