You are previewing Python Testing Cookbook.
O'Reilly logo
Python Testing Cookbook

Book Description

Making your life easier with automated testing of Python is the sole aim of this book. Because it’s a cookbook, you can take things at your own pace, in your own order, and learn practical application rather than nebulous theory.

  • Learn to write tests at every level using a variety of Python testing tools

  • The first book to include detailed screenshots and recipes for using Jenkins continuous integration server (formerly known as Hudson)

  • Explore innovative ways to introduce automated testing to legacy systems

  • Written by Greg L. Turnquist – senior software engineer and author of Spring Python 1.1

  • Part of Packt's Cookbook series: Each recipe is a carefully organized sequence of instructions to complete the task as efficiently as possible

  • In Detail

    Are you looking at new ways to write better, more efficient tests? Are you struggling to add automated testing to your existing system? The Python unit testing framework, originally referred to as "PyUnit" and now known as unittest, is a framework that makes it easier for you to write automated test suites efficiently in Python. This book will show you exactly how to squeeze every ounce of value out of automated testing.

    The Python Testing Cookbook will empower you to write tests using lots of Python test tools, code samples, screenshots, and detailed explanations. By learning how and when to write tests at every level, you can vastly improve the quality of your code and your personal skill set. Packed with lots of test examples, this will become your go-to book for writing good tests.

    This practical cookbook covers lots of test styles including unit-level, test discovery, doctest, BDD, acceptance, smoke, and load testing. It will guide you to use popular Python tools effectively and discover how to write custom extensions. You will learn how to use popular continuous integration systems like Jenkins (formerly known as Hudson) and TeamCity to automatically test your code upon check in. This book explores Python's built-in ability to run code found embedded in doc strings and also plugging in to popular web testing tools like Selenium. By the end of this book, you will be proficient in many test tactics and be ready to apply them to new applications as well as legacy ones.

    Table of Contents

    1. Python Testing Cookbook
      1. Python Testing Cookbook
      2. Credits
      3. About the Author
      4. About the Reviewers
      5. www.PacktPub.com
        1. Support files, eBooks, discount offers and more
          1. Why Subscribe?
          2. Free Access for Packt account holders
      6. Preface
        1. What this book covers
        2. What you need for this book
        3. Who this book is for
        4. Conventions
        5. Reader feedback
        6. Customer support
          1. Downloading the example code
          2. Errata
          3. Piracy
          4. Questions
      7. 1. Using Unittest To Develop Basic Tests
        1. Introduction
        2. Asserting the basics
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. assertEquals is preferred over assertTrue and assertFalse
            2. self.fail([msg]) can usually be rewritten with assertions
            3. Our version of Python can impact our options
        3. Setting up and tearing down a test harness
          1. How to do it...
          2. How it works...
        4. Running test cases from the command line with increased verbosity
          1. How to do it...
          2. How it works...
          3. There's more...
        5. Running a subset of test case methods
          1. How to do it...
          2. How it works...
        6. Chaining together a suite of tests
          1. How to do it...
          2. How it works...
          3. There's more...
            1. Name of the test case should be significant
        7. Defining test suites inside the test module
          1. How to do it...
          2. How it works...
          3. There's more...
            1. Test suite methods must be outside of the test class
            2. Why have different suites?
            3. optparse is being phased out and replaced by argparse
        8. Retooling old test code to run inside unittest
          1. How to do it...
          2. How it works...
          3. There's more...
            1. Where are the bugs?
            2. FunctionTestCase is a temporary measure
        9. Breaking down obscure tests into simple ones
          1. How to do it...
          2. How it works...
          3. There's more...
            1. Where is the bug?
            2. What is the right size for a test method?
            3. Unittests versus integration tests
        10. Testing the edges
          1. How to do it...
          2. How it works...
          3. There's more...
            1. Identifying the edges is important
            2. Testing for unexpected conditions
        11. Testing corner cases by iteration
          1. How to do it...
          2. How it works...
          3. There's more...
            1. Does this defy the recipe Breaking down obscure tests into simple ones?
            2. How does this compare with the recipe Testing the edges?
          4. See also
      8. 2. Running Automated Test Suites with Nose
        1. Introduction
        2. Getting nosy with testing
          1. How to do it...
          2. How it works...
          3. There's more...
            1. Nose is extensible
            2. Nose is embeddable
          4. See also
        3. Embedding nose inside Python
          1. How to do it...
          2. How it works...
          3. There's more
        4. Writing a nose extension to pick tests based on regular expressions
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
        5. Writing a nose extension to generate a CSV report
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
        6. Writing a project-level script that lets you run different test suites
          1. How to do it...
          2. How it works...
          3. There's more
            1. Why use getopt instead of optparse?
      9. 3. Creating Testable Documentation with doctest
        1. Introduction
        2. Documenting the basics
          1. How to do it...
          2. How it works...
          3. There's more...
        3. Catching stack traces
          1. How to do it...
          2. How it works...
        4. Running doctests from the command line
          1. How to do it...
          2. How it works...
        5. Coding a test harness for doctest
          1. How to do it...
          2. How it works...
          3. There's more...
        6. Filtering out test noise
          1. How to do it...
          2. How it works...
          3. There's more...
        7. Printing out all your documentation including a status report
          1. How to do it...
          2. How it works...
          3. There's more...
        8. Testing the edges
          1. How to do it...
          2. How it works...
          3. There's more...
          4. See also
        9. Testing corner cases by iteration
          1. How to do it...
          2. How it works...
            1. Does this type of test fit better into doctest or unittest?
          3. See also
        10. Getting nosy with doctest
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. See also
        11. Updating the project-level script to run this chapter's doctests
          1. How to do it...
          2. How it works...
          3. There's more
          4. See also
      10. 4. Testing Customer Stories with Behavior Driven Development
        1. Introduction
        2. Naming tests that sound like sentences and stories
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more
            1. The plugin isn't installable
          5. See also
        3. Testing separate doctest documents
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Doesn't this defy the usability of docstrings?
        4. Writing a testable story with doctest
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
        5. Writing a testable novel with doctest
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
        6. Writing a testable story with Voidspace Mock and nose
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Tell me more about the spec nose plugin!
            2. Why didn't we reuse the plugin from the recipe 'Naming tests so they sound like sentences and stories'?
          5. See also
        7. Writing a testable story with mockito and nose
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. See also
        8. Writing a testable story with Lettuce
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. How complex should a story be?
            2. Don't mix wiring code with application code
            3. Lettuce works great using folders
          5. See also
        9. Using Should DSL to write succinct assertions with Lettuce
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
          5. See also
        10. Updating the project-level script to run this chapter's BDD tests
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. See also
      11. 5. High Level Customer Scenarios with Acceptance Testing
        1. Introduction
        2. Installing Pyccuracy
          1. How to do it...
          2. How it works...
          3. See also
        3. Testing the basics with Pyccuracy
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
          5. See also
        4. Using Pyccuracy to verify web app security
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. See also
        5. Installing the Robot Framework
          1. How to do it...
          2. There's more...
        6. Creating a data-driven test suite with Robot
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Do I have to write HTML tables?
            2. What are the best ways to write the code that implements our custom keywords?
            3. Robot Framework variables are unicode
          5. See also
        7. Writing a testable story with Robot
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Given-When-Then results in duplicate rules
            2. Do the try-except blocks violate the idea of keeping things light?
          5. See also
        8. Tagging Robot tests and running a subset
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. What about documentation?
          5. See also
        9. Testing web basics with Robot
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Learn about timing configurations they may be important!
          5. See also
        10. Using Robot to verify web app security
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Why not use a 'remember me' option?
            2. Shouldn't we refactor the first test scenario to use the keyword?
            3. Would arguments make the login keyword more flexible?
          5. See also
        11. Creating a project-level script to verify this chapter's acceptance tests
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Can we only use getopt?
            2. What's wrong with using the various command-line tools?
      12. 6. Integrating Automated Tests with Continuous Integration
        1. Introduction
        2. Generating a continuous integration report for Jenkins using NoseXUnit
          1. Getting ready
          2. How to do it...
          3. How it works...
        3. Configuring Jenkins to run Python tests upon commit
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Do I have to use git for source code management?
            2. What is the format of polling?
          5. See also
        4. Configuring Jenkins to run Python tests when scheduled
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Jenkins versus TeamCity
          5. See also
        5. Generating a CI report for TeamCity using teamcity-nose
          1. Getting ready
          2. How to do it...
          3. How it works...
        6. Configuring TeamCity to run Python tests upon commit
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. What did teamcity-nose give us?
          5. See also
        7. Configuring TeamCity to run Python tests when scheduled
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. See also
      13. 7. Measuring your Success with Test Coverage
        1. Introduction
        2. Building a network management application
          1. How to do it...
          2. How it works...
        3. Installing and running coverage on your test suite
          1. How to do it...
          2. There's more...
            1. Why are there no asserts in the unit test?
        4. Generating an HTML report using coverage
          1. How to do it…
          2. How it works...
        5. Generating an XML report using coverage
          1. How to do it…
          2. How it works...
            1. What use is an XML report?
          3. See also
        6. Getting nosy with coverage
          1. How to do it...
          2. How it works...
          3. There's more...
            1. Why use the nose plugin instead of the coverage tool directly?
            2. Why are sqlite3 and springpython included?
        7. Filtering out test noise from coverage
          1. How to do it...
          2. How it works...
          3. There's more...
          4. See also
        8. Letting Jenkins get nosy with coverage
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Nose doesn't directly support coverage's XML option
        9. Updating the project-level script to provide coverage reports
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more…
            1. Can we only use getopt?
            2. What's wrong with using the various command-line tools?
      14. 8. Smoke/Load Testing — Testing Major Parts
        1. Introduction
        2. Defining a subset of test cases using import statements
          1. How to do it...
          2. How it works...
            1. Security, checking, and integration aren't smoke tests!
            2. What provides good flexibility?
          3. See also
        3. Leaving out integration tests
          1. How to do it...
          2. How it works...
          3. There's more...
            1. Should a smoke test include integration or unit tests?
          4. See also
        4. Targeting end-to-end scenarios
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. How does this define smoke tests?
          5. See also
        5. Targeting the test server
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. How likely is it that a dev versus production environment would use two different database systems?
            2. This isn't just confined to database systems
        6. Coding a data simulator
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. Why does the server script initialize the database?
            2. Why MySQL instead of SQLite?
          5. See also
        7. Recording and playing back live data in real time
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. I thought this recipe was about live data!
            2. Is opening and closing a file for every event a good idea?
            3. What about offloading the storage of data?
          5. See also
        8. Recording and playing back live data as fast as possible
          1. Getting ready
          2. How to do it...
          3. How it works...
          4. There's more...
            1. What is the difference between this and playing back in real time?
            2. Where are the breaking points of this application?
            3. What amount of live data should be collected?
          5. See also
        9. Automating your management demo
          1. How to do it...
          2. How it works...
          3. There's more...
            1. What if my manager likes to take detours?
      15. 9. Good Test Habits for New and Legacy Systems
        1. Introduction
        2. Something is better than nothing
          1. How to do it...
          2. How it works...
            1. That can't be everything!
          3. See also
        3. Coverage isn't everything
          1. How to do it...
          2. How it works...
          3. There's more...
            1. Are we not supposed to increase coverage?
            2. But I want to brag about the coverage of my system!
        4. Be willing to invest in test fixtures
          1. How to do it...
          2. How it works...
          3. There's more...
            1. Is this just about setting up a database?
        5. If you aren't convinced on the value of testing, your team won't be either
        6. Harvesting metrics
          1. How to do it...
          2. How it works...
          3. There's more...
            1. Metrics aren't just for defending yourself to management
        7. Capturing a bug in an automated test
          1. How to do it...
          2. How it works...
          3. There's more...
            1. When the time comes to add a completely new module, you will be ready for it
            2. Don't give in to the temptation to skip testing
        8. Separating algorithms from concurrency
          1. How to do it...
          2. How it works...
            1. Research test options provided by your concurrency frameworks
        9. Pause to refactor when test suite takes too long to run
          1. How to do it...
          2. How it works...
          3. There's more...
          4. See also
        10. Cash in on your confidence
          1. How to do it...
          2. How it works...
        11. Be willing to throw away an entire day of changes
          1. How to do it...
          2. How it works...
          3. There's more...
            1. How does this mesh with "Something is better than nothing"
          4. See also
        12. Instead of shooting for 100 percent coverage, try to have a steady growth
          1. How to do it...
          2. How it works...
        13. Randomly breaking your app can lead to better code
          1. How to do it...
          2. How it works...
          3. There's more...
            1. How does this compare to fuzz testing?
            2. Are there any tools to help with this?