Posted on by & filed under python, testing.

Python programmers are fortunate to have a clear, reasonable style guide in PEP8. While PEP8 is widely followed by professional and amateur Python programmers, there’s no widely adopted equivalent style guide for testing in Python.

To help our own team improve code reviews and train newcomers (both to the company and to Python), I’ve started a draft style guide outlining how we write tests for Python code at Safari. It is meant to be expanded and refined over time.

Here’s the outline of the draft style guide (the complete version is available at What would you add or remove?

  • Prerequisites
    • PEP8 first
    • Lint in your editor
    • Prefer imperfect tests to no tests
  • The practice of testing
    • Use Tox
    • Master should always pass 100% of the time
    • Manually run tests often; automatically run tests always
    • Use TDD if it makes sense
    • Pay attention to coverage, but don’t obsess over metrics
    • Remember the QA team: some things are still best tested by humans
    • Delete dead or misleading tests aggressively
    • Focus on tests during code reviews
    • Reproduce before testing
    • Always “Click the thing”: it’s not fixed until you’ve tried it like a real user, regardless of how many unit tests you wrote
    • Write regression tests whenever possible
    • Fail tests first
    • Prefer fewer asserts per test
  • Structure
    • Prefer small tests
    • Separate tests into different files
    • Follow import conventions
  • Write to be read
    • Always write a docstring
    • Use “should” in every docstring
    • Prefer descriptive variables
  • Expectations
    • Always bind an expected value
    • Assert the expected value, then the returned value
    • Prefer explicit expected values
  • Things that people will yell about
    • Never leave behind print statements: use logging
  • Unusual testing conventions
    • Repeat yourself
  • Django
    • Avoid fixtures: they’re fast, but you’ll forget to update them when the data model changes
    • Use CSS selectors to find specific nodes rather than just looking for ‘substring in undifferentiated HTML blob’
    • Prefer extremely targeted functional tests
    • Target testable HTML with CSS classes prefaced with t-, rather than using presentational classes that your front-end team might change.
    • Consider templatetags
  • Testing tools and techniques
    • Use mocks with care
    • Enjoy assertRaises
  • How to avoid traps
    • Really think about boundary values
    • Date-related functionality should test oddball cases like leap years
    • Test the obvious positive and negative cases separately
    • Introduce some randomness: pick random numbers or strings within an expected range
    • Introduce some Unicode: we like upside-down input text or users named .
  • Resources

Tags: automation, Django, pedantry, pep8, Python, testing, tox,


  1.  Zen and the Art of Writing Software | Safari Flow Blog