Wednesday, December 3, 2008

OSDC liveblogging, Unittest: An under-appreciated gem

Unittest: An under-appreciated gem
Andrew Bennetts

What makes a good unit test suite?
- readable: intent and implementation is clear
- reliable: pass/fail when it shoud
- usable: easy to run, fast, debuggable

Very quick overview of unittest:
- Subclass unittest.TestCase
- def setUp(self): foo
- def a bunch of methods (will all be test cases)
- single unit test is represented by a TestCase instance
- then run tearDown
- report the result to a TestResult object

TestResult: passed to methods, record success details
TestLoader: turns test methods int TestCase subclasses into test cases
something else: is glue

Why is this good?
- Standard, most other frameworks will interoperate with it, batts included
- Not Python-centric, xUnit implementation, general programmer awareness
- Encourages good structure in tests
- By default, tests are isolated from eachother
- Easy to reuse a test fixture definition between multiple tests (reuse setUp and tearDown)
- Group tests by common needs
- test cases have explicit names, so clearer reporting, can be specific about what failed & where

Unittest is pretty easy to extend.
- extensions have a good chance of being fairly compatible with other test frameworks
- examined addCleanup logic, add cleanup code immediately after, e.g., adding a lock
- (Bazaar extension) multiply_test_suite_by_scenarios is a function that takes a test suite and a list of scenarios

A couple of extra libs worth knowing about
- miscellaneous extensions from various projects
- runs unit tests in separate processes to support test isolation
- is a library to manage the initialisation and lifetime of expensive text fixtures

Some bits are bad:
- no standard tool to load and invoke a test suite
- parts of the API could be better (lots of 3rd party stuff needed)
- set of built-in assertions is a bit small
- documentation in the std lib is a bit weak

- fundamentally quite capable, pretty easy to usefully extend, would be great to get extensions back into std lib.