Ignore:
Timestamp:
Mar 19, 2014, 11:31:01 PM (11 years ago)
Author:
dmik
Message:

python: Merge vendor 2.7.6 to trunk.

Location:
python/trunk
Files:
2 edited

Legend:

Unmodified
Added
Removed

  • python/trunk

  • python/trunk/Doc/library/unittest.rst

    r2 r391  
    1 
    21:mod:`unittest` --- Unit testing framework
    32==========================================
    … …  
    109.. sectionauthor:: Raymond Hettinger <python@rcn.com>
    1110
    12 
    1311.. versionadded:: 2.1
     12
     13(If you are already familiar with the basic concepts of testing, you might want
     14to skip to :ref:`the list of assert methods <assert-methods>`.)
    1415
    1516The Python unit testing framework, sometimes referred to as "PyUnit," is a
    … …  
    5051used when creating new tests, and the latter can be used when integrating
    5152existing test code with a :mod:`unittest`\ -driven framework. When building test
    52 fixtures using :class:`TestCase`, the :meth:`setUp` and :meth:`tearDown` methods
    53 can be overridden to provide initialization and cleanup for the fixture.  With
    54 :class:`FunctionTestCase`, existing functions can be passed to the constructor
    55 for these purposes.  When the test is run, the fixture initialization is run
    56 first; if it succeeds, the cleanup method is run after the test has been
    57 executed, regardless of the outcome of the test.  Each instance of the
    58 :class:`TestCase` will only be used to run a single test method, so a new
    59 fixture is created for each test.
     53fixtures using :class:`TestCase`, the :meth:`~TestCase.setUp` and
     54:meth:`~TestCase.tearDown` methods can be overridden to provide initialization
     55and cleanup for the fixture.  With :class:`FunctionTestCase`, existing functions
     56can be passed to the constructor for these purposes.  When the test is run, the
     57fixture initialization is run first; if it succeeds, the cleanup method is run
     58after the test has been executed, regardless of the outcome of the test.  Each
     59instance of the :class:`TestCase` will only be used to run a single test method,
     60so a new fixture is created for each test.
    6061
    6162Test suites are implemented by the :class:`TestSuite` class.  This class allows
    … …  
    6364all tests added directly to the suite and in "child" test suites are run.
    6465
    65 A test runner is an object that provides a single method, :meth:`run`, which
    66 accepts a :class:`TestCase` or :class:`TestSuite` object as a parameter, and
    67 returns a result object.  The class :class:`TestResult` is provided for use as
    68 the result object. :mod:`unittest` provides the :class:`TextTestRunner` as an
    69 example test runner which reports test results on the standard error stream by
    70 default.  Alternate runners can be implemented for other environments (such as
    71 graphical environments) without any need to derive from a specific class.
     66A test runner is an object that provides a single method,
     67:meth:`~TestRunner.run`, which accepts a :class:`TestCase` or :class:`TestSuite`
     68object as a parameter, and returns a result object.  The class
     69:class:`TestResult` is provided for use as the result object. :mod:`unittest`
     70provides the :class:`TextTestRunner` as an example test runner which reports
     71test results on the standard error stream by default.  Alternate runners can be
     72implemented for other environments (such as graphical environments) without any
     73need to derive from a specific class.
    7274
    7375
    … …  
    7779      Another test-support module with a very different flavor.
    7880
     81   `unittest2: A backport of new unittest features for Python 2.4-2.6 <http://pypi.python.org/pypi/unittest2>`_
     82      Many new features were added to unittest in Python 2.7, including test
     83      discovery. unittest2 allows you to use these features with earlier
     84      versions of Python.
     85
    7986   `Simple Smalltalk Testing: With Patterns <http://www.XProgramming.com/testfram.htm>`_
    80       Kent Beck's original paper on testing frameworks using the pattern shared by
    81       :mod:`unittest`.
     87      Kent Beck's original paper on testing frameworks using the pattern shared
     88      by :mod:`unittest`.
    8289
    8390   `Nose <http://code.google.com/p/python-nose/>`_ and `py.test <http://pytest.org>`_
    84       Third-party unittest frameworks with a lighter-weight syntax
    85       for writing tests.  For example, ``assert func(10) == 42``.
    86 
    87    `python-mock <http://python-mock.sourceforge.net/>`_ and `minimock <http://blog.ianbicking.org/minimock.html>`_
    88       Tools for creating mock test objects (objects simulating external resources).
     91      Third-party unittest frameworks with a lighter-weight syntax for writing
     92      tests.  For example, ``assert func(10) == 42``.
     93
     94   `The Python Testing Tools Taxonomy <http://wiki.python.org/moin/PythonTestingToolsTaxonomy>`_
     95      An extensive list of Python testing tools including functional testing
     96      frameworks and mock object libraries.
     97
     98   `Testing in Python Mailing List <http://lists.idyll.org/listinfo/testing-in-python>`_
     99      A special-interest-group for discussion of testing, and testing tools,
     100      in Python.
     101
    89102
    90103.. _unittest-minimal-example:
    … …  
    113126           self.assertEqual(self.seq, range(10))
    114127
     128           # should raise an exception for an immutable sequence
     129           self.assertRaises(TypeError, random.shuffle, (1,2,3))
     130
    115131       def test_choice(self):
    116132           element = random.choice(self.seq)
    … …  
    118134
    119135       def test_sample(self):
    120            self.assertRaises(ValueError, random.sample, self.seq, 20)
     136           with self.assertRaises(ValueError):
     137               random.sample(self.seq, 20)
    121138           for element in random.sample(self.seq, 5):
    122139               self.assertTrue(element in self.seq)
    … …  
    125142       unittest.main()
    126143
    127 A testcase is created by subclassing :class:`unittest.TestCase`. The three
     144A testcase is created by subclassing :class:`unittest.TestCase`.  The three
    128145individual tests are defined with methods whose names start with the letters
    129146``test``.  This naming convention informs the test runner about which methods
    130147represent tests.
    131148
    132 The crux of each test is a call to :meth:`assertEqual` to check for an expected
    133 result; :meth:`assert_` to verify a condition; or :meth:`assertRaises` to verify
    134 that an expected exception gets raised.  These methods are used instead of the
    135 :keyword:`assert` statement so the test runner can accumulate all test results
    136 and produce a report.
    137 
    138 When a :meth:`setUp` method is defined, the test runner will run that method
    139 prior to each test.  Likewise, if a :meth:`tearDown` method is defined, the test
    140 runner will invoke that method after each test.  In the example, :meth:`setUp`
    141 was used to create a fresh sequence for each test.
     149The crux of each test is a call to :meth:`~TestCase.assertEqual` to check for an
     150expected result; :meth:`~TestCase.assertTrue` to verify a condition; or
     151:meth:`~TestCase.assertRaises` to verify that an expected exception gets raised.
     152These methods are used instead of the :keyword:`assert` statement so the test
     153runner can accumulate all test results and produce a report.
     154
     155When a :meth:`~TestCase.setUp` method is defined, the test runner will run that
     156method prior to each test.  Likewise, if a :meth:`~TestCase.tearDown` method is
     157defined, the test runner will invoke that method after each test.  In the
     158example, :meth:`~TestCase.setUp` was used to create a fresh sequence for each
     159test.
    142160
    143161The final block shows a simple way to run the tests. :func:`unittest.main`
    144 provides a command line interface to the test script.  When run from the command
     162provides a command-line interface to the test script.  When run from the command
    145163line, the above script produces an output that looks like this::
    146164
    … …  
    173191are sufficient to meet many everyday testing needs.  The remainder of the
    174192documentation explores the full feature set from first principles.
     193
     194
     195.. _unittest-command-line-interface:
     196
     197Command-Line Interface
     198----------------------
     199
     200The unittest module can be used from the command line to run tests from
     201modules, classes or even individual test methods::
     202
     203   python -m unittest test_module1 test_module2
     204   python -m unittest test_module.TestClass
     205   python -m unittest test_module.TestClass.test_method
     206
     207You can pass in a list with any combination of module names, and fully
     208qualified class or method names.
     209
     210You can run tests with more detail (higher verbosity) by passing in the -v flag::
     211
     212   python -m unittest -v test_module
     213
     214For a list of all the command-line options::
     215
     216   python -m unittest -h
     217
     218..  versionchanged:: 2.7
     219   In earlier versions it was only possible to run individual test methods and
     220   not modules or classes.
     221
     222
     223Command-line options
     224~~~~~~~~~~~~~~~~~~~~
     225
     226:program:`unittest` supports these command-line options:
     227
     228.. program:: unittest
     229
     230.. cmdoption:: -b, --buffer
     231
     232   The standard output and standard error streams are buffered during the test
     233   run. Output during a passing test is discarded. Output is echoed normally
     234   on test fail or error and is added to the failure messages.
     235
     236.. cmdoption:: -c, --catch
     237
     238   Control-C during the test run waits for the current test to end and then
     239   reports all the results so far. A second control-C raises the normal
     240   :exc:`KeyboardInterrupt` exception.
     241
     242   See `Signal Handling`_ for the functions that provide this functionality.
     243
     244.. cmdoption:: -f, --failfast
     245
     246   Stop the test run on the first error or failure.
     247
     248.. versionadded:: 2.7
     249   The command-line options ``-b``, ``-c`` and ``-f`` were added.
     250
     251The command line can also be used for test discovery, for running all of the
     252tests in a project or just a subset.
     253
     254
     255.. _unittest-test-discovery:
     256
     257Test Discovery
     258--------------
     259
     260.. versionadded:: 2.7
     261
     262Unittest supports simple test discovery. In order to be compatible with test
     263discovery, all of the test files must be :ref:`modules <tut-modules>` or
     264:ref:`packages <tut-packages>` importable from the top-level directory of
     265the project (this means that their filenames must be valid
     266:ref:`identifiers <identifiers>`).
     267
     268Test discovery is implemented in :meth:`TestLoader.discover`, but can also be
     269used from the command line. The basic command-line usage is::
     270
     271   cd project_directory
     272   python -m unittest discover
     273
     274The ``discover`` sub-command has the following options:
     275
     276.. program:: unittest discover
     277
     278.. cmdoption:: -v, --verbose
     279
     280   Verbose output
     281
     282.. cmdoption:: -s, --start-directory directory
     283
     284   Directory to start discovery (``.`` default)
     285
     286.. cmdoption:: -p, --pattern pattern
     287
     288   Pattern to match test files (``test*.py`` default)
     289
     290.. cmdoption:: -t, --top-level-directory directory
     291
     292   Top level directory of project (defaults to start directory)
     293
     294The :option:`-s`, :option:`-p`, and :option:`-t` options can be passed in
     295as positional arguments in that order. The following two command lines
     296are equivalent::
     297
     298   python -m unittest discover -s project_directory -p '*_test.py'
     299   python -m unittest discover project_directory '*_test.py'
     300
     301As well as being a path it is possible to pass a package name, for example
     302``myproject.subpackage.test``, as the start directory. The package name you
     303supply will then be imported and its location on the filesystem will be used
     304as the start directory.
     305
     306.. caution::
     307
     308    Test discovery loads tests by importing them. Once test discovery has
     309    found all the test files from the start directory you specify it turns the
     310    paths into package names to import. For example :file:`foo/bar/baz.py` will be
     311    imported as ``foo.bar.baz``.
     312
     313    If you have a package installed globally and attempt test discovery on
     314    a different copy of the package then the import *could* happen from the
     315    wrong place. If this happens test discovery will warn you and exit.
     316
     317    If you supply the start directory as a package name rather than a
     318    path to a directory then discover assumes that whichever location it
     319    imports from is the location you intended, so you will not get the
     320    warning.
     321
     322Test modules and packages can customize test loading and discovery by through
     323the `load_tests protocol`_.
    175324
    176325
    … …  
    194343combination with any number of other test cases.
    195344
    196 The simplest :class:`TestCase` subclass will simply override the :meth:`runTest`
    197 method in order to perform specific testing code::
     345The simplest :class:`TestCase` subclass will simply override the
     346:meth:`~TestCase.runTest` method in order to perform specific testing code::
    198347
    199348   import unittest
    … …  
    204353           self.assertEqual(widget.size(), (50, 50), 'incorrect default size')
    205354
    206 Note that in order to test something, we use the one of the :meth:`assert\*` or
    207 :meth:`fail\*` methods provided by the :class:`TestCase` base class.  If the
    208 test fails, an exception will be raised, and :mod:`unittest` will identify the
    209 test case as a :dfn:`failure`.  Any other exceptions will be treated as
    210 :dfn:`errors`. This helps you identify where the problem is: :dfn:`failures` are
    211 caused by incorrect results - a 5 where you expected a 6. :dfn:`Errors` are
    212 caused by incorrect code - e.g., a :exc:`TypeError` caused by an incorrect
    213 function call.
     355Note that in order to test something, we use one of the :meth:`assert\*`
     356methods provided by the :class:`TestCase` base class.  If the test fails, an
     357exception will be raised, and :mod:`unittest` will identify the test case as a
     358:dfn:`failure`.  Any other exceptions will be treated as :dfn:`errors`. This
     359helps you identify where the problem is: :dfn:`failures` are caused by incorrect
     360results - a 5 where you expected a 6. :dfn:`Errors` are caused by incorrect
     361code - e.g., a :exc:`TypeError` caused by an incorrect function call.
    214362
    215363The way to run a test case will be described later.  For now, note that to
    … …  
    224372
    225373Luckily, we can factor out such set-up code by implementing a method called
    226 :meth:`setUp`, which the testing framework will automatically call for us when
    227 we run the test::
     374:meth:`~TestCase.setUp`, which the testing framework will automatically call for
     375us when we run the test::
    228376
    229377   import unittest
    … …  
    244392                            'wrong size after resize')
    245393
    246 If the :meth:`setUp` method raises an exception while the test is running, the
    247 framework will consider the test to have suffered an error, and the
    248 :meth:`runTest` method will not be executed.
    249 
    250 Similarly, we can provide a :meth:`tearDown` method that tidies up after the
    251 :meth:`runTest` method has been run::
     394If the :meth:`~TestCase.setUp` method raises an exception while the test is
     395running, the framework will consider the test to have suffered an error, and the
     396:meth:`~TestCase.runTest` method will not be executed.
     397
     398Similarly, we can provide a :meth:`~TestCase.tearDown` method that tidies up
     399after the :meth:`~TestCase.runTest` method has been run::
    252400
    253401   import unittest
    … …  
    261409           self.widget = None
    262410
    263 If :meth:`setUp` succeeded, the :meth:`tearDown` method will be run whether
    264 :meth:`runTest` succeeded or not.
     411If :meth:`~TestCase.setUp` succeeded, the :meth:`~TestCase.tearDown` method will
     412be run whether :meth:`~TestCase.runTest` succeeded or not.
    265413
    266414Such a working environment for the testing code is called a :dfn:`fixture`.
    … …  
    337485name prefix to identify test methods automatically.
    338486
    339 Note that the order in which the various test cases will be run is determined by
    340 sorting the test function names with the built-in :func:`cmp` function.
     487Note that the order in which the various test cases will be run is
     488determined by sorting the test function names with respect to the
     489built-in ordering for strings.
    341490
    342491Often it is desirable to group suites of test cases together, so as to run tests
    … …  
    410559.. note::
    411560
    412    Even though :class:`FunctionTestCase` can be used to quickly convert an existing
    413    test base over to a :mod:`unittest`\ -based system, this approach is not
    414    recommended.  Taking the time to set up proper :class:`TestCase` subclasses will
    415    make future test refactorings infinitely easier.
     561   Even though :class:`FunctionTestCase` can be used to quickly convert an
     562   existing test base over to a :mod:`unittest`\ -based system, this approach is
     563   not recommended.  Taking the time to set up proper :class:`TestCase`
     564   subclasses will make future test refactorings infinitely easier.
     565
     566In some cases, the existing tests may have been written using the :mod:`doctest`
     567module.  If so, :mod:`doctest` provides a :class:`DocTestSuite` class that can
     568automatically build :class:`unittest.TestSuite` instances from the existing
     569:mod:`doctest`\ -based tests.
     570
     571
     572.. _unittest-skipping:
     573
     574Skipping tests and expected failures
     575------------------------------------
     576
     577.. versionadded:: 2.7
     578
     579Unittest supports skipping individual test methods and even whole classes of
     580tests.  In addition, it supports marking a test as a "expected failure," a test
     581that is broken and will fail, but shouldn't be counted as a failure on a
     582:class:`TestResult`.
     583
     584Skipping a test is simply a matter of using the :func:`skip` :term:`decorator`
     585or one of its conditional variants.
     586
     587Basic skipping looks like this::
     588
     589   class MyTestCase(unittest.TestCase):
     590
     591       @unittest.skip("demonstrating skipping")
     592       def test_nothing(self):
     593           self.fail("shouldn't happen")
     594
     595       @unittest.skipIf(mylib.__version__ < (1, 3),
     596                        "not supported in this library version")
     597       def test_format(self):
     598           # Tests that work for only a certain version of the library.
     599           pass
     600
     601       @unittest.skipUnless(sys.platform.startswith("win"), "requires Windows")
     602       def test_windows_support(self):
     603           # windows specific testing code
     604           pass
     605
     606This is the output of running the example above in verbose mode::
     607
     608   test_format (__main__.MyTestCase) ... skipped 'not supported in this library version'
     609   test_nothing (__main__.MyTestCase) ... skipped 'demonstrating skipping'
     610   test_windows_support (__main__.MyTestCase) ... skipped 'requires Windows'
     611
     612   ----------------------------------------------------------------------
     613   Ran 3 tests in 0.005s
     614
     615   OK (skipped=3)
     616
     617Classes can be skipped just like methods::
     618
     619   @unittest.skip("showing class skipping")
     620   class MySkippedTestCase(unittest.TestCase):
     621       def test_not_run(self):
     622           pass
     623
     624:meth:`TestCase.setUp` can also skip the test.  This is useful when a resource
     625that needs to be set up is not available.
     626
     627Expected failures use the :func:`expectedFailure` decorator. ::
     628
     629   class ExpectedFailureTestCase(unittest.TestCase):
     630       @unittest.expectedFailure
     631       def test_fail(self):
     632           self.assertEqual(1, 0, "broken")
     633
     634It's easy to roll your own skipping decorators by making a decorator that calls
     635:func:`skip` on the test when it wants it to be skipped.  This decorator skips
     636the test unless the passed object has a certain attribute::
     637
     638   def skipUnlessHasattr(obj, attr):
     639       if hasattr(obj, attr):
     640           return lambda func: func
     641       return unittest.skip("{!r} doesn't have {!r}".format(obj, attr))
     642
     643The following decorators implement test skipping and expected failures:
     644
     645.. function:: skip(reason)
     646
     647   Unconditionally skip the decorated test.  *reason* should describe why the
     648   test is being skipped.
     649
     650.. function:: skipIf(condition, reason)
     651
     652   Skip the decorated test if *condition* is true.
     653
     654.. function:: skipUnless(condition, reason)
     655
     656   Skip the decorated test unless *condition* is true.
     657
     658.. function:: expectedFailure
     659
     660   Mark the test as an expected failure.  If the test fails when run, the test
     661   is not counted as a failure.
     662
     663.. exception:: SkipTest(reason)
     664
     665   This exception is raised to skip a test.
     666
     667   Usually you can use :meth:`TestCase.skipTest` or one of the skipping
     668   decorators instead of raising this directly.
     669
     670Skipped tests will not have :meth:`setUp` or :meth:`tearDown` run around them.
     671Skipped classes will not have :meth:`setUpClass` or :meth:`tearDownClass` run.
    416672
    417673
    … …  
    421677---------------------
    422678
    423 
    424 .. class:: TestCase([methodName])
     679This section describes in depth the API of :mod:`unittest`.
     680
     681
     682.. _testcase-objects:
     683
     684Test cases
     685~~~~~~~~~~
     686
     687.. class:: TestCase(methodName='runTest')
    425688
    426689   Instances of the :class:`TestCase` class represent the smallest testable units
    … …  
    444707   single test.
    445708
    446    *methodName* defaults to ``'runTest'``.
    447 
    448 
    449 .. class:: FunctionTestCase(testFunc[, setUp[, tearDown[, description]]])
     709   *methodName* defaults to :meth:`runTest`.
     710
     711   :class:`TestCase` instances provide three groups of methods: one group used
     712   to run the test, another used by the test implementation to check conditions
     713   and report failures, and some inquiry methods allowing information about the
     714   test itself to be gathered.
     715
     716   Methods in the first group (running the test) are:
     717
     718
     719   .. method:: setUp()
     720
     721      Method called to prepare the test fixture.  This is called immediately
     722      before calling the test method; any exception raised by this method will
     723      be considered an error rather than a test failure. The default
     724      implementation does nothing.
     725
     726
     727   .. method:: tearDown()
     728
     729      Method called immediately after the test method has been called and the
     730      result recorded.  This is called even if the test method raised an
     731      exception, so the implementation in subclasses may need to be particularly
     732      careful about checking internal state.  Any exception raised by this
     733      method will be considered an error rather than a test failure.  This
     734      method will only be called if the :meth:`setUp` succeeds, regardless of
     735      the outcome of the test method. The default implementation does nothing.
     736
     737
     738   .. method:: setUpClass()
     739
     740      A class method called before tests in an individual class run.
     741      ``setUpClass`` is called with the class as the only argument
     742      and must be decorated as a :func:`classmethod`::
     743
     744        @classmethod
     745        def setUpClass(cls):
     746            ...
     747
     748      See `Class and Module Fixtures`_ for more details.
     749
     750      .. versionadded:: 2.7
     751
     752
     753   .. method:: tearDownClass()
     754
     755      A class method called after tests in an individual class have run.
     756      ``tearDownClass`` is called with the class as the only argument
     757      and must be decorated as a :meth:`classmethod`::
     758
     759        @classmethod
     760        def tearDownClass(cls):
     761            ...
     762
     763      See `Class and Module Fixtures`_ for more details.
     764
     765      .. versionadded:: 2.7
     766
     767
     768   .. method:: run(result=None)
     769
     770      Run the test, collecting the result into the test result object passed as
     771      *result*.  If *result* is omitted or ``None``, a temporary result
     772      object is created (by calling the :meth:`defaultTestResult` method) and
     773      used. The result object is not returned to :meth:`run`'s caller.
     774
     775      The same effect may be had by simply calling the :class:`TestCase`
     776      instance.
     777
     778
     779   .. method:: skipTest(reason)
     780
     781      Calling this during a test method or :meth:`setUp` skips the current
     782      test.  See :ref:`unittest-skipping` for more information.
     783
     784      .. versionadded:: 2.7
     785
     786
     787   .. method:: debug()
     788
     789      Run the test without collecting the result.  This allows exceptions raised
     790      by the test to be propagated to the caller, and can be used to support
     791      running tests under a debugger.
     792
     793   .. _assert-methods:
     794
     795   The :class:`TestCase` class provides a number of methods to check for and
     796   report failures, such as:
     797
     798   +-----------------------------------------+-----------------------------+---------------+
     799   | Method                                  | Checks that                 | New in        |
     800   +=========================================+=============================+===============+
     801   | :meth:`assertEqual(a, b)                | ``a == b``                  |               |
     802   | <TestCase.assertEqual>`                 |                             |               |
     803   +-----------------------------------------+-----------------------------+---------------+
     804   | :meth:`assertNotEqual(a, b)             | ``a != b``                  |               |
     805   | <TestCase.assertNotEqual>`              |                             |               |
     806   +-----------------------------------------+-----------------------------+---------------+
     807   | :meth:`assertTrue(x)                    | ``bool(x) is True``         |               |
     808   | <TestCase.assertTrue>`                  |                             |               |
     809   +-----------------------------------------+-----------------------------+---------------+
     810   | :meth:`assertFalse(x)                   | ``bool(x) is False``        |               |
     811   | <TestCase.assertFalse>`                 |                             |               |
     812   +-----------------------------------------+-----------------------------+---------------+
     813   | :meth:`assertIs(a, b)                   | ``a is b``                  | 2.7           |
     814   | <TestCase.assertIs>`                    |                             |               |
     815   +-----------------------------------------+-----------------------------+---------------+
     816   | :meth:`assertIsNot(a, b)                | ``a is not b``              | 2.7           |
     817   | <TestCase.assertIsNot>`                 |                             |               |
     818   +-----------------------------------------+-----------------------------+---------------+
     819   | :meth:`assertIsNone(x)                  | ``x is None``               | 2.7           |
     820   | <TestCase.assertIsNone>`                |                             |               |
     821   +-----------------------------------------+-----------------------------+---------------+
     822   | :meth:`assertIsNotNone(x)               | ``x is not None``           | 2.7           |
     823   | <TestCase.assertIsNotNone>`             |                             |               |
     824   +-----------------------------------------+-----------------------------+---------------+
     825   | :meth:`assertIn(a, b)                   | ``a in b``                  | 2.7           |
     826   | <TestCase.assertIn>`                    |                             |               |
     827   +-----------------------------------------+-----------------------------+---------------+
     828   | :meth:`assertNotIn(a, b)                | ``a not in b``              | 2.7           |
     829   | <TestCase.assertNotIn>`                 |                             |               |
     830   +-----------------------------------------+-----------------------------+---------------+
     831   | :meth:`assertIsInstance(a, b)           | ``isinstance(a, b)``        | 2.7           |
     832   | <TestCase.assertIsInstance>`            |                             |               |
     833   +-----------------------------------------+-----------------------------+---------------+
     834   | :meth:`assertNotIsInstance(a, b)        | ``not isinstance(a, b)``    | 2.7           |
     835   | <TestCase.assertNotIsInstance>`         |                             |               |
     836   +-----------------------------------------+-----------------------------+---------------+
     837
     838   All the assert methods (except :meth:`assertRaises`,
     839   :meth:`assertRaisesRegexp`)
     840   accept a *msg* argument that, if specified, is used as the error message on
     841   failure (see also :data:`longMessage`).
     842
     843   .. method:: assertEqual(first, second, msg=None)
     844
     845      Test that *first* and *second* are equal.  If the values do not compare
     846      equal, the test will fail.
     847
     848      In addition, if *first* and *second* are the exact same type and one of
     849      list, tuple, dict, set, frozenset or unicode or any type that a subclass
     850      registers with :meth:`addTypeEqualityFunc` the type-specific equality
     851      function will be called in order to generate a more useful default
     852      error message (see also the :ref:`list of type-specific methods
     853      <type-specific-methods>`).
     854
     855      .. versionchanged:: 2.7
     856         Added the automatic calling of type-specific equality function.
     857
     858
     859   .. method:: assertNotEqual(first, second, msg=None)
     860
     861      Test that *first* and *second* are not equal.  If the values do compare
     862      equal, the test will fail.
     863
     864   .. method:: assertTrue(expr, msg=None)
     865               assertFalse(expr, msg=None)
     866
     867      Test that *expr* is true (or false).
     868
     869      Note that this is equivalent to ``bool(expr) is True`` and not to ``expr
     870      is True`` (use ``assertIs(expr, True)`` for the latter).  This method
     871      should also be avoided when more specific methods are available (e.g.
     872      ``assertEqual(a, b)`` instead of ``assertTrue(a == b)``), because they
     873      provide a better error message in case of failure.
     874
     875
     876   .. method:: assertIs(first, second, msg=None)
     877               assertIsNot(first, second, msg=None)
     878
     879      Test that *first* and *second* evaluate (or don't evaluate) to the same object.
     880
     881      .. versionadded:: 2.7
     882
     883
     884   .. method:: assertIsNone(expr, msg=None)
     885               assertIsNotNone(expr, msg=None)
     886
     887      Test that *expr* is (or is not) None.
     888
     889      .. versionadded:: 2.7
     890
     891
     892   .. method:: assertIn(first, second, msg=None)
     893               assertNotIn(first, second, msg=None)
     894
     895      Test that *first* is (or is not) in *second*.
     896
     897      .. versionadded:: 2.7
     898
     899
     900   .. method:: assertIsInstance(obj, cls, msg=None)
     901               assertNotIsInstance(obj, cls, msg=None)
     902
     903      Test that *obj* is (or is not) an instance of *cls* (which can be a
     904      class or a tuple of classes, as supported by :func:`isinstance`).
     905      To check for the exact type, use :func:`assertIs(type(obj), cls) <assertIs>`.
     906
     907      .. versionadded:: 2.7
     908
     909
     910   It is also possible to check that exceptions and warnings are raised using
     911   the following methods:
     912
     913   +---------------------------------------------------------+--------------------------------------+------------+
     914   | Method                                                  | Checks that                          | New in     |
     915   +=========================================================+======================================+============+
     916   | :meth:`assertRaises(exc, fun, *args, **kwds)            | ``fun(*args, **kwds)`` raises *exc*  |            |
     917   | <TestCase.assertRaises>`                                |                                      |            |
     918   +---------------------------------------------------------+--------------------------------------+------------+
     919   | :meth:`assertRaisesRegexp(exc, r, fun, *args, **kwds)   | ``fun(*args, **kwds)`` raises *exc*  | 2.7        |
     920   | <TestCase.assertRaisesRegexp>`                          | and the message matches regex *r*    |            |
     921   +---------------------------------------------------------+--------------------------------------+------------+
     922
     923   .. method:: assertRaises(exception, callable, *args, **kwds)
     924               assertRaises(exception)
     925
     926      Test that an exception is raised when *callable* is called with any
     927      positional or keyword arguments that are also passed to
     928      :meth:`assertRaises`.  The test passes if *exception* is raised, is an
     929      error if another exception is raised, or fails if no exception is raised.
     930      To catch any of a group of exceptions, a tuple containing the exception
     931      classes may be passed as *exception*.
     932
     933      If only the *exception* argument is given, returns a context manager so
     934      that the code under test can be written inline rather than as a function::
     935
     936         with self.assertRaises(SomeException):
     937             do_something()
     938
     939      The context manager will store the caught exception object in its
     940      :attr:`exception` attribute.  This can be useful if the intention
     941      is to perform additional checks on the exception raised::
     942
     943        with self.assertRaises(SomeException) as cm:
     944            do_something()
     945
     946        the_exception = cm.exception
     947        self.assertEqual(the_exception.error_code, 3)
     948
     949      .. versionchanged:: 2.7
     950         Added the ability to use :meth:`assertRaises` as a context manager.
     951
     952
     953   .. method:: assertRaisesRegexp(exception, regexp, callable, *args, **kwds)
     954               assertRaisesRegexp(exception, regexp)
     955
     956      Like :meth:`assertRaises` but also tests that *regexp* matches
     957      on the string representation of the raised exception.  *regexp* may be
     958      a regular expression object or a string containing a regular expression
     959      suitable for use by :func:`re.search`.  Examples::
     960
     961         self.assertRaisesRegexp(ValueError, "invalid literal for.*XYZ'$",
     962                                 int, 'XYZ')
     963
     964      or::
     965
     966         with self.assertRaisesRegexp(ValueError, 'literal'):
     967            int('XYZ')
     968
     969      .. versionadded:: 2.7
     970
     971
     972
     973   There are also other methods used to perform more specific checks, such as:
     974
     975   +---------------------------------------+--------------------------------+--------------+
     976   | Method                                | Checks that                    | New in       |
     977   +=======================================+================================+==============+
     978   | :meth:`assertAlmostEqual(a, b)        | ``round(a-b, 7) == 0``         |              |
     979   | <TestCase.assertAlmostEqual>`         |                                |              |
     980   +---------------------------------------+--------------------------------+--------------+
     981   | :meth:`assertNotAlmostEqual(a, b)     | ``round(a-b, 7) != 0``         |              |
     982   | <TestCase.assertNotAlmostEqual>`      |                                |              |
     983   +---------------------------------------+--------------------------------+--------------+
     984   | :meth:`assertGreater(a, b)            | ``a > b``                      | 2.7          |
     985   | <TestCase.assertGreater>`             |                                |              |
     986   +---------------------------------------+--------------------------------+--------------+
     987   | :meth:`assertGreaterEqual(a, b)       | ``a >= b``                     | 2.7          |
     988   | <TestCase.assertGreaterEqual>`        |                                |              |
     989   +---------------------------------------+--------------------------------+--------------+
     990   | :meth:`assertLess(a, b)               | ``a < b``                      | 2.7          |
     991   | <TestCase.assertLess>`                |                                |              |
     992   +---------------------------------------+--------------------------------+--------------+
     993   | :meth:`assertLessEqual(a, b)          | ``a <= b``                     | 2.7          |
     994   | <TestCase.assertLessEqual>`           |                                |              |
     995   +---------------------------------------+--------------------------------+--------------+
     996   | :meth:`assertRegexpMatches(s, r)      | ``r.search(s)``                | 2.7          |
     997   | <TestCase.assertRegexpMatches>`       |                                |              |
     998   +---------------------------------------+--------------------------------+--------------+
     999   | :meth:`assertNotRegexpMatches(s, r)   | ``not r.search(s)``            | 2.7          |
     1000   | <TestCase.assertNotRegexpMatches>`    |                                |              |
     1001   +---------------------------------------+--------------------------------+--------------+
     1002   | :meth:`assertItemsEqual(a, b)         | sorted(a) == sorted(b) and     | 2.7          |
     1003   | <TestCase.assertItemsEqual>`          | works with unhashable objs     |              |
     1004   +---------------------------------------+--------------------------------+--------------+
     1005   | :meth:`assertDictContainsSubset(a, b) | all the key/value pairs        | 2.7          |
     1006   | <TestCase.assertDictContainsSubset>`  | in *a* exist in *b*            |              |
     1007   +---------------------------------------+--------------------------------+--------------+
     1008
     1009
     1010   .. method:: assertAlmostEqual(first, second, places=7, msg=None, delta=None)
     1011               assertNotAlmostEqual(first, second, places=7, msg=None, delta=None)
     1012
     1013      Test that *first* and *second* are approximately (or not approximately)
     1014      equal by computing the difference, rounding to the given number of
     1015      decimal *places* (default 7), and comparing to zero.  Note that these
     1016      methods round the values to the given number of *decimal places* (i.e.
     1017      like the :func:`round` function) and not *significant digits*.
     1018
     1019      If *delta* is supplied instead of *places* then the difference
     1020      between *first* and *second* must be less or equal to (or greater than) *delta*.
     1021
     1022      Supplying both *delta* and *places* raises a ``TypeError``.
     1023
     1024      .. versionchanged:: 2.7
     1025         :meth:`assertAlmostEqual` automatically considers almost equal objects
     1026         that compare equal.  :meth:`assertNotAlmostEqual` automatically fails
     1027         if the objects compare equal.  Added the *delta* keyword argument.
     1028
     1029
     1030
     1031   .. method:: assertGreater(first, second, msg=None)
     1032               assertGreaterEqual(first, second, msg=None)
     1033               assertLess(first, second, msg=None)
     1034               assertLessEqual(first, second, msg=None)
     1035
     1036      Test that *first* is respectively >, >=, < or <= than *second* depending
     1037      on the method name.  If not, the test will fail::
     1038
     1039         >>> self.assertGreaterEqual(3, 4)
     1040         AssertionError: "3" unexpectedly not greater than or equal to "4"
     1041
     1042      .. versionadded:: 2.7
     1043
     1044
     1045   .. method:: assertRegexpMatches(text, regexp, msg=None)
     1046
     1047      Test that a *regexp* search matches *text*.  In case
     1048      of failure, the error message will include the pattern and the *text* (or
     1049      the pattern and the part of *text* that unexpectedly matched).  *regexp*
     1050      may be a regular expression object or a string containing a regular
     1051      expression suitable for use by :func:`re.search`.
     1052
     1053      .. versionadded:: 2.7
     1054
     1055
     1056   .. method:: assertNotRegexpMatches(text, regexp, msg=None)
     1057
     1058      Verifies that a *regexp* search does not match *text*.  Fails with an error
     1059      message including the pattern and the part of *text* that matches.  *regexp*
     1060      may be a regular expression object or a string containing a regular
     1061      expression suitable for use by :func:`re.search`.
     1062
     1063      .. versionadded:: 2.7
     1064
     1065
     1066   .. method:: assertItemsEqual(actual, expected, msg=None)
     1067
     1068      Test that sequence *expected* contains the same elements as *actual*,
     1069      regardless of their order. When they don't, an error message listing the
     1070      differences between the sequences will be generated.
     1071
     1072      Duplicate elements are *not* ignored when comparing *actual* and
     1073      *expected*. It verifies if each element has the same count in both
     1074      sequences. It is the equivalent of ``assertEqual(sorted(expected),
     1075      sorted(actual))`` but it works with sequences of unhashable objects as
     1076      well.
     1077
     1078      In Python 3, this method is named ``assertCountEqual``.
     1079
     1080      .. versionadded:: 2.7
     1081
     1082
     1083   .. method:: assertDictContainsSubset(expected, actual, msg=None)
     1084
     1085      Tests whether the key/value pairs in dictionary *actual* are a
     1086      superset of those in *expected*.  If not, an error message listing
     1087      the missing keys and mismatched values is generated.
     1088
     1089      .. versionadded:: 2.7
     1090      .. deprecated:: 3.2
     1091
     1092
     1093
     1094   .. _type-specific-methods:
     1095
     1096   The :meth:`assertEqual` method dispatches the equality check for objects of
     1097   the same type to different type-specific methods.  These methods are already
     1098   implemented for most of the built-in types, but it's also possible to
     1099   register new methods using :meth:`addTypeEqualityFunc`:
     1100
     1101   .. method:: addTypeEqualityFunc(typeobj, function)
     1102
     1103      Registers a type-specific method called by :meth:`assertEqual` to check
     1104      if two objects of exactly the same *typeobj* (not subclasses) compare
     1105      equal.  *function* must take two positional arguments and a third msg=None
     1106      keyword argument just as :meth:`assertEqual` does.  It must raise
     1107      :data:`self.failureException(msg) <failureException>` when inequality
     1108      between the first two parameters is detected -- possibly providing useful
     1109      information and explaining the inequalities in details in the error
     1110      message.
     1111
     1112      .. versionadded:: 2.7
     1113
     1114   The list of type-specific methods automatically used by
     1115   :meth:`~TestCase.assertEqual` are summarized in the following table.  Note
     1116   that it's usually not necessary to invoke these methods directly.
     1117
     1118   +-----------------------------------------+-----------------------------+--------------+
     1119   | Method                                  | Used to compare             | New in       |
     1120   +=========================================+=============================+==============+
     1121   | :meth:`assertMultiLineEqual(a, b)       | strings                     | 2.7          |
     1122   | <TestCase.assertMultiLineEqual>`        |                             |              |
     1123   +-----------------------------------------+-----------------------------+--------------+
     1124   | :meth:`assertSequenceEqual(a, b)        | sequences                   | 2.7          |
     1125   | <TestCase.assertSequenceEqual>`         |                             |              |
     1126   +-----------------------------------------+-----------------------------+--------------+
     1127   | :meth:`assertListEqual(a, b)            | lists                       | 2.7          |
     1128   | <TestCase.assertListEqual>`             |                             |              |
     1129   +-----------------------------------------+-----------------------------+--------------+
     1130   | :meth:`assertTupleEqual(a, b)           | tuples                      | 2.7          |
     1131   | <TestCase.assertTupleEqual>`            |                             |              |
     1132   +-----------------------------------------+-----------------------------+--------------+
     1133   | :meth:`assertSetEqual(a, b)             | sets or frozensets          | 2.7          |
     1134   | <TestCase.assertSetEqual>`              |                             |              |
     1135   +-----------------------------------------+-----------------------------+--------------+
     1136   | :meth:`assertDictEqual(a, b)            | dicts                       | 2.7          |
     1137   | <TestCase.assertDictEqual>`             |                             |              |
     1138   +-----------------------------------------+-----------------------------+--------------+
     1139
     1140
     1141
     1142   .. method:: assertMultiLineEqual(first, second, msg=None)
     1143
     1144      Test that the multiline string *first* is equal to the string *second*.
     1145      When not equal a diff of the two strings highlighting the differences
     1146      will be included in the error message. This method is used by default
     1147      when comparing strings with :meth:`assertEqual`.
     1148
     1149      .. versionadded:: 2.7
     1150
     1151
     1152   .. method:: assertSequenceEqual(seq1, seq2, msg=None, seq_type=None)
     1153
     1154      Tests that two sequences are equal.  If a *seq_type* is supplied, both
     1155      *seq1* and *seq2* must be instances of *seq_type* or a failure will
     1156      be raised.  If the sequences are different an error message is
     1157      constructed that shows the difference between the two.
     1158
     1159      This method is not called directly by :meth:`assertEqual`, but
     1160      it's used to implement :meth:`assertListEqual` and
     1161      :meth:`assertTupleEqual`.
     1162
     1163      .. versionadded:: 2.7
     1164
     1165
     1166   .. method:: assertListEqual(list1, list2, msg=None)
     1167               assertTupleEqual(tuple1, tuple2, msg=None)
     1168
     1169      Tests that two lists or tuples are equal.  If not, an error message is
     1170      constructed that shows only the differences between the two.  An error
     1171      is also raised if either of the parameters are of the wrong type.
     1172      These methods are used by default when comparing lists or tuples with
     1173      :meth:`assertEqual`.
     1174
     1175      .. versionadded:: 2.7
     1176
     1177
     1178   .. method:: assertSetEqual(set1, set2, msg=None)
     1179
     1180      Tests that two sets are equal.  If not, an error message is constructed
     1181      that lists the differences between the sets.  This method is used by
     1182      default when comparing sets or frozensets with :meth:`assertEqual`.
     1183
     1184      Fails if either of *set1* or *set2* does not have a :meth:`set.difference`
     1185      method.
     1186
     1187      .. versionadded:: 2.7
     1188
     1189
     1190   .. method:: assertDictEqual(expected, actual, msg=None)
     1191
     1192      Test that two dictionaries are equal.  If not, an error message is
     1193      constructed that shows the differences in the dictionaries. This
     1194      method will be used by default to compare dictionaries in
     1195      calls to :meth:`assertEqual`.
     1196
     1197      .. versionadded:: 2.7
     1198
     1199
     1200
     1201   .. _other-methods-and-attrs:
     1202
     1203   Finally the :class:`TestCase` provides the following methods and attributes:
     1204
     1205
     1206   .. method:: fail(msg=None)
     1207
     1208      Signals a test failure unconditionally, with *msg* or ``None`` for
     1209      the error message.
     1210
     1211
     1212   .. attribute:: failureException
     1213
     1214      This class attribute gives the exception raised by the test method.  If a
     1215      test framework needs to use a specialized exception, possibly to carry
     1216      additional information, it must subclass this exception in order to "play
     1217      fair" with the framework.  The initial value of this attribute is
     1218      :exc:`AssertionError`.
     1219
     1220
     1221   .. attribute:: longMessage
     1222
     1223      If set to ``True`` then any explicit failure message you pass in to the
     1224      :ref:`assert methods <assert-methods>` will be appended to the end of the
     1225      normal failure message.  The normal messages contain useful information
     1226      about the objects involved, for example the message from assertEqual
     1227      shows you the repr of the two unequal objects. Setting this attribute
     1228      to ``True`` allows you to have a custom error message in addition to the
     1229      normal one.
     1230
     1231      This attribute defaults to ``False``, meaning that a custom message passed
     1232      to an assert method will silence the normal message.
     1233
     1234      The class setting can be overridden in individual tests by assigning an
     1235      instance attribute to ``True`` or ``False`` before calling the assert methods.
     1236
     1237      .. versionadded:: 2.7
     1238
     1239
     1240   .. attribute:: maxDiff
     1241
     1242      This attribute controls the maximum length of diffs output by assert
     1243      methods that report diffs on failure. It defaults to 80*8 characters.
     1244      Assert methods affected by this attribute are
     1245      :meth:`assertSequenceEqual` (including all the sequence comparison
     1246      methods that delegate to it), :meth:`assertDictEqual` and
     1247      :meth:`assertMultiLineEqual`.
     1248
     1249      Setting ``maxDiff`` to None means that there is no maximum length of
     1250      diffs.
     1251
     1252      .. versionadded:: 2.7
     1253
     1254
     1255   Testing frameworks can use the following methods to collect information on
     1256   the test:
     1257
     1258
     1259   .. method:: countTestCases()
     1260
     1261      Return the number of tests represented by this test object.  For
     1262      :class:`TestCase` instances, this will always be ``1``.
     1263
     1264
     1265   .. method:: defaultTestResult()
     1266
     1267      Return an instance of the test result class that should be used for this
     1268      test case class (if no other result instance is provided to the
     1269      :meth:`run` method).
     1270
     1271      For :class:`TestCase` instances, this will always be an instance of
     1272      :class:`TestResult`; subclasses of :class:`TestCase` should override this
     1273      as necessary.
     1274
     1275
     1276   .. method:: id()
     1277
     1278      Return a string identifying the specific test case.  This is usually the
     1279      full name of the test method, including the module and class name.
     1280
     1281
     1282   .. method:: shortDescription()
     1283
     1284      Returns a description of the test, or ``None`` if no description
     1285      has been provided.  The default implementation of this method
     1286      returns the first line of the test method's docstring, if available,
     1287      or :const:`None`.
     1288
     1289
     1290
     1291   .. method:: addCleanup(function, *args, **kwargs)
     1292
     1293      Add a function to be called after :meth:`tearDown` to cleanup resources
     1294      used during the test. Functions will be called in reverse order to the
     1295      order they are added (LIFO). They are called with any arguments and
     1296      keyword arguments passed into :meth:`addCleanup` when they are
     1297      added.
     1298
     1299      If :meth:`setUp` fails, meaning that :meth:`tearDown` is not called,
     1300      then any cleanup functions added will still be called.
     1301
     1302      .. versionadded:: 2.7
     1303
     1304
     1305   .. method:: doCleanups()
     1306
     1307      This method is called unconditionally after :meth:`tearDown`, or
     1308      after :meth:`setUp` if :meth:`setUp` raises an exception.
     1309
     1310      It is responsible for calling all the cleanup functions added by
     1311      :meth:`addCleanup`. If you need cleanup functions to be called
     1312      *prior* to :meth:`tearDown` then you can call :meth:`doCleanups`
     1313      yourself.
     1314
     1315      :meth:`doCleanups` pops methods off the stack of cleanup
     1316      functions one at a time, so it can be called at any time.
     1317
     1318      .. versionadded:: 2.7
     1319
     1320
     1321.. class:: FunctionTestCase(testFunc, setUp=None, tearDown=None, description=None)
    4501322
    4511323   This class implements the portion of the :class:`TestCase` interface which
    452    allows the test runner to drive the test, but does not provide the methods which
    453    test code can use to check and report errors. This is used to create test cases
    454    using legacy test code, allowing it to be integrated into a :mod:`unittest`\
    455    -based test framework.
    456 
    457 
    458 .. class:: TestSuite([tests])
     1324   allows the test runner to drive the test, but does not provide the methods
     1325   which test code can use to check and report errors.  This is used to create
     1326   test cases using legacy test code, allowing it to be integrated into a
     1327   :mod:`unittest`-based test framework.
     1328
     1329
     1330Deprecated aliases
     1331##################
     1332
     1333For historical reasons, some of the :class:`TestCase` methods had one or more
     1334aliases that are now deprecated.  The following table lists the correct names
     1335along with their deprecated aliases:
     1336
     1337   ==============================  ===============================
     1338    Method Name                     Deprecated alias(es)
     1339   ==============================  ===============================
     1340    :meth:`.assertEqual`            failUnlessEqual, assertEquals
     1341    :meth:`.assertNotEqual`         failIfEqual
     1342    :meth:`.assertTrue`             failUnless, assert\_
     1343    :meth:`.assertFalse`            failIf
     1344    :meth:`.assertRaises`           failUnlessRaises
     1345    :meth:`.assertAlmostEqual`      failUnlessAlmostEqual
     1346    :meth:`.assertNotAlmostEqual`   failIfAlmostEqual
     1347   ==============================  ===============================
     1348
     1349   .. deprecated:: 2.7
     1350         the aliases listed in the second column
     1351
     1352
     1353
     1354.. _testsuite-objects:
     1355
     1356Grouping tests
     1357~~~~~~~~~~~~~~
     1358
     1359.. class:: TestSuite(tests=())
    4591360
    4601361   This class represents an aggregation of individual tests cases and test suites.
    … …  
    4671368   are provided to add test cases and suites to the collection later on.
    4681369
     1370   :class:`TestSuite` objects behave much like :class:`TestCase` objects, except
     1371   they do not actually implement a test.  Instead, they are used to aggregate
     1372   tests into groups of tests that should be run together. Some additional
     1373   methods are available to add tests to :class:`TestSuite` instances:
     1374
     1375
     1376   .. method:: TestSuite.addTest(test)
     1377
     1378      Add a :class:`TestCase` or :class:`TestSuite` to the suite.
     1379
     1380
     1381   .. method:: TestSuite.addTests(tests)
     1382
     1383      Add all the tests from an iterable of :class:`TestCase` and :class:`TestSuite`
     1384      instances to this test suite.
     1385
     1386      This is equivalent to iterating over *tests*, calling :meth:`addTest` for
     1387      each element.
     1388
     1389   :class:`TestSuite` shares the following methods with :class:`TestCase`:
     1390
     1391
     1392   .. method:: run(result)
     1393
     1394      Run the tests associated with this suite, collecting the result into the
     1395      test result object passed as *result*.  Note that unlike
     1396      :meth:`TestCase.run`, :meth:`TestSuite.run` requires the result object to
     1397      be passed in.
     1398
     1399
     1400   .. method:: debug()
     1401
     1402      Run the tests associated with this suite without collecting the
     1403      result. This allows exceptions raised by the test to be propagated to the
     1404      caller and can be used to support running tests under a debugger.
     1405
     1406
     1407   .. method:: countTestCases()
     1408
     1409      Return the number of tests represented by this test object, including all
     1410      individual tests and sub-suites.
     1411
     1412
     1413   .. method:: __iter__()
     1414
     1415      Tests grouped by a :class:`TestSuite` are always accessed by iteration.
     1416      Subclasses can lazily provide tests by overriding :meth:`__iter__`. Note
     1417      that this method maybe called several times on a single suite
     1418      (for example when counting tests or comparing for equality)
     1419      so the tests returned must be the same for repeated iterations.
     1420
     1421      .. versionchanged:: 2.7
     1422         In earlier versions the :class:`TestSuite` accessed tests directly rather
     1423         than through iteration, so overriding :meth:`__iter__` wasn't sufficient
     1424         for providing tests.
     1425
     1426   In the typical usage of a :class:`TestSuite` object, the :meth:`run` method
     1427   is invoked by a :class:`TestRunner` rather than by the end-user test harness.
     1428
     1429
     1430Loading and running tests
     1431~~~~~~~~~~~~~~~~~~~~~~~~~
    4691432
    4701433.. class:: TestLoader()
    4711434
    472    This class is responsible for loading tests according to various criteria and
    473    returning them wrapped in a :class:`TestSuite`. It can load all tests within a
    474    given module or :class:`TestCase` subclass.
    475 
    476 
    477 .. class:: TestResult()
    478 
    479    This class is used to compile information about which tests have succeeded and
    480    which have failed.
    481 
     1435   The :class:`TestLoader` class is used to create test suites from classes and
     1436   modules.  Normally, there is no need to create an instance of this class; the
     1437   :mod:`unittest` module provides an instance that can be shared as
     1438   :data:`unittest.defaultTestLoader`.  Using a subclass or instance, however,
     1439   allows customization of some configurable properties.
     1440
     1441   :class:`TestLoader` objects have the following methods:
     1442
     1443
     1444   .. method:: loadTestsFromTestCase(testCaseClass)
     1445
     1446      Return a suite of all tests cases contained in the :class:`TestCase`\ -derived
     1447      :class:`testCaseClass`.
     1448
     1449
     1450   .. method:: loadTestsFromModule(module)
     1451
     1452      Return a suite of all tests cases contained in the given module. This
     1453      method searches *module* for classes derived from :class:`TestCase` and
     1454      creates an instance of the class for each test method defined for the
     1455      class.
     1456
     1457      .. note::
     1458
     1459         While using a hierarchy of :class:`TestCase`\ -derived classes can be
     1460         convenient in sharing fixtures and helper functions, defining test
     1461         methods on base classes that are not intended to be instantiated
     1462         directly does not play well with this method.  Doing so, however, can
     1463         be useful when the fixtures are different and defined in subclasses.
     1464
     1465      If a module provides a ``load_tests`` function it will be called to
     1466      load the tests. This allows modules to customize test loading.
     1467      This is the `load_tests protocol`_.
     1468
     1469      .. versionchanged:: 2.7
     1470         Support for ``load_tests`` added.
     1471
     1472
     1473   .. method:: loadTestsFromName(name, module=None)
     1474
     1475      Return a suite of all tests cases given a string specifier.
     1476
     1477      The specifier *name* is a "dotted name" that may resolve either to a
     1478      module, a test case class, a test method within a test case class, a
     1479      :class:`TestSuite` instance, or a callable object which returns a
     1480      :class:`TestCase` or :class:`TestSuite` instance.  These checks are
     1481      applied in the order listed here; that is, a method on a possible test
     1482      case class will be picked up as "a test method within a test case class",
     1483      rather than "a callable object".
     1484
     1485      For example, if you have a module :mod:`SampleTests` containing a
     1486      :class:`TestCase`\ -derived class :class:`SampleTestCase` with three test
     1487      methods (:meth:`test_one`, :meth:`test_two`, and :meth:`test_three`), the
     1488      specifier ``'SampleTests.SampleTestCase'`` would cause this method to
     1489      return a suite which will run all three test methods. Using the specifier
     1490      ``'SampleTests.SampleTestCase.test_two'`` would cause it to return a test
     1491      suite which will run only the :meth:`test_two` test method. The specifier
     1492      can refer to modules and packages which have not been imported; they will
     1493      be imported as a side-effect.
     1494
     1495      The method optionally resolves *name* relative to the given *module*.
     1496
     1497
     1498   .. method:: loadTestsFromNames(names, module=None)
     1499
     1500      Similar to :meth:`loadTestsFromName`, but takes a sequence of names rather
     1501      than a single name.  The return value is a test suite which supports all
     1502      the tests defined for each name.
     1503
     1504
     1505   .. method:: getTestCaseNames(testCaseClass)
     1506
     1507      Return a sorted sequence of method names found within *testCaseClass*;
     1508      this should be a subclass of :class:`TestCase`.
     1509
     1510
     1511   .. method:: discover(start_dir, pattern='test*.py', top_level_dir=None)
     1512
     1513      Find and return all test modules from the specified start directory,
     1514      recursing into subdirectories to find them. Only test files that match
     1515      *pattern* will be loaded. (Using shell style pattern matching.) Only
     1516      module names that are importable (i.e. are valid Python identifiers) will
     1517      be loaded.
     1518
     1519      All test modules must be importable from the top level of the project. If
     1520      the start directory is not the top level directory then the top level
     1521      directory must be specified separately.
     1522
     1523      If importing a module fails, for example due to a syntax error, then this
     1524      will be recorded as a single error and discovery will continue.
     1525
     1526      If a test package name (directory with :file:`__init__.py`) matches the
     1527      pattern then the package will be checked for a ``load_tests``
     1528      function. If this exists then it will be called with *loader*, *tests*,
     1529      *pattern*.
     1530
     1531      If load_tests exists then discovery does *not* recurse into the package,
     1532      ``load_tests`` is responsible for loading all tests in the package.
     1533
     1534      The pattern is deliberately not stored as a loader attribute so that
     1535      packages can continue discovery themselves. *top_level_dir* is stored so
     1536      ``load_tests`` does not need to pass this argument in to
     1537      ``loader.discover()``.
     1538
     1539      *start_dir* can be a dotted module name as well as a directory.
     1540
     1541      .. versionadded:: 2.7
     1542
     1543   The following attributes of a :class:`TestLoader` can be configured either by
     1544   subclassing or assignment on an instance:
     1545
     1546
     1547   .. attribute:: testMethodPrefix
     1548
     1549      String giving the prefix of method names which will be interpreted as test
     1550      methods.  The default value is ``'test'``.
     1551
     1552      This affects :meth:`getTestCaseNames` and all the :meth:`loadTestsFrom\*`
     1553      methods.
     1554
     1555
     1556   .. attribute:: sortTestMethodsUsing
     1557
     1558      Function to be used to compare method names when sorting them in
     1559      :meth:`getTestCaseNames` and all the :meth:`loadTestsFrom\*` methods. The
     1560      default value is the built-in :func:`cmp` function; the attribute can also
     1561      be set to :const:`None` to disable the sort.
     1562
     1563
     1564   .. attribute:: suiteClass
     1565
     1566      Callable object that constructs a test suite from a list of tests. No
     1567      methods on the resulting object are needed.  The default value is the
     1568      :class:`TestSuite` class.
     1569
     1570      This affects all the :meth:`loadTestsFrom\*` methods.
     1571
     1572
     1573.. class:: TestResult
     1574
     1575   This class is used to compile information about which tests have succeeded
     1576   and which have failed.
     1577
     1578   A :class:`TestResult` object stores the results of a set of tests.  The
     1579   :class:`TestCase` and :class:`TestSuite` classes ensure that results are
     1580   properly recorded; test authors do not need to worry about recording the
     1581   outcome of tests.
     1582
     1583   Testing frameworks built on top of :mod:`unittest` may want access to the
     1584   :class:`TestResult` object generated by running a set of tests for reporting
     1585   purposes; a :class:`TestResult` instance is returned by the
     1586   :meth:`TestRunner.run` method for this purpose.
     1587
     1588   :class:`TestResult` instances have the following attributes that will be of
     1589   interest when inspecting the results of running a set of tests:
     1590
     1591
     1592   .. attribute:: errors
     1593
     1594      A list containing 2-tuples of :class:`TestCase` instances and strings
     1595      holding formatted tracebacks. Each tuple represents a test which raised an
     1596      unexpected exception.
     1597
     1598      .. versionchanged:: 2.2
     1599         Contains formatted tracebacks instead of :func:`sys.exc_info` results.
     1600
     1601
     1602   .. attribute:: failures
     1603
     1604      A list containing 2-tuples of :class:`TestCase` instances and strings
     1605      holding formatted tracebacks. Each tuple represents a test where a failure
     1606      was explicitly signalled using the :meth:`TestCase.assert\*` methods.
     1607
     1608      .. versionchanged:: 2.2
     1609         Contains formatted tracebacks instead of :func:`sys.exc_info` results.
     1610
     1611   .. attribute:: skipped
     1612
     1613      A list containing 2-tuples of :class:`TestCase` instances and strings
     1614      holding the reason for skipping the test.
     1615
     1616      .. versionadded:: 2.7
     1617
     1618   .. attribute:: expectedFailures
     1619
     1620      A list containing 2-tuples of :class:`TestCase` instances and strings
     1621      holding formatted tracebacks.  Each tuple represents an expected failure
     1622      of the test case.
     1623
     1624   .. attribute:: unexpectedSuccesses
     1625
     1626      A list containing :class:`TestCase` instances that were marked as expected
     1627      failures, but succeeded.
     1628
     1629   .. attribute:: shouldStop
     1630
     1631      Set to ``True`` when the execution of tests should stop by :meth:`stop`.
     1632
     1633
     1634   .. attribute:: testsRun
     1635
     1636      The total number of tests run so far.
     1637
     1638
     1639   .. attribute:: buffer
     1640
     1641      If set to true, ``sys.stdout`` and ``sys.stderr`` will be buffered in between
     1642      :meth:`startTest` and :meth:`stopTest` being called. Collected output will
     1643      only be echoed onto the real ``sys.stdout`` and ``sys.stderr`` if the test
     1644      fails or errors. Any output is also attached to the failure / error message.
     1645
     1646      .. versionadded:: 2.7
     1647
     1648
     1649   .. attribute:: failfast
     1650
     1651      If set to true :meth:`stop` will be called on the first failure or error,
     1652      halting the test run.
     1653
     1654      .. versionadded:: 2.7
     1655
     1656
     1657   .. method:: wasSuccessful()
     1658
     1659      Return ``True`` if all tests run so far have passed, otherwise returns
     1660      ``False``.
     1661
     1662
     1663   .. method:: stop()
     1664
     1665      This method can be called to signal that the set of tests being run should
     1666      be aborted by setting the :attr:`shouldStop` attribute to ``True``.
     1667      :class:`TestRunner` objects should respect this flag and return without
     1668      running any additional tests.
     1669
     1670      For example, this feature is used by the :class:`TextTestRunner` class to
     1671      stop the test framework when the user signals an interrupt from the
     1672      keyboard.  Interactive tools which provide :class:`TestRunner`
     1673      implementations can use this in a similar manner.
     1674
     1675   The following methods of the :class:`TestResult` class are used to maintain
     1676   the internal data structures, and may be extended in subclasses to support
     1677   additional reporting requirements.  This is particularly useful in building
     1678   tools which support interactive reporting while tests are being run.
     1679
     1680
     1681   .. method:: startTest(test)
     1682
     1683      Called when the test case *test* is about to be run.
     1684
     1685   .. method:: stopTest(test)
     1686
     1687      Called after the test case *test* has been executed, regardless of the
     1688      outcome.
     1689
     1690   .. method:: startTestRun(test)
     1691
     1692      Called once before any tests are executed.
     1693
     1694      .. versionadded:: 2.7
     1695
     1696
     1697   .. method:: stopTestRun(test)
     1698
     1699      Called once after all tests are executed.
     1700
     1701      .. versionadded:: 2.7
     1702
     1703
     1704   .. method:: addError(test, err)
     1705
     1706      Called when the test case *test* raises an unexpected exception. *err* is a
     1707      tuple of the form returned by :func:`sys.exc_info`: ``(type, value,
     1708      traceback)``.
     1709
     1710      The default implementation appends a tuple ``(test, formatted_err)`` to
     1711      the instance's :attr:`errors` attribute, where *formatted_err* is a
     1712      formatted traceback derived from *err*.
     1713
     1714
     1715   .. method:: addFailure(test, err)
     1716
     1717      Called when the test case *test* signals a failure. *err* is a tuple of
     1718      the form returned by :func:`sys.exc_info`: ``(type, value, traceback)``.
     1719
     1720      The default implementation appends a tuple ``(test, formatted_err)`` to
     1721      the instance's :attr:`failures` attribute, where *formatted_err* is a
     1722      formatted traceback derived from *err*.
     1723
     1724
     1725   .. method:: addSuccess(test)
     1726
     1727      Called when the test case *test* succeeds.
     1728
     1729      The default implementation does nothing.
     1730
     1731
     1732   .. method:: addSkip(test, reason)
     1733
     1734      Called when the test case *test* is skipped.  *reason* is the reason the
     1735      test gave for skipping.
     1736
     1737      The default implementation appends a tuple ``(test, reason)`` to the
     1738      instance's :attr:`skipped` attribute.
     1739
     1740
     1741   .. method:: addExpectedFailure(test, err)
     1742
     1743      Called when the test case *test* fails, but was marked with the
     1744      :func:`expectedFailure` decorator.
     1745
     1746      The default implementation appends a tuple ``(test, formatted_err)`` to
     1747      the instance's :attr:`expectedFailures` attribute, where *formatted_err*
     1748      is a formatted traceback derived from *err*.
     1749
     1750
     1751   .. method:: addUnexpectedSuccess(test)
     1752
     1753      Called when the test case *test* was marked with the
     1754      :func:`expectedFailure` decorator, but succeeded.
     1755
     1756      The default implementation appends the test to the instance's
     1757      :attr:`unexpectedSuccesses` attribute.
     1758
     1759.. class:: TextTestResult(stream, descriptions, verbosity)
     1760
     1761    A concrete implementation of :class:`TestResult` used by the
     1762    :class:`TextTestRunner`.
     1763
     1764    .. versionadded:: 2.7
     1765        This class was previously named ``_TextTestResult``. The old name still
     1766        exists as an alias but is deprecated.
    4821767
    4831768.. data:: defaultTestLoader
    … …  
    4881773
    4891774
    490 .. class:: TextTestRunner([stream[, descriptions[, verbosity]]])
     1775.. class:: TextTestRunner(stream=sys.stderr, descriptions=True, verbosity=1)
    4911776
    4921777   A basic test runner implementation which prints results on standard error.  It
    … …  
    4941779   applications which run test suites should provide alternate implementations.
    4951780
    496 
    497 .. function:: main([module[, defaultTest[, argv[, testRunner[, testLoader]]]]])
    498 
    499    A command-line program that runs a set of tests; this is primarily for making
    500    test modules conveniently executable.  The simplest use for this function is to
    501    include the following line at the end of a test script::
     1781   .. method:: _makeResult()
     1782
     1783      This method returns the instance of ``TestResult`` used by :meth:`run`.
     1784      It is not intended to be called directly, but can be overridden in
     1785      subclasses to provide a custom ``TestResult``.
     1786
     1787      ``_makeResult()`` instantiates the class or callable passed in the
     1788      ``TextTestRunner`` constructor as the ``resultclass`` argument. It
     1789      defaults to :class:`TextTestResult` if no ``resultclass`` is provided.
     1790      The result class is instantiated with the following arguments::
     1791
     1792            stream, descriptions, verbosity
     1793
     1794
     1795.. function:: main([module[, defaultTest[, argv[, testRunner[, testLoader[, exit[, verbosity[, failfast[, catchbreak[, buffer]]]]]]]]]])
     1796
     1797   A command-line program that loads a set of tests from *module* and runs them;
     1798   this is primarily for making test modules conveniently executable.
     1799   The simplest use for this function is to include the following line at the
     1800   end of a test script::
    5021801
    5031802      if __name__ == '__main__':
    5041803          unittest.main()
    5051804
     1805   You can run tests with more detailed information by passing in the verbosity
     1806   argument::
     1807
     1808      if __name__ == '__main__':
     1809          unittest.main(verbosity=2)
     1810
     1811   The *argv* argument can be a list of options passed to the program, with the
     1812   first element being the program name.  If not specified or ``None``,
     1813   the values of :data:`sys.argv` are used.
     1814
    5061815   The *testRunner* argument can either be a test runner class or an already
    507    created instance of it.
    508 
    509 In some cases, the existing tests may have been written using the :mod:`doctest`
    510 module.  If so, that module provides a  :class:`DocTestSuite` class that can
    511 automatically build :class:`unittest.TestSuite` instances from the existing
    512 :mod:`doctest`\ -based tests.
    513 
    514 .. versionadded:: 2.3
    515 
    516 
    517 .. _testcase-objects:
    518 
    519 TestCase Objects
    520 ----------------
    521 
    522 Each :class:`TestCase` instance represents a single test, but each concrete
    523 subclass may be used to define multiple tests --- the concrete class represents
    524 a single test fixture.  The fixture is created and cleaned up for each test
    525 case.
    526 
    527 :class:`TestCase` instances provide three groups of methods: one group used to
    528 run the test, another used by the test implementation to check conditions and
    529 report failures, and some inquiry methods allowing information about the test
    530 itself to be gathered.
    531 
    532 Methods in the first group (running the test) are:
    533 
    534 
    535 .. method:: TestCase.setUp()
    536 
    537    Method called to prepare the test fixture.  This is called immediately before
    538    calling the test method; any exception raised by this method will be considered
    539    an error rather than a test failure. The default implementation does nothing.
    540 
    541 
    542 .. method:: TestCase.tearDown()
    543 
    544    Method called immediately after the test method has been called and the result
    545    recorded.  This is called even if the test method raised an exception, so the
    546    implementation in subclasses may need to be particularly careful about checking
    547    internal state.  Any exception raised by this method will be considered an error
    548    rather than a test failure.  This method will only be called if the
    549    :meth:`setUp` succeeds, regardless of the outcome of the test method. The
    550    default implementation does nothing.
    551 
    552 
    553 .. method:: TestCase.run([result])
    554 
    555    Run the test, collecting the result into the test result object passed as
    556    *result*.  If *result* is omitted or :const:`None`, a temporary result object is
    557    created (by calling the :meth:`defaultTestCase` method) and used; this result
    558    object is not returned to :meth:`run`'s caller.
    559 
    560    The same effect may be had by simply calling the :class:`TestCase` instance.
    561 
    562 
    563 .. method:: TestCase.debug()
    564 
    565    Run the test without collecting the result.  This allows exceptions raised by
    566    the test to be propagated to the caller, and can be used to support running
    567    tests under a debugger.
    568 
    569 The test code can use any of the following methods to check for and report
    570 failures.
    571 
    572 
    573 .. method:: TestCase.assert_(expr[, msg])
    574             TestCase.failUnless(expr[, msg])
    575             TestCase.assertTrue(expr[, msg])
    576 
    577    Signal a test failure if *expr* is false; the explanation for the error will be
    578    *msg* if given, otherwise it will be :const:`None`.
    579 
    580 
    581 .. method:: TestCase.assertEqual(first, second[, msg])
    582             TestCase.failUnlessEqual(first, second[, msg])
    583 
    584    Test that *first* and *second* are equal.  If the values do not compare equal,
    585    the test will fail with the explanation given by *msg*, or :const:`None`.  Note
    586    that using :meth:`failUnlessEqual` improves upon doing the comparison as the
    587    first parameter to :meth:`failUnless`:  the default value for *msg* can be
    588    computed to include representations of both *first* and *second*.
    589 
    590 
    591 .. method:: TestCase.assertNotEqual(first, second[, msg])
    592             TestCase.failIfEqual(first, second[, msg])
    593 
    594    Test that *first* and *second* are not equal.  If the values do compare equal,
    595    the test will fail with the explanation given by *msg*, or :const:`None`.  Note
    596    that using :meth:`failIfEqual` improves upon doing the comparison as the first
    597    parameter to :meth:`failUnless` is that the default value for *msg* can be
    598    computed to include representations of both *first* and *second*.
    599 
    600 
    601 .. method:: TestCase.assertAlmostEqual(first, second[, places[, msg]])
    602             TestCase.failUnlessAlmostEqual(first, second[, places[, msg]])
    603 
    604    Test that *first* and *second* are approximately equal by computing the
    605    difference, rounding to the given number of decimal *places* (default 7),
    606    and comparing to zero.
    607    Note that comparing a given number of decimal places is not the same as
    608    comparing a given number of significant digits. If the values do not compare
    609    equal, the test will fail with the explanation given by *msg*, or :const:`None`.
    610 
    611 
    612 .. method:: TestCase.assertNotAlmostEqual(first, second[, places[, msg]])
    613             TestCase.failIfAlmostEqual(first, second[, places[, msg]])
    614 
    615    Test that *first* and *second* are not approximately equal by computing the
    616    difference, rounding to the given number of decimal *places* (default 7),
    617    and comparing to zero.
    618    Note that comparing a given number of decimal places is not the same as
    619    comparing a given number of significant digits. If the values do not compare
    620    equal, the test will fail with the explanation given by *msg*, or :const:`None`.
    621 
    622 
    623 .. method:: TestCase.assertRaises(exception, callable, ...)
    624             TestCase.failUnlessRaises(exception, callable, ...)
    625 
    626    Test that an exception is raised when *callable* is called with any positional
    627    or keyword arguments that are also passed to :meth:`assertRaises`.  The test
    628    passes if *exception* is raised, is an error if another exception is raised, or
    629    fails if no exception is raised.  To catch any of a group of exceptions, a tuple
    630    containing the exception classes may be passed as *exception*.
    631 
    632 
    633 .. method:: TestCase.failIf(expr[, msg])
    634             TestCase.assertFalse(expr[, msg])
    635 
    636    The inverse of the :meth:`failUnless` method is the :meth:`failIf` method.  This
    637    signals a test failure if *expr* is true, with *msg* or :const:`None` for the
    638    error message.
    639 
    640 
    641 .. method:: TestCase.fail([msg])
    642 
    643    Signals a test failure unconditionally, with *msg* or :const:`None` for the
    644    error message.
    645 
    646 
    647 .. attribute:: TestCase.failureException
    648 
    649    This class attribute gives the exception raised by the :meth:`test` method.  If
    650    a test framework needs to use a specialized exception, possibly to carry
    651    additional information, it must subclass this exception in order to "play fair"
    652    with the framework.  The initial value of this attribute is
    653    :exc:`AssertionError`.
    654 
    655 Testing frameworks can use the following methods to collect information on the
    656 test:
    657 
    658 
    659 .. method:: TestCase.countTestCases()
    660 
    661    Return the number of tests represented by this test object.  For
    662    :class:`TestCase` instances, this will always be ``1``.
    663 
    664 
    665 .. method:: TestCase.defaultTestResult()
    666 
    667    Return an instance of the test result class that should be used for this test
    668    case class (if no other result instance is provided to the :meth:`run` method).
    669 
    670    For :class:`TestCase` instances, this will always be an instance of
    671    :class:`TestResult`;  subclasses of :class:`TestCase` should override this as
    672    necessary.
    673 
    674 
    675 .. method:: TestCase.id()
    676 
    677    Return a string identifying the specific test case.  This is usually the full
    678    name of the test method, including the module and class name.
    679 
    680 
    681 .. method:: TestCase.shortDescription()
    682 
    683    Returns a one-line description of the test, or :const:`None` if no description
    684    has been provided.  The default implementation of this method returns the first
    685    line of the test method's docstring, if available, or :const:`None`.
    686 
    687 
    688 .. _testsuite-objects:
    689 
    690 TestSuite Objects
    691 -----------------
    692 
    693 :class:`TestSuite` objects behave much like :class:`TestCase` objects, except
    694 they do not actually implement a test.  Instead, they are used to aggregate
    695 tests into groups of tests that should be run together. Some additional methods
    696 are available to add tests to :class:`TestSuite` instances:
    697 
    698 
    699 .. method:: TestSuite.addTest(test)
    700 
    701    Add a :class:`TestCase` or :class:`TestSuite` to the suite.
    702 
    703 
    704 .. method:: TestSuite.addTests(tests)
    705 
    706    Add all the tests from an iterable of :class:`TestCase` and :class:`TestSuite`
    707    instances to this test suite.
    708 
    709    This is equivalent to iterating over *tests*, calling :meth:`addTest` for each
    710    element.
    711 
    712 :class:`TestSuite` shares the following methods with :class:`TestCase`:
    713 
    714 
    715 .. method:: TestSuite.run(result)
    716 
    717    Run the tests associated with this suite, collecting the result into the test
    718    result object passed as *result*.  Note that unlike :meth:`TestCase.run`,
    719    :meth:`TestSuite.run` requires the result object to be passed in.
    720 
    721 
    722 .. method:: TestSuite.debug()
    723 
    724    Run the tests associated with this suite without collecting the result. This
    725    allows exceptions raised by the test to be propagated to the caller and can be
    726    used to support running tests under a debugger.
    727 
    728 
    729 .. method:: TestSuite.countTestCases()
    730 
    731    Return the number of tests represented by this test object, including all
    732    individual tests and sub-suites.
    733 
    734 In the typical usage of a :class:`TestSuite` object, the :meth:`run` method is
    735 invoked by a :class:`TestRunner` rather than by the end-user test harness.
    736 
    737 
    738 .. _testresult-objects:
    739 
    740 TestResult Objects
    741 ------------------
    742 
    743 A :class:`TestResult` object stores the results of a set of tests.  The
    744 :class:`TestCase` and :class:`TestSuite` classes ensure that results are
    745 properly recorded; test authors do not need to worry about recording the outcome
    746 of tests.
    747 
    748 Testing frameworks built on top of :mod:`unittest` may want access to the
    749 :class:`TestResult` object generated by running a set of tests for reporting
    750 purposes; a :class:`TestResult` instance is returned by the
    751 :meth:`TestRunner.run` method for this purpose.
    752 
    753 :class:`TestResult` instances have the following attributes that will be of
    754 interest when inspecting the results of running a set of tests:
    755 
    756 
    757 .. attribute:: TestResult.errors
    758 
    759    A list containing 2-tuples of :class:`TestCase` instances and strings holding
    760    formatted tracebacks. Each tuple represents a test which raised an unexpected
    761    exception.
    762 
    763    .. versionchanged:: 2.2
    764       Contains formatted tracebacks instead of :func:`sys.exc_info` results.
    765 
    766 
    767 .. attribute:: TestResult.failures
    768 
    769    A list containing 2-tuples of :class:`TestCase` instances and strings holding
    770    formatted tracebacks. Each tuple represents a test where a failure was
    771    explicitly signalled using the :meth:`TestCase.fail\*` or
    772    :meth:`TestCase.assert\*` methods.
    773 
    774    .. versionchanged:: 2.2
    775       Contains formatted tracebacks instead of :func:`sys.exc_info` results.
    776 
    777 
    778 .. attribute:: TestResult.testsRun
    779 
    780    The total number of tests run so far.
    781 
    782 
    783 .. method:: TestResult.wasSuccessful()
    784 
    785    Returns :const:`True` if all tests run so far have passed, otherwise returns
    786    :const:`False`.
    787 
    788 
    789 .. method:: TestResult.stop()
    790 
    791    This method can be called to signal that the set of tests being run should be
    792    aborted by setting the :class:`TestResult`'s ``shouldStop`` attribute to
    793    :const:`True`.  :class:`TestRunner` objects should respect this flag and return
    794    without running any additional tests.
    795 
    796    For example, this feature is used by the :class:`TextTestRunner` class to stop
    797    the test framework when the user signals an interrupt from the keyboard.
    798    Interactive tools which provide :class:`TestRunner` implementations can use this
    799    in a similar manner.
    800 
    801 The following methods of the :class:`TestResult` class are used to maintain the
    802 internal data structures, and may be extended in subclasses to support
    803 additional reporting requirements.  This is particularly useful in building
    804 tools which support interactive reporting while tests are being run.
    805 
    806 
    807 .. method:: TestResult.startTest(test)
    808 
    809    Called when the test case *test* is about to be run.
    810 
    811    The default implementation simply increments the instance's ``testsRun``
    812    counter.
    813 
    814 
    815 .. method:: TestResult.stopTest(test)
    816 
    817    Called after the test case *test* has been executed, regardless of the outcome.
    818 
    819    The default implementation does nothing.
    820 
    821 
    822 .. method:: TestResult.addError(test, err)
    823 
    824    Called when the test case *test* raises an unexpected exception *err* is a tuple
    825    of the form returned by :func:`sys.exc_info`: ``(type, value, traceback)``.
    826 
    827    The default implementation appends a tuple ``(test, formatted_err)`` to the
    828    instance's ``errors`` attribute, where *formatted_err* is a formatted
    829    traceback derived from *err*.
    830 
    831 
    832 .. method:: TestResult.addFailure(test, err)
    833 
    834    Called when the test case *test* signals a failure. *err* is a tuple of the form
    835    returned by :func:`sys.exc_info`:  ``(type, value, traceback)``.
    836 
    837    The default implementation appends a tuple ``(test, formatted_err)`` to the
    838    instance's ``failures`` attribute, where *formatted_err* is a formatted
    839    traceback derived from *err*.
    840 
    841 
    842 .. method:: TestResult.addSuccess(test)
    843 
    844    Called when the test case *test* succeeds.
    845 
    846    The default implementation does nothing.
    847 
    848 
    849 .. _testloader-objects:
    850 
    851 TestLoader Objects
    852 ------------------
    853 
    854 The :class:`TestLoader` class is used to create test suites from classes and
    855 modules.  Normally, there is no need to create an instance of this class; the
    856 :mod:`unittest` module provides an instance that can be shared as
    857 ``unittest.defaultTestLoader``. Using a subclass or instance, however, allows
    858 customization of some configurable properties.
    859 
    860 :class:`TestLoader` objects have the following methods:
    861 
    862 
    863 .. method:: TestLoader.loadTestsFromTestCase(testCaseClass)
    864 
    865    Return a suite of all tests cases contained in the :class:`TestCase`\ -derived
    866    :class:`testCaseClass`.
    867 
    868 
    869 .. method:: TestLoader.loadTestsFromModule(module)
    870 
    871    Return a suite of all tests cases contained in the given module. This method
    872    searches *module* for classes derived from :class:`TestCase` and creates an
    873    instance of the class for each test method defined for the class.
    874 
    875    .. warning::
    876 
    877       While using a hierarchy of :class:`TestCase`\ -derived classes can be convenient
    878       in sharing fixtures and helper functions, defining test methods on base classes
    879       that are not intended to be instantiated directly does not play well with this
    880       method.  Doing so, however, can be useful when the fixtures are different and
    881       defined in subclasses.
    882 
    883 
    884 .. method:: TestLoader.loadTestsFromName(name[, module])
    885 
    886    Return a suite of all tests cases given a string specifier.
    887 
    888    The specifier *name* is a "dotted name" that may resolve either to a module, a
    889    test case class, a test method within a test case class, a :class:`TestSuite`
    890    instance, or a callable object which returns a :class:`TestCase` or
    891    :class:`TestSuite` instance.  These checks are applied in the order listed here;
    892    that is, a method on a possible test case class will be picked up as "a test
    893    method within a test case class", rather than "a callable object".
    894 
    895    For example, if you have a module :mod:`SampleTests` containing a
    896    :class:`TestCase`\ -derived class :class:`SampleTestCase` with three test
    897    methods (:meth:`test_one`, :meth:`test_two`, and :meth:`test_three`), the
    898    specifier ``'SampleTests.SampleTestCase'`` would cause this method to return a
    899    suite which will run all three test methods.  Using the specifier
    900    ``'SampleTests.SampleTestCase.test_two'`` would cause it to return a test suite
    901    which will run only the :meth:`test_two` test method.  The specifier can refer
    902    to modules and packages which have not been imported; they will be imported as a
    903    side-effect.
    904 
    905    The method optionally resolves *name* relative to the given *module*.
    906 
    907 
    908 .. method:: TestLoader.loadTestsFromNames(names[, module])
    909 
    910    Similar to :meth:`loadTestsFromName`, but takes a sequence of names rather than
    911    a single name.  The return value is a test suite which supports all the tests
    912    defined for each name.
    913 
    914 
    915 .. method:: TestLoader.getTestCaseNames(testCaseClass)
    916 
    917    Return a sorted sequence of method names found within *testCaseClass*; this
    918    should be a subclass of :class:`TestCase`.
    919 
    920 The following attributes of a :class:`TestLoader` can be configured either by
    921 subclassing or assignment on an instance:
    922 
    923 
    924 .. attribute:: TestLoader.testMethodPrefix
    925 
    926    String giving the prefix of method names which will be interpreted as test
    927    methods.  The default value is ``'test'``.
    928 
    929    This affects :meth:`getTestCaseNames` and all the :meth:`loadTestsFrom\*`
    930    methods.
    931 
    932 
    933 .. attribute:: TestLoader.sortTestMethodsUsing
    934 
    935    Function to be used to compare method names when sorting them in
    936    :meth:`getTestCaseNames` and all the :meth:`loadTestsFrom\*` methods. The
    937    default value is the built-in :func:`cmp` function; the attribute can also be
    938    set to :const:`None` to disable the sort.
    939 
    940 
    941 .. attribute:: TestLoader.suiteClass
    942 
    943    Callable object that constructs a test suite from a list of tests. No methods on
    944    the resulting object are needed.  The default value is the :class:`TestSuite`
    945    class.
    946 
    947    This affects all the :meth:`loadTestsFrom\*` methods.
    948 
     1816   created instance of it. By default ``main`` calls :func:`sys.exit` with
     1817   an exit code indicating success or failure of the tests run.
     1818
     1819   The *testLoader* argument has to be a :class:`TestLoader` instance,
     1820   and defaults to :data:`defaultTestLoader`.
     1821
     1822   ``main`` supports being used from the interactive interpreter by passing in the
     1823   argument ``exit=False``. This displays the result on standard output without
     1824   calling :func:`sys.exit`::
     1825
     1826      >>> from unittest import main
     1827      >>> main(module='test_module', exit=False)
     1828
     1829   The *failfast*, *catchbreak* and *buffer* parameters have the same
     1830   effect as the same-name `command-line options`_.
     1831
     1832   Calling ``main`` actually returns an instance of the ``TestProgram`` class.
     1833   This stores the result of the tests run as the ``result`` attribute.
     1834
     1835   .. versionchanged:: 2.7
     1836      The *exit*, *verbosity*, *failfast*, *catchbreak* and *buffer*
     1837      parameters were added.
     1838
     1839
     1840load_tests Protocol
     1841###################
     1842
     1843.. versionadded:: 2.7
     1844
     1845Modules or packages can customize how tests are loaded from them during normal
     1846test runs or test discovery by implementing a function called ``load_tests``.
     1847
     1848If a test module defines ``load_tests`` it will be called by
     1849:meth:`TestLoader.loadTestsFromModule` with the following arguments::
     1850
     1851    load_tests(loader, standard_tests, None)
     1852
     1853It should return a :class:`TestSuite`.
     1854
     1855*loader* is the instance of :class:`TestLoader` doing the loading.
     1856*standard_tests* are the tests that would be loaded by default from the
     1857module. It is common for test modules to only want to add or remove tests
     1858from the standard set of tests.
     1859The third argument is used when loading packages as part of test discovery.
     1860
     1861A typical ``load_tests`` function that loads tests from a specific set of
     1862:class:`TestCase` classes may look like::
     1863
     1864    test_cases = (TestCase1, TestCase2, TestCase3)
     1865
     1866    def load_tests(loader, tests, pattern):
     1867        suite = TestSuite()
     1868        for test_class in test_cases:
     1869            tests = loader.loadTestsFromTestCase(test_class)
     1870            suite.addTests(tests)
     1871        return suite
     1872
     1873If discovery is started, either from the command line or by calling
     1874:meth:`TestLoader.discover`, with a pattern that matches a package
     1875name then the package :file:`__init__.py` will be checked for ``load_tests``.
     1876
     1877.. note::
     1878
     1879   The default pattern is ``'test*.py'``. This matches all Python files
     1880   that start with ``'test'`` but *won't* match any test directories.
     1881
     1882   A pattern like ``'test*'`` will match test packages as well as
     1883   modules.
     1884
     1885If the package :file:`__init__.py` defines ``load_tests`` then it will be
     1886called and discovery not continued into the package. ``load_tests``
     1887is called with the following arguments::
     1888
     1889    load_tests(loader, standard_tests, pattern)
     1890
     1891This should return a :class:`TestSuite` representing all the tests
     1892from the package. (``standard_tests`` will only contain tests
     1893collected from :file:`__init__.py`.)
     1894
     1895Because the pattern is passed into ``load_tests`` the package is free to
     1896continue (and potentially modify) test discovery. A 'do nothing'
     1897``load_tests`` function for a test package would look like::
     1898
     1899    def load_tests(loader, standard_tests, pattern):
     1900        # top level directory cached on loader instance
     1901        this_dir = os.path.dirname(__file__)
     1902        package_tests = loader.discover(start_dir=this_dir, pattern=pattern)
     1903        standard_tests.addTests(package_tests)
     1904        return standard_tests
     1905
     1906
     1907Class and Module Fixtures
     1908-------------------------
     1909
     1910Class and module level fixtures are implemented in :class:`TestSuite`. When
     1911the test suite encounters a test from a new class then :meth:`tearDownClass`
     1912from the previous class (if there is one) is called, followed by
     1913:meth:`setUpClass` from the new class.
     1914
     1915Similarly if a test is from a different module from the previous test then
     1916``tearDownModule`` from the previous module is run, followed by
     1917``setUpModule`` from the new module.
     1918
     1919After all the tests have run the final ``tearDownClass`` and
     1920``tearDownModule`` are run.
     1921
     1922Note that shared fixtures do not play well with [potential] features like test
     1923parallelization and they break test isolation. They should be used with care.
     1924
     1925The default ordering of tests created by the unittest test loaders is to group
     1926all tests from the same modules and classes together. This will lead to
     1927``setUpClass`` / ``setUpModule`` (etc) being called exactly once per class and
     1928module. If you randomize the order, so that tests from different modules and
     1929classes are adjacent to each other, then these shared fixture functions may be
     1930called multiple times in a single test run.
     1931
     1932Shared fixtures are not intended to work with suites with non-standard
     1933ordering. A ``BaseTestSuite`` still exists for frameworks that don't want to
     1934support shared fixtures.
     1935
     1936If there are any exceptions raised during one of the shared fixture functions
     1937the test is reported as an error. Because there is no corresponding test
     1938instance an ``_ErrorHolder`` object (that has the same interface as a
     1939:class:`TestCase`) is created to represent the error. If you are just using
     1940the standard unittest test runner then this detail doesn't matter, but if you
     1941are a framework author it may be relevant.
     1942
     1943
     1944setUpClass and tearDownClass
     1945~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     1946
     1947These must be implemented as class methods::
     1948
     1949    import unittest
     1950
     1951    class Test(unittest.TestCase):
     1952        @classmethod
     1953        def setUpClass(cls):
     1954            cls._connection = createExpensiveConnectionObject()
     1955
     1956        @classmethod
     1957        def tearDownClass(cls):
     1958            cls._connection.destroy()
     1959
     1960If you want the ``setUpClass`` and ``tearDownClass`` on base classes called
     1961then you must call up to them yourself. The implementations in
     1962:class:`TestCase` are empty.
     1963
     1964If an exception is raised during a ``setUpClass`` then the tests in the class
     1965are not run and the ``tearDownClass`` is not run. Skipped classes will not
     1966have ``setUpClass`` or ``tearDownClass`` run. If the exception is a
     1967:exc:`SkipTest` exception then the class will be reported as having been skipped
     1968instead of as an error.
     1969
     1970
     1971setUpModule and tearDownModule
     1972~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     1973
     1974These should be implemented as functions::
     1975
     1976    def setUpModule():
     1977        createConnection()
     1978
     1979    def tearDownModule():
     1980        closeConnection()
     1981
     1982If an exception is raised in a ``setUpModule`` then none of the tests in the
     1983module will be run and the ``tearDownModule`` will not be run. If the exception is a
     1984:exc:`SkipTest` exception then the module will be reported as having been skipped
     1985instead of as an error.
     1986
     1987
     1988Signal Handling
     1989---------------
     1990
     1991The :option:`-c/--catch <unittest -c>` command-line option to unittest,
     1992along with the ``catchbreak`` parameter to :func:`unittest.main()`, provide
     1993more friendly handling of control-C during a test run. With catch break
     1994behavior enabled control-C will allow the currently running test to complete,
     1995and the test run will then end and report all the results so far. A second
     1996control-c will raise a :exc:`KeyboardInterrupt` in the usual way.
     1997
     1998The control-c handling signal handler attempts to remain compatible with code or
     1999tests that install their own :const:`signal.SIGINT` handler. If the ``unittest``
     2000handler is called but *isn't* the installed :const:`signal.SIGINT` handler,
     2001i.e. it has been replaced by the system under test and delegated to, then it
     2002calls the default handler. This will normally be the expected behavior by code
     2003that replaces an installed handler and delegates to it. For individual tests
     2004that need ``unittest`` control-c handling disabled the :func:`removeHandler`
     2005decorator can be used.
     2006
     2007There are a few utility functions for framework authors to enable control-c
     2008handling functionality within test frameworks.
     2009
     2010.. function:: installHandler()
     2011
     2012   Install the control-c handler. When a :const:`signal.SIGINT` is received
     2013   (usually in response to the user pressing control-c) all registered results
     2014   have :meth:`~TestResult.stop` called.
     2015
     2016   .. versionadded:: 2.7
     2017
     2018.. function:: registerResult(result)
     2019
     2020   Register a :class:`TestResult` object for control-c handling. Registering a
     2021   result stores a weak reference to it, so it doesn't prevent the result from
     2022   being garbage collected.
     2023
     2024   Registering a :class:`TestResult` object has no side-effects if control-c
     2025   handling is not enabled, so test frameworks can unconditionally register
     2026   all results they create independently of whether or not handling is enabled.
     2027
     2028   .. versionadded:: 2.7
     2029
     2030.. function:: removeResult(result)
     2031
     2032   Remove a registered result. Once a result has been removed then
     2033   :meth:`~TestResult.stop` will no longer be called on that result object in
     2034   response to a control-c.
     2035
     2036   .. versionadded:: 2.7
     2037
     2038.. function:: removeHandler(function=None)
     2039
     2040   When called without arguments this function removes the control-c handler
     2041   if it has been installed. This function can also be used as a test decorator
     2042   to temporarily remove the handler whilst the test is being executed::
     2043
     2044      @unittest.removeHandler
     2045      def test_signal_handling(self):
     2046          ...
     2047
     2048   .. versionadded:: 2.7
     2049
Note: See TracChangeset for help on using the changeset viewer.