| 1 | ---------------------
|
|---|
| 2 | subunit release notes
|
|---|
| 3 | ---------------------
|
|---|
| 4 |
|
|---|
| 5 | NEXT (In development)
|
|---|
| 6 | ---------------------
|
|---|
| 7 |
|
|---|
| 8 | The Subunit Python test runner ``python -m subunit.run`` can now report the
|
|---|
| 9 | test ids and also filter via a test id list file thanks to improvements in
|
|---|
| 10 | ``testtools.run``. See the testtools manual, or testrepository - a major
|
|---|
| 11 | user of such functionality.
|
|---|
| 12 |
|
|---|
| 13 | IMPROVEMENTS
|
|---|
| 14 | ~~~~~~~~~~~~
|
|---|
| 15 |
|
|---|
| 16 | * The ``subunit.run`` Python module supports ``-l`` and ``--load-list`` as
|
|---|
| 17 | per ``testtools.run``. This required a dependency bump due to a small
|
|---|
| 18 | API change in ``testtools``. (Robert Collins)
|
|---|
| 19 |
|
|---|
| 20 |
|
|---|
| 21 | 0.0.6
|
|---|
| 22 | -----
|
|---|
| 23 |
|
|---|
| 24 | This release of subunit fixes a number of unicode related bugs. This depends on
|
|---|
| 25 | testtools 0.9.4 and will not function without it. Thanks to Tres Seaver there
|
|---|
| 26 | is also an optional native setup.py file for use with easy_install and the
|
|---|
| 27 | like.
|
|---|
| 28 |
|
|---|
| 29 | BUG FIXES
|
|---|
| 30 | ~~~~~~~~~
|
|---|
| 31 |
|
|---|
| 32 | * Be consistent about delivering unicode content to testtools StringException
|
|---|
| 33 | class which has become (appropriately) conservative. (Robert Collins)
|
|---|
| 34 |
|
|---|
| 35 | * Fix incorrect reference to subunit_test_failf in c/README.
|
|---|
| 36 | (Brad Hards, #524341)
|
|---|
| 37 |
|
|---|
| 38 | * Fix incorrect ordering of tags method parameters in TestResultDecorator. This
|
|---|
| 39 | is purely cosmetic as the parameters are passed down with no interpretation.
|
|---|
| 40 | (Robert Collins, #537611)
|
|---|
| 41 |
|
|---|
| 42 | * Old style tracebacks with no encoding info are now treated as UTF8 rather
|
|---|
| 43 | than some-random-codec-like-ascii. (Robert Collins)
|
|---|
| 44 |
|
|---|
| 45 | * On windows, ProtocolTestCase and TestProtocolClient will set their streams to
|
|---|
| 46 | binary mode by calling into msvcrt; this avoids having their input or output
|
|---|
| 47 | mangled by the default line ending translation on that platform.
|
|---|
| 48 | (Robert Collins, Martin [gz], #579296)
|
|---|
| 49 |
|
|---|
| 50 | IMPROVEMENTS
|
|---|
| 51 | ~~~~~~~~~~~~
|
|---|
| 52 |
|
|---|
| 53 | * Subunit now has a setup.py for python deployments that are not using
|
|---|
| 54 | distribution packages. (Tres Seaver, #538181)
|
|---|
| 55 |
|
|---|
| 56 | * Subunit now supports test discovery by building on the testtools support for
|
|---|
| 57 | it. You can take advantage of it with "python -m subunit.run discover [path]"
|
|---|
| 58 | and see "python -m subunit.run discover --help" for more options.
|
|---|
| 59 |
|
|---|
| 60 | * Subunit now uses the improved unicode support in testtools when outputting
|
|---|
| 61 | non-details based test information; this should consistently UTF8 encode such
|
|---|
| 62 | strings.
|
|---|
| 63 |
|
|---|
| 64 | * The Python TestProtocolClient now flushes output on startTest and stopTest.
|
|---|
| 65 | (Martin [gz]).
|
|---|
| 66 |
|
|---|
| 67 |
|
|---|
| 68 | 0.0.5
|
|---|
| 69 | -----
|
|---|
| 70 |
|
|---|
| 71 | BUG FIXES
|
|---|
| 72 | ~~~~~~~~~
|
|---|
| 73 |
|
|---|
| 74 | * make check was failing if subunit wasn't installed due to a missing include
|
|---|
| 75 | path for the test program test_child.
|
|---|
| 76 |
|
|---|
| 77 | * make distcheck was failing due to a missing $(top_srcdir) rune.
|
|---|
| 78 |
|
|---|
| 79 | IMPROVEMENTS
|
|---|
| 80 | ~~~~~~~~~~~~
|
|---|
| 81 |
|
|---|
| 82 | * New filter `subunit-notify` that will show a notification window with test
|
|---|
| 83 | statistics when the test run finishes.
|
|---|
| 84 |
|
|---|
| 85 | * subunit.run will now pipe its output to the command in the
|
|---|
| 86 | SUBUNIT_FORMATTER environment variable, if set.
|
|---|
| 87 |
|
|---|
| 88 | 0.0.4
|
|---|
| 89 | -----
|
|---|
| 90 |
|
|---|
| 91 | BUG FIXES
|
|---|
| 92 | ~~~~~~~~~
|
|---|
| 93 |
|
|---|
| 94 | * subunit2junitxml -f required a value, this is now fixed and -f acts as a
|
|---|
| 95 | boolean switch with no parameter.
|
|---|
| 96 |
|
|---|
| 97 | * Building with autoconf 2.65 is now supported.
|
|---|
| 98 |
|
|---|
| 99 |
|
|---|
| 100 | 0.0.3
|
|---|
| 101 | -----
|
|---|
| 102 |
|
|---|
| 103 | CHANGES:
|
|---|
| 104 |
|
|---|
| 105 | * License change, by unanimous agreement of contributors to BSD/Apache
|
|---|
| 106 | License Version 2.0. This makes Subunit compatible with more testing
|
|---|
| 107 | frameworks.
|
|---|
| 108 |
|
|---|
| 109 | IMPROVEMENTS:
|
|---|
| 110 |
|
|---|
| 111 | * CPPUnit is now directly supported: subunit builds a cppunit listener
|
|---|
| 112 | ``libcppunit-subunit``.
|
|---|
| 113 |
|
|---|
| 114 | * In the python API ``addExpectedFailure`` and ``addUnexpectedSuccess``
|
|---|
| 115 | from python 2.7/3.1 are now supported. ``addExpectedFailure`` is
|
|---|
| 116 | serialised as ``xfail``, and ``addUnexpectedSuccess`` as ``success``.
|
|---|
| 117 | The ``ProtocolTestCase`` parser now calls outcomes using an extended
|
|---|
| 118 | API that permits attaching arbitrary MIME resources such as text files
|
|---|
| 119 | log entries and so on. This extended API is being developed with the
|
|---|
| 120 | Python testing community, and is in flux. ``TestResult`` objects that
|
|---|
| 121 | do not support the API will be detected and transparently downgraded
|
|---|
| 122 | back to the regular Python unittest API.
|
|---|
| 123 |
|
|---|
| 124 | * INSTALLDIRS can be set to control the perl MakeMaker 'INSTALLDIRS'
|
|---|
| 125 | viarable when installing.
|
|---|
| 126 |
|
|---|
| 127 | * Multipart test outcomes are tentatively supported; the exact protocol
|
|---|
| 128 | for them, both serialiser and object is not yet finalised. Testers and
|
|---|
| 129 | early adopters are sought. As part of this and also in an attempt to
|
|---|
| 130 | provider a more precise focus on the wire protocol and toolchain,
|
|---|
| 131 | Subunit now depends on testtools (http://launchpad.net/testtools)
|
|---|
| 132 | release 0.9.0 or newer.
|
|---|
| 133 |
|
|---|
| 134 | * subunit2junitxml supports a new option, --forward which causes it
|
|---|
| 135 | to forward the raw subunit stream in a similar manner to tee. This
|
|---|
| 136 | is used with the -o option to both write a xml report and get some
|
|---|
| 137 | other subunit filter to process the stream.
|
|---|
| 138 |
|
|---|
| 139 | * The C library now has ``subunit_test_skip``.
|
|---|
| 140 |
|
|---|
| 141 | BUG FIXES:
|
|---|
| 142 |
|
|---|
| 143 | * Install progress_model.py correctly.
|
|---|
| 144 |
|
|---|
| 145 | * Non-gcc builds will no longer try to use gcc specific flags.
|
|---|
| 146 | (Thanks trondn-norbye)
|
|---|
| 147 |
|
|---|
| 148 | API CHANGES:
|
|---|
| 149 |
|
|---|
| 150 | INTERNALS:
|
|---|
| 151 |
|
|---|
| 152 | 0.0.2
|
|---|
| 153 | -----
|
|---|
| 154 |
|
|---|
| 155 | CHANGES:
|
|---|
| 156 |
|
|---|
| 157 | IMPROVEMENTS:
|
|---|
| 158 |
|
|---|
| 159 | * A number of filters now support ``--no-passthrough`` to cause all
|
|---|
| 160 | non-subunit content to be discarded. This is useful when precise control
|
|---|
| 161 | over what is output is required - such as with subunit2junitxml.
|
|---|
| 162 |
|
|---|
| 163 | * A small perl parser is now included, and a new ``subunit-diff`` tool
|
|---|
| 164 | using that is included. (Jelmer Vernooij)
|
|---|
| 165 |
|
|---|
| 166 | * Subunit streams can now include optional, incremental lookahead
|
|---|
| 167 | information about progress. This allows reporters to make estimates
|
|---|
| 168 | about completion, when such information is available. See the README
|
|---|
| 169 | under ``progress`` for more details.
|
|---|
| 170 |
|
|---|
| 171 | * ``subunit-filter`` now supports regex filtering via ``--with`` and
|
|---|
| 172 | ``without`` options. (Martin Pool)
|
|---|
| 173 |
|
|---|
| 174 | * ``subunit2gtk`` has been added, a filter that shows a GTK summary of a
|
|---|
| 175 | test stream.
|
|---|
| 176 |
|
|---|
| 177 | * ``subunit2pyunit`` has a --progress flag which will cause the bzrlib
|
|---|
| 178 | test reporter to be used, which has a textual progress bar. This requires
|
|---|
| 179 | a recent bzrlib as a minor bugfix was required in bzrlib to support this.
|
|---|
| 180 |
|
|---|
| 181 | * ``subunit2junitxml`` has been added. This filter converts a subunit
|
|---|
| 182 | stream to a single JUnit style XML stream using the pyjunitxml
|
|---|
| 183 | python library.
|
|---|
| 184 |
|
|---|
| 185 | * The shell functions support skipping via ``subunit_skip_test`` now.
|
|---|
| 186 |
|
|---|
| 187 | BUG FIXES:
|
|---|
| 188 |
|
|---|
| 189 | * ``xfail`` outcomes are now passed to python TestResult's via
|
|---|
| 190 | addExpectedFailure if it is present on the TestResult. Python 2.6 and
|
|---|
| 191 | earlier which do not have this function will have ``xfail`` outcomes
|
|---|
| 192 | passed through as success outcomes as earlier versions of subunit did.
|
|---|
| 193 |
|
|---|
| 194 | API CHANGES:
|
|---|
| 195 |
|
|---|
| 196 | * tags are no longer passed around in python via the ``TestCase.tags``
|
|---|
| 197 | attribute. Instead ``TestResult.tags(new_tags, gone_tags)`` is called,
|
|---|
| 198 | and like in the protocol, if called while a test is active only applies
|
|---|
| 199 | to that test. (Robert Collins)
|
|---|
| 200 |
|
|---|
| 201 | * ``TestResultFilter`` takes a new optional constructor parameter
|
|---|
| 202 | ``filter_predicate``. (Martin Pool)
|
|---|
| 203 |
|
|---|
| 204 | * When a progress: directive is encountered in a subunit stream, the
|
|---|
| 205 | python bindings now call the ``progress(offset, whence)`` method on
|
|---|
| 206 | ``TestResult``.
|
|---|
| 207 |
|
|---|
| 208 | * When a time: directive is encountered in a subunit stream, the python
|
|---|
| 209 | bindings now call the ``time(seconds)`` method on ``TestResult``.
|
|---|
| 210 |
|
|---|
| 211 | INTERNALS:
|
|---|
| 212 |
|
|---|
| 213 | * (python) Added ``subunit.test_results.AutoTimingTestResultDecorator``. Most
|
|---|
| 214 | users of subunit will want to wrap their ``TestProtocolClient`` objects
|
|---|
| 215 | in this decorator to get test timing data for performance analysis.
|
|---|
| 216 |
|
|---|
| 217 | * (python) ExecTestCase supports passing arguments to test scripts.
|
|---|
| 218 |
|
|---|
| 219 | * (python) New helper ``subunit.test_results.HookedTestResultDecorator``
|
|---|
| 220 | which can be used to call some code on every event, without having to
|
|---|
| 221 | implement all the event methods.
|
|---|
| 222 |
|
|---|
| 223 | * (python) ``TestProtocolClient.time(a_datetime)`` has been added which
|
|---|
| 224 | causes a timestamp to be output to the stream.
|
|---|
