| 1 | # vim: ft=rst
|
|---|
| 2 |
|
|---|
| 3 | This directory contains test scripts that are useful for running a
|
|---|
| 4 | bunch of tests all at once.
|
|---|
| 5 |
|
|---|
| 6 | There are two parts to this:
|
|---|
| 7 |
|
|---|
| 8 | * The test runner (selftest/selftest.pl)
|
|---|
| 9 | * The test formatter
|
|---|
| 10 |
|
|---|
| 11 | selftest.pl simply outputs subunit, which can then be formatted or analyzed
|
|---|
| 12 | by tools that understand the subunit protocol. One of these tools is
|
|---|
| 13 | format-subunit, which is used by default as part of "make test".
|
|---|
| 14 |
|
|---|
| 15 | Available testsuites
|
|---|
| 16 | ====================
|
|---|
| 17 | The available testsuites are obtained from a script, usually
|
|---|
| 18 | source{3,4}/selftest/tests.sh. This script should for each testsuite output
|
|---|
| 19 | the name of the test, the command to run and the environment that should be
|
|---|
| 20 | provided. Use the included "plantest" function to generate the required output.
|
|---|
| 21 |
|
|---|
| 22 | Testsuite behaviour
|
|---|
| 23 | ===================
|
|---|
| 24 |
|
|---|
| 25 | Exit code
|
|---|
| 26 | ------------
|
|---|
| 27 | The testsuites should exit with a non-zero exit code if at least one
|
|---|
| 28 | test failed. Skipped tests should not influence the exit code.
|
|---|
| 29 |
|
|---|
| 30 | Output format
|
|---|
| 31 | -------------
|
|---|
| 32 | Testsuites can simply use the exit code to indicate whether all of their
|
|---|
| 33 | tests have succeeded or one or more have failed. It is also possible to
|
|---|
| 34 | provide more granular information using the Subunit protocol.
|
|---|
| 35 |
|
|---|
| 36 | This protocol works by writing simple messages to standard output. Any
|
|---|
| 37 | messages that can not be interpreted by this protocol are considered comments
|
|---|
| 38 | for the last announced test.
|
|---|
| 39 |
|
|---|
| 40 | For a full description of the subunit protocol, see ../lib/subunit/README.
|
|---|
| 41 |
|
|---|
| 42 | The following commands are Samba extensions to Subunit:
|
|---|
| 43 |
|
|---|
| 44 | testsuite-count
|
|---|
| 45 | ~~~~~~~~~~~~~~~
|
|---|
| 46 | testsuite-count: number
|
|---|
| 47 |
|
|---|
| 48 | Announce the number of tests that is going to be run.
|
|---|
| 49 |
|
|---|
| 50 | start-testsuite
|
|---|
| 51 | ~~~~~~~~~~~~~~~
|
|---|
| 52 | start-testsuite: name
|
|---|
| 53 |
|
|---|
| 54 | The testsuite name is used as prefix for all containing tests.
|
|---|
| 55 |
|
|---|
| 56 | skip-testsuite
|
|---|
| 57 | ~~~~~~~~~~~~~~
|
|---|
| 58 | skip-testsuite: name
|
|---|
| 59 |
|
|---|
| 60 | Mark the testsuite with the specified name as skipped.
|
|---|
| 61 |
|
|---|
| 62 | testsuite-success
|
|---|
| 63 | ~~~~~~~~~~~~~~~~~
|
|---|
| 64 | testsuite-success: name
|
|---|
| 65 |
|
|---|
| 66 | Indicate that the testsuite has succeeded successfully.
|
|---|
| 67 |
|
|---|
| 68 | testsuite-fail
|
|---|
| 69 | ~~~~~~~~~~~~~~
|
|---|
| 70 | testsuite-fail: name
|
|---|
| 71 |
|
|---|
| 72 | Indicate that a testsuite has failed.
|
|---|
| 73 |
|
|---|
| 74 | Environments
|
|---|
| 75 | ============
|
|---|
| 76 | Tests often need to run against a server with particular things set up,
|
|---|
| 77 | a "environment". This environment is provided by the test "target": Samba 3,
|
|---|
| 78 | Samba 4 or Windows.
|
|---|
| 79 |
|
|---|
| 80 | The following environments are currently available:
|
|---|
| 81 |
|
|---|
| 82 | - none: No server set up, no variables set.
|
|---|
| 83 | - dc: Domain controller set up. The following environment variables will
|
|---|
| 84 | be set:
|
|---|
| 85 |
|
|---|
| 86 | * USERNAME: Administrator user name
|
|---|
| 87 | * PASSWORD: Administrator password
|
|---|
| 88 | * DOMAIN: Domain name
|
|---|
| 89 | * REALM: Realm name
|
|---|
| 90 | * SERVER: DC host name
|
|---|
| 91 | * SERVER_IP: DC IPv4 address
|
|---|
| 92 | * NETBIOSNAME: DC NetBIOS name
|
|---|
| 93 | * NETIOSALIAS: DC NetBIOS alias
|
|---|
| 94 |
|
|---|
| 95 | - member: Domain controller and member server that is joined to it set up. The
|
|---|
| 96 | following environment variables will be set:
|
|---|
| 97 |
|
|---|
| 98 | * USERNAME: Domain administrator user name
|
|---|
| 99 | * PASSWORD: Domain administrator password
|
|---|
| 100 | * DOMAIN: Domain name
|
|---|
| 101 | * REALM: Realm name
|
|---|
| 102 | * SERVER: Name of the member server
|
|---|
| 103 |
|
|---|
| 104 |
|
|---|
| 105 | Running tests
|
|---|
| 106 | =============
|
|---|
| 107 |
|
|---|
| 108 | To run all the tests use::
|
|---|
| 109 |
|
|---|
| 110 | make test
|
|---|
| 111 |
|
|---|
| 112 | To run a quick subset (aiming for about 1 minute of testing) run::
|
|---|
| 113 |
|
|---|
| 114 | make quicktest
|
|---|
| 115 |
|
|---|
| 116 | To run a specific test, use this syntax::
|
|---|
| 117 |
|
|---|
| 118 | make test TESTS=testname
|
|---|
| 119 |
|
|---|
| 120 | for example::
|
|---|
| 121 |
|
|---|
| 122 | make test TESTS=samba4.BASE-DELETE
|
|---|
| 123 |
|
|---|
