diff options
Diffstat (limited to 'selftest/README')
-rw-r--r-- | selftest/README | 119 |
1 files changed, 119 insertions, 0 deletions
diff --git a/selftest/README b/selftest/README new file mode 100644 index 0000000000..f8be20a569 --- /dev/null +++ b/selftest/README @@ -0,0 +1,119 @@ +# vim: ft=rst + +This directory contains test scripts that are useful for running a +bunch of tests all at once. + +Available testsuites +==================== +The available testsuites are obtained from a script, usually +selftest/samba{3,4}_tests.sh. This script should for each testsuite output +the name of the test, the command to run and the environment that should be +provided. Use the included "plantest" function to generate the required output. + +Testsuite behaviour +================================ + +Exit code +------------ +The testsuites should exit with a non-zero exit code if at least one +test failed. Skipped tests should not influence the exit code. + +Output format +------------- +Testsuites can simply use the exit code to indicate whether all of their +tests have succeeded or one or more have failed. It is also possible to +provide more granular information using the Subunit protocol. + +This protocol works by writing simple messages to standard output. Any +messages that can not be interpreted by this protocol are considered comments +for the last announced test. + +Accepted commands are: + +test +~~~~~~~~~~~~ +test: <NAME> + +Announce that a new test with the specified name is starting + +success +~~~~~~~~~~~~~~~ +success: <NAME> + +Announce that the test with the specified name is done and ran successfully. + +failure +~~~~~~~~~~~~~~~ +failure: <NAME> +failure: <NAME> [ REASON ] + +Announce that the test with the specified name failed. Optionally, it is +possible to specify a reason it failed. + +skip +~~~~~~~~~~~~ +skip: <NAME> +skip: <NAME> [ REASON ] + +Announce that the test with the specified name was skipped. Optionally a +reason can be specified. + +knownfail +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +knownfail: <NAME> +knownfail: <NAME> [ REASON ] + +Announce that the test with the specified name was run and failed as expected. +Alternatively it is also possible to simply return "failure:" here but +specify in the samba4-knownfailures file that it is failing. + +Environments +============ +Tests often need to run against a server with particular things set up, +a "environment". This environment is provided by the test "target": Samba 3, +Samba 4 or Windows. + +The following environments are currently available: + + - none: No server set up, no variables set. + - dc: Domain controller set up. The following environment variables will + be set: + + * USERNAME: Administrator user name + * PASSWORD: Administrator password + * DOMAIN: Domain name + * REALM: Realm name + * SERVER: DC host name + * SERVER_IP: DC IPv4 address + * NETBIOSNAME: DC NetBIOS name + * NETIOSALIAS: DC NetBIOS alias + + - member: Domain controller and member server that is joined to it set up. The + following environment variables will be set: + + * USERNAME: Domain administrator user name + * PASSWORD: Domain administrator password + * DOMAIN: Domain name + * REALM: Realm name + * SERVER: Name of the member server + + +Running tests +============= + +To run all the tests use:: + + make test + +To run a quick subset (aiming for about 1 minute of testing) run:: + + make quicktest + +To run a specific test, use this syntax:: + + make test TESTS=testname + +for example:: + + make test TESTS=samba4.BASE-DELETE + |