kea/tests/lettuce

BIND10 system testing with Lettuce
or: to BDD or not to BDD

In this directory, we define a set of behavioral tests for BIND 10. Currently,
these tests are specific for BIND10, but we are keeping in mind that RFC-related
tests could be separated, so that we can test other systems as well.

Prerequisites:
- Installed version of BIND 10 (but see below how to run it from source tree)
- dig
- lettuce (http://lettuce.it)

To install lettuce, if you have the python pip installation tool, simply do
pip install lettuce
See http://lettuce.it/intro/install.html

Most systems have the pip tool in a separate package; on Debian-based systems
it is called python-pip. On FreeBSD the port is devel/py-pip.

Running the tests
-----------------

At this moment, we have a fixed port for local tests in our setups, port 47806.
This port must be free. (TODO: can we make this run-time discovered?).
Port 47805 is used for cmdctl, and must also be available.
(note, we will need to extend this to a range, or if possible, we will need to
do some on-the-fly available port finding)

The bind10 main script, bindctl script, and dig must all be in the default
search path of your environment, and BIND 10 must not be running if you use
the installed version when you run the tests.

To run the tests, just run 'lettuce' in this directory.

We have provided a script that sets up the shell environment to run the tests
with the build tree version of bind. If your shell uses export to set
environment variables, you can source the script setup_intree_bind10.sh, then
run lettuce.

Due to the default way lettuce prints its output, it is advisable to run it
in a terminal that is wide than the default. If you see a lot of lines twice
in different colors, the terminal is not wide enough.

If you just want to run one specific feature test, use
lettuce features/

To run a specific scenario from a feature, use
lettuce features/ -s

If any scenario fails, the output from long-running processes will be stored
in the output directory. The name of the files will be
--.stdout and
--.stderr
Where spaces and other non-standard characters are replaced by an underscore.
The process name is either the standard name for said process (e.g. 'bind10'),
or the name given to it by the test ('when i run bind10 as ').

These files *will* be overwritten or deleted if the same scenarios are run
again, so if you want to inspect them after a failed test, either do so
immediately or move the files.

Extending tests
---------------

[TODO]