XTS(7)

NAME
XTS - The X Test Suite

DESCRIPTION

This is a revamped version of X Test Suite (XTS) which removes some of the ugliness of building and running the tests.

DOWNLOADING

Currently xts is only available from a git repository:

    $ git clone git://anongit.freedesktop.org/git/xorg/test/xts

Eventually there will be a tarball release when it is stabilized.

REQUIREMENTS

Required libraries: libX11, libXau, libXext, libXi, libXtst

Required utilities: xdpyinfo, xset, perl, bdftopcf, mkfontdir

If you are using packages outside of the standard system directories, make sure to set PATH and PKG_CONFIG_PATH appropriately. See pkg-config(1) for details on the latter.

BUILD AND INSTALL

The build and install steps are now straightforward:

    $ ./autogen.sh  # using a git checkout
    $ ./configure   # using a tarball
    $ make
    $ make install

It is not necessary to install the suite. Tests may be run from the build directories.

RUNNING THE TESTS

The tests can be run from the build directory or from the installed package. The check.sh script sets up the environment to run tests from the build tree.

    $ ./check.sh    # run all the tests
    $ ./check.sh XI # run just the XI test scenario

The "make run-tests" framework can also be used from the build tree.

    $ make run-tests                # run all the tests
    $ make run-tests scenario=XI    # run just the XI test scenario
    $ make test-XI                  # shortcut for the previous

A wrapper script, xts-run, is the easiest way to run the installed tests.

    $ xts-run       # run all the tests
    $ xts-run XI    # run just the XI test scenario

By default, the test output will go to xts-results/<date> in the current directory. This can be changed with the xts-run -o parameter. The output contains the full journal of all test output plus individual logs for each test program. Journal files from completed runs can be analyzed by the xts-report tool (formerly vswrpt):

    $ xts-report -f /xts/output/dir/journal

RUNNING A SINGLE TEST

The pt script can be used to run a single set of tests or a single test case only. For example, to run only the XGrabButton tests, run $ pt ./XGrabButton

The logs for the test are saved in a file <testcase>.log, for the example above the log file would be ./XGrabButton.log.

The pt script sets up some environment variables, based on TET_ROOT and other settings. To start the debugger automatically after the environment is set up, run: $ pt -gdb ./XGrabButton

To run only a single test-case inside one test set, use the -i flag. $ pt -i 10 ./XGrabButton

This command only runs test case 10 (t010() in the source).

BUGS

Some tests provide different results when run as a set versus run one-by-one with the pt script. This has been observed with Xlib13/XGrabButton where the modifier state is not reset correctly. When debugging tests, it is recommended that failing tests are run separately to verify the result.

XTS was only recently resurrected after a period of neglect. Tests have not been verified. At this point, XTS results should only be used to notice differences between server versions. A failing test may not necessarily indicate a bug in the server and a successful test may not necessarily indicate correct functioning of the server.