This page documents the heartbeat (aka linux-ha) test plan.
Initial Author: AlanRobertson
Subscribers must include Stephanie Glass. Distribution of new versions is automatic via Wiki change tracking system, including facilities for comparing any two versions.
Maintained automatically by Wiki change tracking system.
The test plan consists running and evaluating the results from of a set of automated tests which are shipped with the Linux-HA software, plus a set of manually performed GUI tests, as docmented in section VII.
There are two suites of tests which ship with Linux-HA:
Each of these tests is simple to run, and simple to evaluate. In addition, BasicSanityCheck does not require any special setup to make it work correctly and runs very quickly (in approximately a minute). cts, on the other hand requires a significant amount of setup before running it, and normally runs for many hours.
We do not yet have a set of automated GUI tests, so they are (for the time being) manual, and described in section 7.
BasicSanityCheck can run on any single Linux machine without any special preparation or architectural constraints.
cts, on the other hand, requires three machines to run it on. The tests are controlled from the test monitor machine, and they are run on the other two machines. The Linux-HA package must be installed (in a consistent version) on all three machines. Detailed information about setting up cts is supplied by the cts/README file.
It is highly desirable that the clocks on all the test machines be closely synchronized in order to track down any errors easily.
cts executes in the context of a particular configuration, but does not create different configurations. As a result, it is often desirable to run it once with each of several possible configurations. Most often, the main variant used in the configurations is the value of the auto_failback directive. If two or more configurations are to be considered, one should have auto_failback on and one with it off.
The BasicSanityCheck software verifies that many of the major paths in heartbeat andsome associated programs are exercised. This is useful to find problems where the linux-HA software might not be running for reason of some significant platform-related bug.
The cts package is a suite of random tests which eventually run the system through many possible configurations and thoroughly test a configuration with a random sequence of tests chosen from its battery of tests.
In order to do any testing, the heartbeat, heartbeat-pils, and heartbeat-stonith packages must all be installed with the version which is to be tested. For the cts tests this means on all three machines. For the BasicSanityCheck test, this means only the single machine to be tested.
Software configuration for the cts tests is complex and is described in detail in the cts README file. This README file can be found at /usr/lib/heartbeat/cts/README. The latest greatest version of the README (which might not match the version you're running) can be found in CVS.
The GUI is a program which can be run on any Linux system with the proper set of libraries installed. It is X-Windows based, so that tester must have access to an X-Windows server to view the results on.
The CTS tests create significant system stress.
Both tests perform a version of regression testing. The GUI regression tests are described in section VII.
All tests should pass with zero errors when properly configured.
Run the tests. If any cts tests fail when a reliable message transport is configured for syslog as described in the README, then this is a bug. If it is configured with UDP transport for syslog, it is possible that certain messages might get lost in a busy networking environment. Then the logs on the original machines can be examined to see if the missing message was actually issued but is missing. This is very difficult unless the system clocks are closely synchronized.
Each CTS test case is described in the file CTStests.py. Each test is a class definition similar to this one:
The tests which are currently in the test suite are described in the CTS page.
This tests DRBD to see if it is maintaining proper integrity of the disk data. This requires that you configure DRBD into your configuration. If you do not, then it will not be run.
We are developing a set of testcases for manually tests of GUI. These testcases should cover most functions of GUI. GuiTest