Getting Started With Heartbeat Version 2

Rather than duplicate vast slabs of the GettingStarted page for the 1.x series of Heartbeat, this page will attempt to cover the differences.

Background

The Heartbeat program has been around for a while. It has a great many strengths, and yet there were a few weaknesses in version 1 that needed to be addressed:

• limitation on two nodes for cluster size

• inability to monitor resources for their correct operation

• minimal ability to express dependency information

This release removes these limitations while preserving as many of its strengths as possible.

The latest and greatest versions of Heartbeat can be obtained here.

Release Notes (What has Changed)

* Version 2.0.0 - First stable release of the next generation of Heartbeat

• Core infrastructure improvements:
  • Messaging (message acknowledging and flow control)
  • Logging (asynchronous logging daemon)
• Release 1.x style (2-node) clusters fully supported
• Multi-node support (so far up to 16-node clusters tested)
• New components:

  • - ClusterInformationBase	(replicated resource configuration)
    - ClusterResourceManager	(supporting 1->N nodes)
    - Modular PolicyEngine	(controlling resource placement)
    - LocalResourceManager	(policy free, not cluster aware)
    - StonithDaemon	(stand-alone fencing subsystem)

• Support for OCF and LSB resource agents

• Support for complex resource types (groups, clones)

• Support for a rich set of resource location, colocation and ordering constraints

• Conversion tool for existing haresources

• Resources monitored by request

• Resource maintenance mode

• Several failback, failure and "No Quorum" behaviours to choose from

  • (global defaults and per action or resource)

• Sample cluster status and configuration monitoring tools

While the core infrastructure improvements and the new STONITH API (and of course bug fixes) may provide motivation for upgrading to the 2.0.x series, the main focus of this release is the richer cluster manager and ability to utilize more than 2 nodes.

What Hasn't Changed from Version 1

GettingStarted sections:

• Getting Started

• Installing Heartbeat

• Configuring ha.cf (however this section has been superseded by this page: ha.cf)

• Selecting an Interface

• Configuring Authkeys

• Starting and testing Heartbeat

• Appendix A - Ethernet Crossover Cable Construction

What's New

Logging

Asynchronous Logging
One drawback of using the syslog API directly is that the calls are blocking. This is generally not desirable and becomes a major problem as the volume of logging increases. The HeartbeatLogDaemon (new in version 2) seeks to remove this problem by being a logging broker which will make the synchronous API call on behalf of its clients.

See ha.cf/UseLogdDirective for enabling and disabling this feature.

Choice of Logging Package
It is recommended that syslog-ng be used for logging in preference to syslog. Given the capabilities of Heartbeat version 2 and the increased cluster sizes, one can expect an increased volume of logging to occur. It has been the experience of the development team that syslog loses log messages more frequently than syslog-ng.

A Sample Syslog-NG Configuration
The following is the extra configuration I use for Syslog-ng on SLES9. I add this to the end of /etc/syslog-ng/syslog-ng.conf on every node in the cluster.

source s_tcp { tcp(port(9999) max-connections(16)); };
filter f_ha  { facility(local7); };
filter f_ha_tcp  { facility(local7); };
destination ha_local { file("/var/log/cluster.log" perm(0644)); };
destination ha_tcp { tcp(c001n16 port(9999));};
log { source(src); filter(f_ha_tcp); destination(ha_tcp); };
log { source(src); source(s_tcp); filter(f_ha); destination(ha_local); };

In English, this sends everything logged to the local7 facility to the machine c001n16. In addition, it will send everything logged to the local7 facility to /var/log/cluster.log. In the case of the machine c001n16, this will include everything sent from the other cluster nodes.
NOTE: src refers to logs that originate locally on your machine. The name used in your distro may vary.

Core Files

When all else is lost, parts of Heartbeat may produce core files. These provide extremely valuable information to the developers and increase their ability to resolve any bugs that are encountered.

Administrators are highly encouraged to make use of the new coredumps directive and read GettingCoreDumps to ensure this vital information is not lost.

Enabling the new ClusterResourceManager

This directive is the only mandatory change when looking to use the new ClusterResourceManager.

crm yes

As this hard-reboots your machine if you use a broken cib.xml configuration, you might want to use

crm respawn

for testing purposes instead. This will just respawn the necessary processes like earlier versions did.

See ha.cf/CRMDirective for more information.

Enabling Stonith

stonithd is automatically started if you say "crm yes|respawn". See ha.cf/CRMDirective.

Additional Options

• apiauth cibmon   uid=hacluster
  respawn hacluster /usr/lib/heartbeat/cibmon -d

Resource Configuration

A somewhat comprehensive user guide is available. Please feel free to suggest improvements.

Sample Cluster Configurations

• A Basic Single IP Address Configuration (newbies start here)

• A Two IP address Active/Active Configuration

• A Basic Apache Web Server Configuration

• Two Apache Web Servers in an Active/Active Configuration

Other simple examples are also available.

Reporting Problems

There are a number of things you will likely be asked for when reporting a problem, so please be sure to save them.

• Core files : These will be stored in directories under /var/lib/heartbeat/cores

• Log files : Hopefully you are running syslog-ng or similar... Please supply a combined cluster log (I recommend at least local7.info and higher).

If the problem relates to the calculated response (resource starts/stops/restarts etc) to a given cluster state, your chances of it being fixed or explained are much higher if you can include a CIB which exhibits the behaviour. In this task, ptest is your friend. Invoke as ptest -X path/to/cib.file with multiple -V increasing the verbosity of the output.

Please see ClusterResourceManager/BugReports for more details on submitting bug reports.

Related Links

ContactUs, DownloadSoftware, ClusterInformationBase/UserGuide, ClusterInformationBase/SimpleExamples, Heartbeat version 2 information, Alternate Getting Started document for version 2

Linux-HA site built by the Linux-HA crowd. HTML4 / CSS2