This document is still a draft.
Quorum server is called Tiebreaker Server also. And the program of Quorum Sever is named as quorumd
The code of quorumd, revised ccm layer and the new quorumd plugin have been pushed to hg.linux-ha.org.
Any testing or comment on this issue are welcome!
The concept of quorum server is described in Tiebreaker Server and Split Site.
The design of quorum server is decribed in Design and Detail Design
Attention: quorumd is deemed broken and complex to deploy correctly and will therefore not be shipped in heartbeat version 3.
Current quorumd is in the heartbeat package. So we have to install the whole heartbeat package on the computer on which we want the quorum server running. After installing the heartbeat package, quorumd should be found under /usr/lib/heartbeat/.
Please create a file named quorumd.conf under /etc/ha.d/
The content should be like:
cluster mycluster version 2_0_8 interval 1000 timeout 5000 takeover 3000 giveup 2000 nodenum 3 weight 300 cluster yourcluster ...
All these directives are required. Here is the meanings of the directives. (all time unit is ms)
directive |
comment |
cluster |
the name of the cluster which wants to connect to this quorum server |
version |
the version of the protocol between the quorum server and its clients (2_0_8 is the only version supported now) |
interval |
the interval between the clients renewing their status |
timeout |
without renew, how long will the quorum server waits before declaring a client dead |
takeover |
how long does the client takeover the resources to other node (in the case of DC changed) |
giveup |
how long does the client give up all resources it hold (in the case of lost quorum) |
nodenum* |
the total number of nodes in the cluster |
weight* |
the total weight of nodes in the cluster |
*The current code calculates the quorum based on the comparison of the weight of the partitions. So we don't need "nodenum" and "weight" of cluster.They are there for that we may implement other algorithm later.
more about "takeover" and "giveup"
Let's image that we have a cluster which has splited to two partitions, A and B. and let's say that node a is the leader node of A and node b is the leader node of B. Both a and b connect to the quorum server. The quorum server tells node a that A has quorum and tells node b that B hasn't quorum.
Now something happens in the partition A, the leader will change from node a to node a'. So node a has to disconnect from the quorum server, but we know node a' will connect to the quorum server soon. The "takeover" is the time that the leader takeovers from a to a'.
If a new node adds to B so the weight of B is larger than the weight of A, the quorum will transfer to B. However, after we tell A that you don't have quorum anymore, we need wait some time to let A "giveup" all the resoures A is holding. "giveup" is the time to "giveup" all resources when a partition lost quorum.
The next step is to put the x.509 certificates used by quorumd to /etc/ha.d/. There should be four certs. The last section of this guide shows how to create these certs.
ca-cert.pem |
the cert of the ca root, used to verify client's cert |
ca-crl.pem |
the revoked cert list of the ca root |
server-key.pem |
the private key of the quorum server |
server-cert.pem |
the cert of the quorum server, signed by the ca |
The administator of the quorum server should create a ca root key and cert and quorum server key and cert. The server cert should be signed by the ca root key. And the administrator should issue the certs of clients. Please notice that the CN of the client certs must be the name of the cluster.
To enable connect to a quorum server, we must indicate the cluster name and quorum server in the /etc/ha.d/ha.cf. The value of quorum_server is the name of quorum server which can be resolved.
cluster mycluster quorum_server plinuxt11
The default weight of node is 100. We can change it by hb_setweight command.
There should be three certificates under the /etc/ha.d/ They are:
ca-cert.pem |
the cert of the ca root, used to verify server's cert |
client-key.pem |
the private key of the client |
client-cert.pem |
the cert of the client, signed by the ca |
These three certs should be issued by the administrator of the quorum server.
To let heartbeat connect to quorumd, we have to set the env variable HA_quorum before we start the heartbeat.
export HA_quorum=quorumd /usr/lib/heartbeat
Above setting will let heartbeat use the quorumd plugin which will connect to the quorum server.
We can use the certtool provided by gnutls to create the certs. The online guide of certtool is Here
$ certtool --generate-privkey --outfile ca-key.pem
$ certtool --generate-self-signed --load-privkey ca-key.pem --outfile ca-cert.pem
$ certtool --generate-privkey --outfile server-key.pem
$ certtool --generate-request --load-privkey server-key.pem --outfile server-request.pem $ certtool --generate-certificate --load-request server-request.pem --outfile server-cert.pem --load-ca-certificate ca-cert.pem --load-ca-privkey ca-key.pem
$ certtool --generate-privkey --outfile client-key.pem
$ certtool --generate-request --load-privkey client-key.pem --outfile client-request.pem $ certtool --generate-certificate --load-request client-request.pem --outfile client-cert.pem --load-ca-certificate ca-cert.pem --load-ca-privkey ca-key.pem