With this step, the cluster temporarily relinquishes control of its resources. This means that the cluster no longer monitors its nodes or resources for the duration of the upgrade, and will not rectify and application or node failures during this time. Currently running resources, however, will continue to run.

# crm_attribute -t crm_config -n is_managed_default -v false

In most configurations, individual resources do not set the is_managed attribute individually, and hence the cluster-wide attribute is_managed_default applies to all of them.

If in your specific configuration you do have resources that have this attribute set, you should remove it to make sure the default applies:

# crm_resource -t primitive -r <resource-id> -p is_managed -D

At this point, is is important to save a copy of the Cluster Information Base (CIB). The CIB to save is stored in a file named cib.xml, normally located in /var/lib/heartbeat/crm.

# cp /var/lib/heartbeat/crm/cib.xml ~

You need to perform this step on only one node currently connected to the cluster. Do not delete this file, it will be restored later.

You may now stop Heartbeat with /etc/init.d/heartbeat stop or the preferred command to stop a system service on your distribution (service heartbeat stop, rcheartbeat stop, etc.).

In case you are running a legacy version of Heartbeat affected by a shutdown bug, then graceful crmd shutdown will not work properly in unmanaged mode.

In this case, after you have initiated a graceful service shutdown with the above command, kill the crmd process manually:

	Warning
	Before proceeding with this section, verify that you have created a backup copy of the CIB on one of your cluster nodes, as outlined in Section 6.2.2, “Backing up the CIB”.

You should now wipe local CRM-related files from your node. To do so, remove all files from the directory where the CRM stores CIB information, commonly /var/lib/heartbeat/crm.

# rm /var/lib/heartbeat/crm/*

	Note
	You should only proceed with this step if Heartbeat is still stopped on all cluster nodes, and all cluster nodes have had their CIB contents wiped. If you still have remaining nodes that have a residual CIB configuration, proceed as outlined in Section 6.2.4, “Wiping files related to the CRM”.

Restoring the CIB means copying the CIB backup described in Section 6.2.2, “Backing up the CIB” to the /var/lib/heartbeat/crm directory.

# cp ~/cib.xml /var/lib/heartbeat/crm/cib.xml
# chown hacluster:haclient /var/lib/heartbeat/crm/cib.xml
# chmod 0600 /var/lib/heartbeat/crm/cib.xml

You must perform this step on one node only, namely the first node on which you are about to upgrade the cluster software. On all other nodes, the /var/lib/heartbeat/crm directory must remain empty — Pacemaker distributes the CIB automatically.

While upgrading, it is important to recall that the monolithic Heartbeat 2.1 tree has been split up into modular parts. Thus you will replace Heartbeat with three individual pieces of software: Cluster Glue, Pacemaker, and Heartbeat 3 which comprises just the cluster messaging layer.

If this is the last node to be upgraded in your cluster, and your package management system did not restart Heartbeat services after the software upgrade, you should now proceed to Section 6.2.7, “Restarting Heartbeat services”. Otherwise, you should move to the next node and proceed as outlined in Section 6.1.1, “Stopping Heartbeat services” Section 6.2.6, “Upgrading software”.

	Note
	This step may be omitted in case your package management system automatically restarts Heartbeat services during post-install.

First, restart Heartbeat on the node where you restored the CIB (see Section 6.2.5, “Restoring the CIB”) with /etc/init.d/heartbeat start. Then, repeat this command on the remaining cluster nodes. At this time,

Once the cluster software has been upgraded, it is recommended to return the cluster into managed mode:

# crm_attribute -t crm_config -n is_managed_default -v true

Although an upgraded cluster can theoretically operate on the pre-upgrade CIB schema indefinitely, it is strongly recommended to upgrade the CIB to the current schema. To do so, run the following command after cluster communications between all nodes have been re-established:

# cibadmin --upgrade --force