CRM Successor

• Pacemaker

• Documentation

Heartbeat 3.0

• Documentation

• Download

• Pages linked to below may be out of date or right out obsolete.
  See Site News.

Introduction

• Fact Sheet (v1)

• Press Room

• Success Stories

• Talks

• Technical Overview

• Technical Papers

Using Linux-HA

• Download Software

• Learning Heartbeat

• Tutorials

• FAQ

Links

• Related Technologies

• Project Friends

• Partner Projects

• Support

• Commercial Software

• Recent Changes

Common Searches

• ha.cf / GuiGuide

• DRBD / pingd

• STONITH / NFS

• auto_failback

• haresources / softdog

• ServeRAID / DTD

Last site update:
2021-01-20 07:52:56

Setting up the hardware for ServeRAID failover

This page explains how to setup the hardware for a Linux-HA cluster using IBM's ServeRAID cards in clustering mode.

The basic components you need are:

• 2 servers

• 2 ServeRAID cards (the same model) that support ServeRAID clustering.

• 1 or more EXP disk cabinet(s) to hold the shared storage
• 2 or more disk(s) to go into the EXP cabinet(s)
• The necessary cables to hook everything up

Note

Make sure you also read the documentation that comes with your equipment. Especially the ServeRAID and EXP users and installation guides.

Minimum requirements

The servers need to at least have a PCI slot free for the ServeRAID adapter. Different ServeRAID cards have different PCI specs with regards to speed and voltage. So make sure your server can handle them. But as always the faster the slot the better.

For details on which cards are supported in clustering mode by Linux-HA see the ServeRAID page for more details.

The EXP cabinet you need depends on the speed you would like. The EXP200 supports Ultra SCSI (80 MB/s), EXP300 supports Ultra160 SCSI and the EXP400 supports Ultra320 SCSI. The EXP300 cabinet automatically supports two SCSI interfaces like are needed for this configruation, but for the EXP400 cabinet, you'll have to add an option not included in the base EXP400 cabinet.

It is best to use disks at the same speed as the EXP cabinet you are using. Another important note here is that the slowest disk will determine the total speed of the SCSI channel. Meaning if you have 5 Ultra320 disks and 1 Ultra SCSI disk, then everything will run at Ultra SCSI speeds, and you won't be using the everything at its maximum speed. Moreover, IBM only tests and certifies these drives and cabinets in matched sets. So, if you want IBM to support your system, make sure you make all the pieces match!

Also the SCSI cabling is important. Make sure you use correct cables, connectors, and terminators as specified in the documentation that comes with your equipment. Bad cabling and bad termination can lead to strange, hard to debug problems.

Connect Your Hardware

The order in which these steps is very important. Failure to do things in the right order will lead to great confusion and frustration, and may require starting the hardware configuration over.

At this point, you can do is to install the ServeRAID cards in the servers, Provide power to the EXP cabinet and put the disks in it, and cable things up according to the documentation which comes with the hardware. Once your hardware is installed, proceed to the next section.

Read your hardware documentation for all the details, but for our purposes, what you're headed towards is to have the three components configured with paul on one end of the SCSI bus, the EXP cabinet in the middle, and silas other end of the SCSI bus.

This means that the SCSI bus should have terminators connected in paul and silas, but not anywhere inside the EXP cabinet.

Configuring IBM ServeRAID for Clustering

The following information was adapted from material found on the IBM web site.

Like our other examples, one server is named paul, and the other silas.

Create Utilities CD

1. Download the Linux version of the

   IBM ServeRAID Management CD

2. Burn a CD from the downloaded image. This CD can be booted, or mounted from Linux. We will use it both ways.

3. Read the good background information on the CD contents from the IBM web site

Updating and Configuring Server paul

1. Boot Server paul from the IBM ServeRAID Management CD in the CD-ROM Drive. Make sure Server silas is powered off.

2. Upgrade the firmware on paul's ServeRAID controller. The ServeRAID Support CD will automatically check the controller firmware and prompt the user to upgrade if it is found to be downlevel. For this reason, you should always grab the latest version of the ServeRAID Support CD from the IBM web site before you begin.

3. Once the ServeRAID Support CD is satisfied that your controller(s) firmware is current, configuration will begin.

4. If the ServeRAID controller is not already configured, the user will be prompt with Express Configuration. Click Cancel.

5. Select the appropriate ServeRAID controller.
6. Using the right mouse button, click Controller.
7. Select Restore to Factory Default Settings.

8. Confirm Window will appear. Click Yes to confirm. When complete, a message will appear at the bottom of the screen.

9. Using the right mouse button, click the same Controller.
10. Select Create Arrays to create an array.

11. Select either Express Configuration or Custom Configuration for controller.

12. Select Next.

13. If Express Configuration was selected, the software will automatically create the arrays or hot spares.

14. If Custom Configuration was selected, the user has the option to create the arrays.

    1. To create Raid level 1, two or more drives are needed. ServeRAID firmware does not reliably support any RAID5-based configuration in cluster mode.

    2. All ready physical drives will appear on the left side of the windows. Double click on the ready drives to add them to an array.
    3. To create Hot-Spare, select Hot-Spare on the right side of Window and Double click on any ready drives on the left side of the window.

    4. Click Next to continue.

    5. Select the Raid level for the logical drive from the pull-down menu.
    6. Click Next.

    7. Configuration Summary screen will appear. Click Apply to confirm.

    8. Click Apply to accept and save the configuration.
15. To create more arrays, repeat the steps above.
16. After creating all the arrays needed, Right-click on the same Controller.

17. Select Configure for Clustering.

18. Enter the name of first controller under Controller Host ID. It is best to name it the same as the local Linux host name (paul).

19. Enter the name of second controller under Cluster Partner Host ID. It is best to name it after the partner's Linux host name (silas).

20. Be sure to make a note of the names and the SCSI Channel Initiator IDs.

21. Click on shared checkbox for the logical drives that will be shared between the two ServeRAID controllers. Assign a merge group ID to the shared drives. Each shared logical drive must have a unique merge ID. Make note of the merge group ID for use in configuring the ServeRAID heartbeat resource agent.

22. Click OK.

23. Exit and Shut down Server paul.

Updating and Configuring Server silas

1. Boot Server B from the IBM ServeRAID Management CD in the CD-ROM Drive. Make sure Server paul is powered off.

2. Upgrade the firmware on silas's ServeRAID controller. The ServeRAID Support CD will automatically check the controller firmware and prompt the user to upgrade if it is found to be downlevel. For this reason, you should always grab the latest version of the ServeRAID Support CD from the IBM web site before you begin.

3. Once the ServeRAID Support CD is satisfied that your controller(s) firmware is current, configuration will begin.

4. If the ServeRAID controller is not already configured, the user will be prompted with Express Configuration. Click Cancel.

5. Select the appropriate ServeRAID controller.
6. Right-click on the appropriate controller.

7. Select Restore to Factory Defaults.

8. Click Yes to confirm.

9. Using the right mouse key, click on the appropriate controller.

10. Select Configure for Clustering.

11. Enter the name of second controller (silas) under Controller Host ID.

12. Enter the name of the first controller (paul) under partner Controller Host ID.

13. Be sure to change the SCSI Channel Initiator Ids, and make them different from Server A ServeRAID SCSI Channel Initiator Ids on the corresponding physical channels.

14. Exit and Reboot Server silas and bring up paul.

ServeRAID Software Configuration

Install ipssend utility

Heartbeat requires that the ipssend utility be copied from the CD and installed in a directory which is in root's path(as created by init(8)). We recommend /sbin. To do this, execute the following commands on both machines.

  mount /media/cdrom
  cd /media/cdrom/WHATEVER-THE-RIGHT-DIRECTORY-IS
  cp ipssend /sbin
  cd /
  umount /media/cdrom

Install ServeRAID (ips) driver

It is highly improbable that your kernel includes the same version of the driver as you have on your ServeRAID cards. You should have earlier upgraded your firmware to the same level as is on the ServeRAID utility CD.

  mount /media/cdrom
  cd /media/cdrom/WHATEVER-THE-RIGHT-DIRECTORY-IS
  cp ips.c ips.h /usr/src/linux/drivers/scsi
  cd /
  umount /media/cdrom

Then, follow your distribution's directions for building and installing an updated kernel module.

Note that you should keep a copy of this source file somewhere other than in the kernel tree, since later kernel updates may overwrite these files.

When you make later kernel updates, you will have to rebuild this module from the sources you have saved from the CD. The driver version in your kernel and the installed firmware version on your cards must always match.

Heartbeat ServeRAID Resource Agent

The ServeRAID ResourceAgent operates and configures shared ServeRAID partitions. This HeartbeatResourceAgent requires that you use ServeRAID software version 6.10 or later. Like all ResourceAgents, the ServeRAID resource agent is configured by putting an appropriate entry into the haresources file, as illustrated below.

The ServeRAID resource agent makes a given ServeRAID merge group available to the system as a ServeRAIDLogicalVolume.

The ServeRAID resource agent takes two parameters in the haresources file. These parameters are:

• adapter - the number of the ServeRAID adapter. We number ServeRAID adapters on a system starting from one (on Red Hat / Fedora systems, not sure on other systems -- DavidBrown).

• Shared Merge Group number The shared merge group number was chosen when the adapter was installed. Shared merge group numbers are between 1 and 8 inclusive.

Sample haresources resource line

paul ServeRAID::1::1