The basic outline of how to proceed in configuring and setting up ServeRAID hardware and software is as follows:
This page explains how to setup the hardware for a Linux-HA cluster using IBM's ServeRAID cards in clustering mode.
The basic components you need are:
2 ServeRAID cards (the same model) that support ServeRAID clustering.
Make sure you also read the documentation that comes with your equipment. Especially the ServeRAID and EXP users and installation guides.
The servers need to at least have a PCI slot free for the ServeRAID adapter. Different ServeRAID cards have different PCI specs with regards to speed and voltage. So make sure your server can handle them. But as always the faster the slot the better.
For details on which cards are supported in clustering mode by Linux-HA see the ServeRAID page for more details.
The EXP cabinet you need depends on the speed you would like. The EXP200 supports Ultra SCSI (80 MB/s), EXP300 supports Ultra160 SCSI and the EXP400 supports Ultra320 SCSI. The EXP300 cabinet automatically supports two SCSI interfaces like are needed for this configruation, but for the EXP400 cabinet, you'll have to add an option not included in the base EXP400 cabinet.
It is best to use disks at the same speed as the EXP cabinet you are using. Another important note here is that the slowest disk will determine the total speed of the SCSI channel. Meaning if you have 5 Ultra320 disks and 1 Ultra SCSI disk, then everything will run at Ultra SCSI speeds, and you won't be using the everything at its maximum speed. Moreover, IBM only tests and certifies these drives and cabinets in matched sets. So, if you want IBM to support your system, make sure you make all the pieces match!
Also the SCSI cabling is important. Make sure you use correct cables, connectors, and terminators as specified in the documentation that comes with your equipment. Bad cabling and bad termination can lead to strange, hard to debug problems.
The order in which these steps is very important. Failure to do things in the right order will lead to great confusion and frustration, and may require starting the hardware configuration over.
At this point, you can do is to install the ServeRAID cards in the servers, Provide power to the EXP cabinet and put the disks in it, and cable things up according to the documentation which comes with the hardware. Once your hardware is installed, proceed to the next section.
Read your hardware documentation for all the details, but for our purposes, what you're headed towards is to have the three components configured with paul on one end of the SCSI bus, the EXP cabinet in the middle, and silas other end of the SCSI bus.
This means that the SCSI bus should have terminators connected in paul and silas, but not anywhere inside the EXP cabinet.
The following information was adapted from material found on the IBM web site.
Like our other examples, one server is named paul, and the other silas.
Read the good background information on the CD contents from the IBM web site
Boot Server paul from the IBM ServeRAID Management CD in the CD-ROM Drive. Make sure Server silas is powered off.
Upgrade the firmware on paul's ServeRAID controller. The ServeRAID Support CD will automatically check the controller firmware and prompt the user to upgrade if it is found to be downlevel. For this reason, you should always grab the latest version of the ServeRAID Support CD from the IBM web site before you begin.
If the ServeRAID controller is not already configured, the user will be prompt with Express Configuration. Click Cancel.
Confirm Window will appear. Click Yes to confirm. When complete, a message will appear at the bottom of the screen.
Select either Express Configuration or Custom Configuration for controller.
Select Next.
If Express Configuration was selected, the software will automatically create the arrays or hot spares.
If Custom Configuration was selected, the user has the option to create the arrays.
To create Raid level 1, two or more drives are needed. ServeRAID firmware does not reliably support any RAID5-based configuration in cluster mode.
Click Next to continue.
Configuration Summary screen will appear. Click Apply to confirm.
Select Configure for Clustering.
Enter the name of first controller under Controller Host ID. It is best to name it the same as the local Linux host name (paul).
Enter the name of second controller under Cluster Partner Host ID. It is best to name it after the partner's Linux host name (silas).
Click on shared checkbox for the logical drives that will be shared between the two ServeRAID controllers. Assign a merge group ID to the shared drives. Each shared logical drive must have a unique merge ID. Make note of the merge group ID for use in configuring the ServeRAID heartbeat resource agent.
Exit and Shut down Server paul.
Boot Server B from the IBM ServeRAID Management CD in the CD-ROM Drive. Make sure Server paul is powered off.
Upgrade the firmware on silas's ServeRAID controller. The ServeRAID Support CD will automatically check the controller firmware and prompt the user to upgrade if it is found to be downlevel. For this reason, you should always grab the latest version of the ServeRAID Support CD from the IBM web site before you begin.
If the ServeRAID controller is not already configured, the user will be prompted with Express Configuration. Click Cancel.
Select Restore to Factory Defaults.
Click Yes to confirm.
Select Configure for Clustering.
Enter the name of second controller (silas) under Controller Host ID.
Enter the name of the first controller (paul) under partner Controller Host ID.
Exit and Reboot Server silas and bring up paul.
Heartbeat requires that the ipssend utility be copied from the CD and installed in a directory which is in root's path(as created by init(8)). We recommend /sbin. To do this, execute the following commands on both machines.
mount /media/cdrom cd /media/cdrom/WHATEVER-THE-RIGHT-DIRECTORY-IS cp ipssend /sbin cd / umount /media/cdrom
It is highly improbable that your kernel includes the same version of the driver as you have on your ServeRAID cards. You should have earlier upgraded your firmware to the same level as is on the ServeRAID utility CD.
mount /media/cdrom cd /media/cdrom/WHATEVER-THE-RIGHT-DIRECTORY-IS cp ips.c ips.h /usr/src/linux/drivers/scsi cd / umount /media/cdrom
Then, follow your distribution's directions for building and installing an updated kernel module.
Note that you should keep a copy of this source file somewhere other than in the kernel tree, since later kernel updates may overwrite these files.
When you make later kernel updates, you will have to rebuild this module from the sources you have saved from the CD. The driver version in your kernel and the installed firmware version on your cards must always match.
The ServeRAID ResourceAgent operates and configures shared ServeRAID partitions. This HeartbeatResourceAgent requires that you use ServeRAID software version 6.10 or later. Like all ResourceAgents, the ServeRAID resource agent is configured by putting an appropriate entry into the haresources file, as illustrated below.
The ServeRAID resource agent makes a given ServeRAID merge group available to the system as a ServeRAIDLogicalVolume.
The ServeRAID resource agent takes two parameters in the haresources file. These parameters are:
adapter - the number of the ServeRAID adapter. We number ServeRAID adapters on a system starting from one (on Red Hat / Fedora systems, not sure on other systems -- DavidBrown).
Shared Merge Group number The shared merge group number was chosen when the adapter was installed. Shared merge group numbers are between 1 and 8 inclusive.
paul ServeRAID::1::1