dm-multipath and the ds4700
September 2nd, 2009 Posted in LinuxFor around a year now, I’ve been wanting to move away from IBM’s (okay, LSI’s) rdac mpp drivers used for the ds4XXX series disk chassis on RHEL and CentOS. When RHEL 5.3 came out boasting support dm-multipath support for the DS4XXX series, I was understandably overjoyed. The only problem was I couldn’t make it work. After several rounds of cursing, muttering and poking folks smarter than me to help out, the problem became immediately clear.
The example configs which work fine for the ’supported’ platforms have a text string mismatch when using the ‘unsupported’ ds4700. Basically you have to change a bit of text slightly because the hardware identifies itself slightly differently. Who’d have thunk it, right?
Below is the multipath.conf snippet I’ve been using now for the past month with some pretty good success.
defaults {
user_friendly_names yes
}
blacklist {
devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
devnode "^(hd|xvd|vd)[a-z]*"
wwid "*"
}
blacklist_exceptions {
wwid "3600XXXX"
}
devices {
device {
vendor "IBM"
product "1814 FAStT"
getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
prio_callout "/sbin/mpath_prio_rdac /dev/%n"
features "0"
hardware_handler "1 rdac"
path_grouping_policy group_by_prio
failback immediate
rr_weight uniform
no_path_retry queue
rr_min_io 1000
path_checker rdac
}
Now clearly you’ll have to modify the wwid to suit your own environment, and you’ll also want to exclude the non-multipath device references (/dev/sdb for example) from lvm.conf if you’re using lvm. This got things going for me, so hopefully it’ll help others out as well.
6 Responses to “dm-multipath and the ds4700”
By John McNulty on Sep 2, 2009
Don’t IBM provide the correct details for you? I’m really surprised. HP provide an FC enablement kit for dm-multipath on RHEL 5.2 (and above) that populates multipath.conf with correct entries for all their supported FC SAN storage products. Works just great.
By Jim Perrin on Sep 2, 2009
Currently the guidance I’ve seen from IBM is ‘Use the rdac mpp driver from the lsi page’. I haven’t seen any multipath.conf configs from them, which indeed is a bit odd. I don’t claim to be the final resource, so if anyone’s aware of an IBM blessed multipath.conf config, please let me know.
By Jonathan Buzzard on Sep 3, 2009
We where advised by LSI/IBM is that the RDAC drive is being depreciated in favour of dm-multipath. This was in response to asking why you cannot remove a device from a machine without rebooting, when I can do that just fine with a USB stick.
There is also the issue of building a new RDAC initrd for a SAN booting machine following LSI’s published procedure. Though that can be fixed with a simple patch to the RDAC makefile.
However due to what I can only describe as bone headiness on the part of RedHat you still cannot SAN boot from a DS4000 series device using dm-multipath on RedHat/CentOS because the multipath error handler is not included in the initrd.
The following bugs contain the bitch fest, it might be fixed in 5.5!
https://bugzilla.redhat.com/show_bug.cgi?id=460899
https://bugzilla.redhat.com/show_bug.cgi?id=492052
By Jim Perrin on Sep 3, 2009
Yes, I’ve heard similar deprecation talk but I have yet to see any multipath configuration advice from LSI or IBM regarding the DS4XXX series. I worked with some @redhat.com folks to get things moving with the config above, but there are a couple issues where it could improve.
By Jim Perrin on Sep 3, 2009
Also, thanks for the bugzilla links. Those should prove helpful also
By Thiago on Sep 30, 2009
Got here when browsing for multipath.conf information on DS4xxx; just wanted to leave my .2c regarding one comment posted above, about removing devices.. I do not have any problems removing LUNs from recent Linux/kernel, there’s no need for reboot. Are you making use of the “delete” attribute available under the disk/device on /proc ?
Sample:
echo 1 > /sys/bus/scsi/drivers/sd/0\:0\:6\:0/delete
Of course you have to be sure that nobody is using the device.