Contents
This section describes how to manage failover and path load balancing for multiple paths between the servers and block storage devices.
Multipathing is the ability of a server to communicate with the same physical or logical block storage device across multiple physical paths between the host bus adapters in the server and the storage controllers for the device, typically in Fibre Channel (FC) or iSCSI SAN environments. You can also achieve multiple connections with direct attached storage when multiple channels are available.
Linux multipathing provides connection fault tolerance and can provide load balancing across the active connections. When multipathing is configured and running, it automatically isolates and identifies device connection failures, and reroutes I/O to alternate connections.
Typical connection problems involve faulty adapters, cables, or controllers. When you configure multipath I/O for a device, the multipath driver monitors the active connection between devices. When the multipath driver detects I/O errors for an active path, it fails over the traffic to the device’s designated secondary path. When the preferred path becomes healthy again, control can be returned to the preferred path.
Use the guidelines in this section when planning your multipath I/O solution.
Multipathing is managed at the device level.
The storage array you use for the multipathed device must support multipathing. For more information, see Section 7.2.9, “Supported Storage Arrays for Multipathing”.
You need to configure multipathing only if multiple physical paths exist between host bus adapters in the server and host bus controllers for the block storage device. You configure multipathing for the logical device as seen by the server.
For some storage arrays, the vendor provides its own multipathing software to manage multipathing for the array’s physical and logical devices. In this case, you should follow the vendor’s instructions for configuring multipathing for those devices.
Perform the following disk management tasks before you attempt to configure multipathing for a physical or logical device that has multiple paths:
Use third-party tools to carve physical disks into smaller logical disks.
Use third-party tools to partition physical or logical disks. If you change the partitioning in the running system, the Device Mapper Multipath (DM-MP) module does not automatically detect and reflect these changes. DM-MPIO must be reinitialized, which usually requires a reboot.
Use third-party SAN array management tools to create and configure hardware RAID devices.
Use third-party SAN array management tools to create logical devices such as LUNs. Logical device types that are supported for a given array depend on the array vendor.
The Linux software RAID management software runs on top of multipathing. For each device that has multiple I/O paths and that you plan to use in a software RAID, you must configure the device for multipathing before you attempt to create the software RAID device. Automatic discovery of multipathed devices is not available. The software RAID is not aware of the multipathing management running underneath.
High-availability solutions for clustering typically run on top of the multipathing server. For example, the Distributed Replicated Block Device (DRBD) high-availability solution for mirroring devices across a LAN runs on top of multipathing. For each device that has multiple I/O paths and that you plan to use in a DRDB solution, you must configure the device for multipathing before you configure DRBD.
Volume managers such as LVM2 and EVMS run on top of multipathing. You must configure multipathing for a device before you use LVM2 or EVMS to create segment managers and file systems on it.
When using multipathing in a virtualization environment, the multipathing is controlled in the host server environment. Configure multipathing for the device before you assign it to a virtual guest machine.
If you want to use the entire LUN directly (for example, if you are
using the SAN features to partition your storage), you can use the
/dev/disk/by-id/xxx
names for
mkfs, fstab, your application,
etc.
If the user-friendly names option is enabled in the
/etc/multipath.conf
file, you can use the
/dev/disk/by-id/dm-uuid-.*-mpath-.*
device name
because this name is aliased to the device ID. For information, see
Section 7.4.5.3, “Configuring User-Friendly Names or Alias Names in /etc/multipath.conf”.
By default, LVM2 does not recognize multipathed devices. To make LVM2
recognize the multipathed devices as possible physical volumes, you
must modify /etc/lvm/lvm.conf
. It is important to
modify it so that it does not scan and use the physical paths, but only
accesses the multipath I/O storage through the multipath I/O layer. If
you are using user-friendly names, make sure to specify the path so
that it scans only the device mapper names for the device
(/dev/disk/by-id/dm-uuid-.*-mpath-.*
) after
multipathing is configured.
To modify /etc/lvm/lvm.conf
for multipath use:
Open the /etc/lvm/lvm.conf
file in a text
editor.
If /etc/lvm/lvm.conf
does not exist, you can
create one based on your current LVM configuration by entering the
following at a terminal console prompt:
lvm dumpconfig > /etc/lvm/lvm.conf
Change the filter
and types
entries in /etc/lvm/lvm.conf
as follows:
filter = [ "a|/dev/disk/by-id/.*|", "r|.*|" ] types = [ "device-mapper", 1 ]
This allows LVM2 to scan only the by-id paths and reject everything else.
If you are using user-friendly names, specify the path as follows so that only the device mapper names are scanned after multipathing is configured:
filter = [ "a|/dev/disk/by-id/dm-uuid-.*-mpath-.*|", "r|.*|" ]
If you are also using LVM2 on non-multipathed devices, make the
necessary adjustments in the filter
and
types
entries to suit your setup. Otherwise, the
other LVM devices are not visible with a pvscan
after you modify the lvm.conf
file for
multipathing.
You want only those devices that are configured with LVM to be included in the LVM cache, so make sure you are specific about which other non-multipathed devices are included by the filter.
For example, if your local disk is /dev/sda
and
all SAN devices are /dev/sdb
and above, specify
the local and multipathing paths in the filter as follows:
filter = [ "a|/dev/sda.*|", "a|/dev/disk/by-id/.*|", "r|.*|" ] types = [ "device-mapper", 253 ]
Save the file.
Add dm-multipath to
/etc/sysconfig/kernel:INITRD_MODULES
.
Make a new initrd
to ensure that the Device
Mapper Multipath services are loaded with the changed settings. Enter
the following at a terminal console prompt:
mkinitrd -f multipath
Reboot the server to apply the changes.
The mdadm tool requires that the devices be accessed
by the ID rather than by the device node path. Therefore, the
DEVICE
entry in
/etc/mdadm.conf
should be set as follows:
DEVICE /dev/disk/by-id/*
If you are using user-friendly names, specify the path as follows so that only the device mapper names are scanned after multipathing is configured:
DEVICE /dev/disk/by-id/dm-uuid-.*-mpath-.*
The --noflush
option should always be used when
running on multipath devices.
For example, in scripts where you perform a table reload, you use the
--noflush
option on resume to ensure that any
outstanding I/O is not flushed, because you need the multipath topology
information.
load resume --noflush
A system with root (/
) on a multipath device might
stall when all paths have failed and are removed from the system
because a dev_loss_tmo
time-out is received from the
storage subsystem (such as Fibre Channel storage arrays).
If the system device is configured with multiple paths and the
multipath no_path_retry
setting is active, you
should modify the storage subsystem’s dev_loss_tmo
setting accordingly to ensure that no devices are removed during an
all-paths-down scenario. We strongly recommend that you set the
dev_loss_tmo
value to be equal to or higher than the
no_path_retry
setting from multipath.
The recommended setting for the storage subsystem’s
dev_los_tmo
is:
<dev_loss_tmo> = <no_path_retry> * <polling_interval>
where the following definitions apply for the multipath values:
no_path_retry
is the number of retries for
multipath I/O until the path is considered to be lost, and queuing of
IO is stopped.
polling_interval
is the time in seconds between
path checks.
Each of these multipath values should be set from the
/etc/multipath.conf
configuration file. For
information, see Section 7.4.5, “Creating and Configuring the /etc/multipath.conf File”.
Behavior changes for how multipathed devices are handled might affect your configuration if you are upgrading.
In SUSE® Linux Enterprise Server 11, the default multipath setup
relies on udev to overwrite the existing symbolic
links in the /dev/disk/by-id
directory when
multipathing is started. Before you start multipathing, the link
points to the SCSI device by using its scsi-xxx
name. When multipathing is running, the symbolic link points to the
device by using its dm-uuid-xxx
name. This
ensures that the symbolic links in the
/dev/disk/by-id
path persistently point to the
same device regardless of whether multipath is started or not. The
configuration files (such as lvm.conf
and
md.conf
) do not need to be modified because they
automatically point to the correct device.
In SUSE Linux Enterprise Server 10, the kpartx
software is used in the
/etc/init.d/boot.multipath
to add symlinks to the
/dev/dm-*
line in the
multipath.conf
configuration file for any newly
created partitions without requiring a reboot. This triggers
udevd
to fill in the
/dev/disk/by-*
symlinks. The main benefit is that
you can call kpartx
with the new parameters
without rebooting the server.
In SUSE Linux Enterprise Server 9, it is not possible to partition
multipath I/O devices themselves. If the underlying physical device is
already partitioned, the multipath I/O device reflects those
partitions and the layer provides
/dev/disk/by-id/<name>p1 ... pN
devices so
you can access the partitions through the multipath I/O layer. As a
consequence, the devices need to be partitioned prior to enabling
multipath I/O. If you change the partitioning in the running system,
DM-MPIO does not automatically detect and reflect these changes. The
device must be reinitialized, which usually requires a reboot.
The multipathing drivers and tools support all seven of the supported processor architectures: IA32, AMD64/EM64T, IPF/IA64, p-Series (32-bit and 64-bit), and z-Series (31-bit and 64-bit).
The multipathing drivers and tools support most storage arrays. The storage array that houses the multipathed device must support multipathing in order to use the multipathing drivers and tools. Some storage array vendors provide their own multipathing management tools. Consult the vendor’s hardware documentation to determine what settings are required.
The multipath-tools
package automatically detects
the following storage arrays:
3PARdata VV |
Compaq* HSV110 |
Compaq MSA1000 |
DDN SAN MultiDirector |
DEC* HSG80 |
EMC* CLARiiON* CX |
EMC Symmetrix* |
FSC CentricStor* |
Hewlett Packard* (HP*) A6189A |
HP HSV110 |
HP HSV210 |
HP Open |
Hitachi* DF400 |
Hitachi DF500 |
Hitachi DF600 |
IBM 3542 |
IBM ProFibre 4000R |
NetApp* |
SGI* TP9100 |
SGI TP9300 |
SGI TP9400 |
SGI TP9500 |
STK OPENstorage DS280 |
Sun* StorEdge 3510 |
Sun T4 |
In general, most other storage arrays should work. When storage arrays
are automatically detected, the default settings for multipathing
apply. If you want non-default settings, you must manually create and
configure the /etc/multipath.conf
file. For
information, see
Section 7.4.5, “Creating and Configuring the /etc/multipath.conf File”.
Testing of the IBM zSeries* device with multipathing has shown that
the dev_loss_tmo parameter should be set to 90 seconds, and the
fast_io_fail_tmo parameter should be set to 5 seconds. If you are
using zSeries devices, you must manually create and configure the
/etc/multipath.conf
file to specify the values.
For information, see
Section 7.4.5.6, “Configuring Default Settings for zSeries in /etc/multipath.conf”.
Hardware that is not automatically detected requires an appropriate
entry for configuration in the DEVICES
section of
the /etc/multipath.conf
file. In this case, you
must manually create and configure the configuration file. For
information, see
Section 7.4.5, “Creating and Configuring the /etc/multipath.conf File”.
Consider the following caveats:
Not all of the storage arrays that are automatically detected have been tested on SUSE Linux Enterprise Server. For information, see Section 7.2.9.2, “Tested Storage Arrays for Multipathing Support”.
Some storage arrays might require specific hardware handlers. A hardware handler is a kernel module that performs hardware-specific actions when switching path groups and dealing with I/O errors. For information, see Section 7.2.9.3, “Storage Arrays that Require Specific Hardware Handlers”.
After you modify the /etc/multipath.conf
file,
you must run mkinitrd to re-create the INITRD on
your system, then reboot in order for the changes to take effect.
The following storage arrays have been tested with SUSE Linux Enterprise Server:
EMC |
Hitachi |
Hewlett-Packard/Compaq |
IBM |
NetApp |
SGI |
Most other vendor storage arrays should also work. Consult your
vendor’s documentation for guidance. For a list of the default
storage arrays recognized by the multipath-tools
package, see Section 7.2.9.1, “Storage Arrays That Are Automatically Detected for Multipathing”.
Storage arrays that require special commands on failover from one path to the other or that require special nonstandard error handling might require more extensive support. Therefore, the Device Mapper Multipath service has hooks for hardware handlers. For example, one such handler for the EMC CLARiiON CX family of arrays is already provided.
![]() | |
Consult the hardware vendor’s documentation to determine if its hardware handler must be installed for Device Mapper Multipath. |
The multipath -t command shows an internal table of
storage arrays that require special handling with specific hardware
handlers. The displayed list is not an exhaustive list of supported
storage arrays. It lists only those arrays that require special
handling and that the multipath-tools
developers
had access to during the tool development.
![]() | |
Arrays with true active/active multipath support do not require special handling, so they are not listed for the multipath -t command. |
A listing in the multipath -t table does not necessarily mean that SUSE Linux Enterprise Server was tested on that specific hardware. For a list of tested storage arrays, see Section 7.2.9.2, “Tested Storage Arrays for Multipathing Support”.
The multipathing support in SUSE Linux Enterprise Server 10 and later is
based on the Device Mapper Multipath module of the Linux 2.6 kernel and
the multipath-tools
userspace package. You can
use the Multiple Devices Administration utility (MDADM,
mdadm) to view the status of multipathed devices.
The Device Mapper Multipath (DM-MP) module provides the multipathing capability for Linux. DM-MPIO is the preferred solution for multipathing on SUSE Linux Enterprise Server 11. It is the only multipathing option shipped with the product that is completely supported by Novell® and SUSE.
DM-MPIO features automatic configuration of the multipathing subsystem for a large variety of setups. Configurations of up to 8 paths to each device are supported. Configurations are supported for active/passive (one path active, others passive) or active/active (all paths active with round-robin load balancing).
The DM-MPIO framework is extensible in two ways:
Using specific hardware handlers. For information, see Section 7.2.9.3, “Storage Arrays that Require Specific Hardware Handlers”.
Using load-balancing algorithms that are more sophisticated than the round-robin algorithm
The user-space component of DM-MPIO takes care of automatic path discovery and grouping, as well as automated path retesting, so that a previously failed path is automatically reinstated when it becomes healthy again. This minimizes the need for administrator attention in a production environment.
DM-MPIO protects against failures in the paths to the device, and not failures in the device itself. If one of the active paths is lost (for example, a network adapter breaks or a fiber-optic cable is removed), I/O is redirected to the remaining paths. If the configuration is active/passive, then the path fails over to one of the passive paths. If you are using the round-robin load-balancing configuration, the traffic is balanced across the remaining healthy paths. If all active paths fail, inactive secondary paths must be waked up, so failover occurs with a delay of approximately 30 seconds.
If a disk array has more than one storage processor, make sure that the SAN switch has a connection to the storage processor that owns the LUNs you want to access. On most disk arrays, all LUNs belong to both storage processors, so both connections are active.
![]() | |
On some disk arrays, the storage array manages the traffic through storage processors so that it presents only one storage processor at a time. One processor is active and the other one is passive until there is a failure. If you are connected to the wrong storage processor (the one with the passive path) you might not see the expected LUNs, or you might see the LUNs but get errors when you try to access them. |
Table 7.1. Multipath I/O Features of Storage Arrays¶
Device Mapper Multipath detects every path for a multipathed device as
a separate SCSI device. The SCSI device names take the form
/dev/sd
, where
N
is an autogenerated
letter for the device, beginning with a and issued sequentially as the
devices are created, such as N
/dev/sda
,
/dev/sdb
, and so on. If the number of devices
exceeds 26, the letters are duplicated so that the next device after
/dev/sdz
will be named
/dev/sdaa
, /dev/sdab
, and so
on.
If multiple paths are not automatically detected, you can configure
them manually in the /etc/multipath.conf
file. The
multipath.conf
file does not exist until you
create and configure it. For information, see
Section 7.4.5, “Creating and Configuring the /etc/multipath.conf File”.
The multipath-tools user-space package takes care of automatic path discovery and grouping. It automatically tests the path periodically, so that a previously failed path is automatically reinstated when it becomes healthy again. This minimizes the need for administrator attention in a production environment.
Table 7.2. Tools in the multipath-tools Package¶
The file list for a package can vary for different server architectures. For a list of files included in the multipath-tools package, go to the SUSE Linux Enterprise Server Technical Specifications > Package Descriptions Web page, find your architecture and select , then search on “multipath-tools” to find the package list for that architecture.
You can also determine the file list for an RPM file by querying the package itself: using the rpm -ql or rpm -qpl command options.
To query an installed package, enter
rpm -ql <package_name
>
To query a package not installed, enter
rpm -qpl <URL_or_path_to_package
>
To check that the multipath-tools
package is
installed, do the following:
Udev is the default device handler, and devices are automatically known
to the system by the Worldwide ID instead of by the device node name.
This resolves problems in previous releases of MDADM and LVM where the
configuration files (mdadm.conf
and
lvm.conf)
did not properly recognize multipathed
devices.
Just as for LVM2, MDADM requires that the devices be accessed by the ID
rather than by the device node path. Therefore, the
DEVICE
entry in
/etc/mdadm.conf
should be set as follows:
DEVICE /dev/disk/by-id/*
If you are using user-friendly names, specify the path as follows so that only the device mapper names are scanned after multipathing is configured:
DEVICE /dev/disk/by-id/dm-uuid-.*-mpath-.*
To verify that MDADM is installed:
Ensure that the mdadm
package is installed by
entering the following at a terminal console prompt:
rpm -q mdadm
If it is installed, the response repeats the package name and provides the version information. For example:
mdadm-2.6-0.11
If it is not installed, the response reads:
package mdadm is not installed
For information about modifying the /etc/lvm/lvm.conf
file, see
Section 7.2.3, “Using LVM2 on Multipath Devices”.
Use the Linux multipath(8) command to configure and manage multipathed devices.
General syntax for the multipath(8) command:
multipath [-v verbosity] [-d] [-h|-l|-ll|-f|-F] [-p failover | multibus | group_by_serial | group_by_prio| group_by_node_name ]
Configure all multipath devices.
devicename
Configure a specific multipath device.
Replace devicename
with the device node
name such as /dev/sdb
(as shown by udev in the
$DEVNAME variable), or in the major:minor
format.
Selectively suppress a multipath map, and its device-mapped partitions.
Display potential multipath devices, but do not create any devices and do not update device maps (dry run).
Configure multipath devices and display multipath map information for them. The -v2 option in multipath -v2 -d shows only local disks.
devicename
Configure a specific multipath device and display multipath map information for it. The -v2 option shows only local disks.
Configure multipath devices and display multipath map information for them. The -v3 option shows the full path list.
devicename
Configure a specific multipath device and display multipath map information for it. The -v3 option shows the full path list.
Display the status of all multipath devices.
devicename
Display the status of a specified multipath device.
Flush all unused multipath device maps. This unresolves the multiple paths; it does not delete the devices.
devicename
Flush unused multipath device maps for a specified multipath device. This unresolves the multiple paths; it does not delete the device.
Set the group policy by specifying one of the group policy options that are described in Table 7.3, “Group Policy Options for the multipath -p Command”:
Table 7.3. Group Policy Options for the multipath -p Command¶
Before configuring multipath I/O for your SAN devices, prepare the SAN devices, as necessary, by doing the following:
Configure and zone the SAN with the vendor’s tools.
Configure permissions for host LUNs on the storage arrays with the vendor’s tools.
Install the Linux HBA driver module. Upon module installation, the driver automatically scans the HBA to discover any SAN devices that have permissions for the host. It presents them to the host for further configuration.
![]() | |
Ensure that the HBA driver you are using does not have native multipathing enabled. |
See the vendor’s specific instructions for more details.
After the driver module is loaded, discover the device nodes assigned to specific array LUNs or partitions.
If the SAN device will be used as the root device on the server, modify the timeout settings for the device as described in Section 7.2.6, “SAN Timeout Settings When the Root Device Is Multipathed”.
If the LUNs are not seen by the HBA driver, lsscsi can be used to check whether the SCSI devices are seen correctly by the operating system. When the LUNs are not seen by the HBA driver, check the zoning setup of the SAN. In particular, check whether LUN masking is active and whether the LUNs are correctly assigned to the server.
If the LUNs are seen by the HBA driver, but there are no corresponding block devices, additional kernel parameters are needed to change the SCSI device scanning behavior, such as to indicate that LUNs are not numbered consecutively. For information, see Options for SCSI Device Scanning in the Novell Support Knowledgebase.
Partitioning devices that have multiple paths is not recommended, but it is supported.
In SUSE Linux Enterprise Server 10, you can use the kpartx tool to create partitions on multipathed devices without rebooting. You can also partition the device before you attempt to configure multipathing by using the Partitioner function in YaST2 or by using a third-party partitioning tool.
In SUSE Linux Enterprise Server 9, if you want to partition the device, you should configure its partitions before you attempt to configure multipathing by using the Partitioner function in YaST2 or by using a third-party partitioning tool. This is necessary because partitioning an existing multipathed device is not supported. Partitioning operations on multipathed devices fail if attempted.
If you configure partitions for a device, DM-MPIO automatically
recognizes the partitions and indicates them by appending p1 to
pn
to the device’s ID, such as
/dev/disk/by-id/
26353900f02796769p1
To partition multipathed devices, you must disable the DM-MPIO
service, partition the normal device node (such as
/dev/sdc
), then reboot to allow the DM-MPIO
service to see the new partitions.
The system must be manually configured to automatically load the device
drivers for the controllers to which the multipath I/O devices are
connected within the initrd
. You need to add the
necessary driver module to the variable INITRD_MODULES in the file
/etc/sysconfig/kernel
.
For example, if your system contains a RAID controller accessed by the
cciss
driver and multipathed devices connected to
a QLogic* controller accessed by the driver qla2xxx, this entry would
look like:
INITRD_MODULES="cciss"
Because the QLogic driver is not automatically loaded on startup, add it here:
INITRD_MODULES="cciss qla23xx"
After changing /etc/sysconfig/kernel
, you must
re-create the initrd
on your system with the
mkinitrd command, then reboot in order for the
changes to take effect.
When you are using LILO as a boot manager, reinstall it with the /sbin/lilo command. No further action is required if you are using GRUB.
Use either of the methods in this section to add multipath I/O services (multipathd) to the boot sequence.
The /etc/multipath.conf
file does not exist unless
you create it. The
/usr/share/doc/packages/multipath-tools/multipath.conf.synthetic
file contains a sample /etc/multipath.conf
file
that you can use as a guide for multipath settings. See
/usr/share/doc/packages/multipath-tools/multipath.conf.annotated
for a template with extensive comments for each of the attributes and
their options.
If the /etc/multipath.conf
file does not exist,
copy the example to create the file:
In a terminal console, log in as the root
user.
Enter the following command (all on one line, of course) to copy the template:
cp /usr/share/doc/packages/multipath-tools/multipath.conf.synthetic /etc/multipath.conf
Use the
/usr/share/doc/packages/multipath-tools/multipath.conf.annotated
file as a reference to determine how to configure multipathing for
your system.
Make sure there is an appropriate device entry for your SAN. Most vendors provide documentation on the proper setup of the device section.
The /etc/multipath.conf
file requires a
different device section for different SANs. If
you are using a storage subsystem that is automatically detected
(see Section 7.2.9.2, “Tested Storage Arrays for Multipathing Support”), the
default entry for that device can be used; no further configuration
of the /etc/multipath.conf
file is required.
Save the file.
After setting up the configuration, you can perform a “dry run” by entering
multipath -v3 -d
This command scans the devices, then displays what the setup would look like. The output is similar to the following:
26353900f02796769 [size=127 GB] [features="0"] [hwhandler="1 emc"]
\_ round-robin 0 [first] \_ 1:0:1:2 sdav 66:240 [ready ] \_ 0:0:1:2 sdr 65:16 [ready ]
\_ round-robin 0 \_ 1:0:0:2 sdag 66:0 [ready ] \_ 0:0:0:2 sdc 8:32 [ready ]
Paths are grouped into priority groups. Only one priority group is in active use at a time. To model an active/active configuration, all paths end in the same group. To model active/passive configuration, the paths that should not be active in parallel are placed in several distinct priority groups. This normally happens automatically on device discovery.
The output shows the order, the scheduling policy used to balance I/O within the group, and the paths for each priority group. For each path, its physical address (host:bus:target:lun), device node name, major:minor number, and state is shown.
A multipath device can be identified by either its WWID (Worldwide
Identifier) or an alias that you assign for it. The WWID is an
identifier for the multipath device that is guaranteed to be globally
unique and unchanging. The default name used in multipathing is the ID
of the logical unit as found in the
/dev/disk/by-id
directory. Because device node
names in the form of /dev/sdn
and
/dev/dm-n
can change on reboot, referring to
multipath devices by the ID is preferred.
The multipath device names in the /dev/mapper
directory reference the ID of the LUN and are always consistent
because they use the /var/lib/multipath/bindings
file to track the association. These device names are user-friendly
names such as
/dev/disk/by-id/dm-uuid-.*-mpath-.*
.
You can use the ALIAS directive in the
/etc/multipath.conf
file to specify your own
device names. Alias names override the use of ID and
/dev/disk/by-id/dm-uuid-.*-mpath-.*
names.
![]() | |
We recommend that you do not use aliases for the root device, because the ability to seamlessly switch off multipathing via the kernel command line is lost if the device name differs. |
For an example of multipath.conf
settings, see
the
/usr/share/doc/packages/multipath-tools/multipath.conf.synthetic
file.
In a terminal console, log in as the root
user.
Open the /etc/multipath.conf
file in a text
editor.
Uncomment the Defaults
directive and its ending
bracket.
Uncomment the user_friendly_names option
, then
change its value from No to Yes.
For example:
## Use user friendly names, instead of using WWIDs as names. defaults { user_friendly_names yes }
Optionally specify your own user-friendly names for devices using the alias directive in the multipath section.
For example:
multipath { wwid 26353900f02796769 alias sdd4l0 }
Save your changes, then close the file.
The /etc/multipath.conf
file should contain a
blacklist section where all non-multipathed devices
are listed. For example, local IDE hard drives and floppy drives are
not normally multipathed. If you have single-path devices that
multipath is trying to manage and you want
multipath to ignore them, put them in the
blacklist section to resolve the problem.
![]() | |
The keyword devnode_blacklist has been deprecated and replaced with the keyword blacklist. |
For example, to blacklist local devices and all arrays from the
cciss
driver from being managed by multipath, the
blacklist section looks like this:
blacklist { wwid 26353900f02796769 devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st|sda)[0-9]*" devnode "^hd[a-z][0-9]*" devnode "^cciss!c[0-9]d[0-9].*" }
You can also blacklist only the partitions from a driver instead of the entire array. For example, using the following regular expression blacklists only partitions from the cciss driver and not the entire array:
^cciss!c[0-9]d[0-9]*[p[0-9]*]
After you modify the /etc/multipath.conf
file,
you must run mkinitrd to re-create the
initrd
on your system, then reboot in order for
the changes to take effect.
After you do this, the local devices should no longer be listed in the multipath maps when you issue the multipath -ll command.
The /etc/multipath.conf
file should contain a
defaults section where you can specify default
behaviors. If the field is not otherwise specified in a
device section, the default setting is applied for
that SAN configuration.
The following defaults section specifies a simple failover policy:
defaults { multipath_tool "/sbin/multipath -v0" udev_dir /dev polling_interval 10 default_selector "round-robin 0" default_path_grouping_policy failover default_getuid "/sbin/scsi_id -g -u -s /block/%n" default_prio_callout "/bin/true" default_features "0" rr_min_io 100 failback immediate
![]() | |
In the default_getuid command line, use the path
|
Testing of the IBM zSeries device with multipathing has shown that the
dev_loss_tmo parameter should be set to 90 seconds, and the
fast_io_fail_tmo parameter should be set to 5 seconds. If you are
using zSeries devices, modify the
/etc/multipath.conf
file to specify the values as
follows:
defaults { dev_loss_tmo 90 fast_io_fail_tmo 5 }
The dev_loss_tmo parameter sets the number of seconds to wait before marking a multipath link as bad. When the path fails, any current I/O on that failed path fails. The default value varies according to the device driver being used. The valid range of values is 0 to 600 seconds. To use the driver’s internal timeouts, set the value to zero (0) or to any value greater than 600.
The fast_io_fail_tmo parameter sets the length of time to wait before failing I/O when a link problem is detected. I/O that reaches the driver fails. If I/O is in a blocked queue, the I/O does not fail until the dev_loss_tmo time elapses and the queue is unblocked.
Changes to the /etc/multipath.conf
file cannot
take effect when multipathd is running. After you
make changes, save and close the file, then do the following to apply
the changes:
To start multipath services and enable them to start at reboot:
Open a terminal console, then log in as the
root
user or equivalent.
At the terminal console prompt, enter
chkconfig multipathd on
chkconfig boot.multipath on
If the boot.multipath service does not start automatically on system boot, do the following to start them manually:
In a Linux host, when there are multiple paths to a storage controller,
each path appears as a separate block device, and results in multiple
block devices for single LUN. The Device Mapper Multipath service
detects multiple paths with the same LUN ID, and creates a new multipath
device with that ID. For example, a host with two HBAs attached to a
storage controller with two ports via a single unzoned Fibre Channel
switch sees four block devices: /dev/sda
,
/dev/sdb
, /dev/sdc
, and
/dev/sdd
. The Device Mapper Multipath service
creates a single block device, /dev/mpath/mpath1
that reroutes I/O through those four underlying block devices.
This section describes how to specify policies for failover and configure priorities for the paths.
Use the multipath command with the -p option to set the path failover policy:
multipathdevicename
-ppolicy
Replace policy
with one of the following
policy options:
Table 7.4. Group Policy Options for the multipath -p Command¶
You must manually enter the failover priorities for the device in the
/etc/multipath.conf
file. Examples for all
settings and options can be found in the
/usr/share/doc/packages/multipath-tools/multipath.conf.annotated
file.
A priority group is a collection of paths that go to the same physical LUN. By default, I/O is distributed in a round-robin fashion across all paths in the group. The multipath command automatically creates priority groups for each LUN in the SAN based on the path_grouping_policy setting for that SAN. The multipath command multiplies the number of paths in a group by the group’s priority to determine which group is the primary. The group with the highest calculated value is the primary. When all paths in the primary group are failed, the priority group with the next highest value becomes active.
A path priority is an integer value assigned to a path. The higher the value, the higher the priority is. An external program is used to assign priorities for each path. For a given device, the paths with the same priorities belong to the same priority group.
Table 7.5. Multipath Attributes¶
Multipath Attribute |
Description |
Values |
---|---|---|
user_friendly_names |
Specifies whether to use IDs or to use the
|
yes: Autogenerate user-friendly names as aliases for the multipath devices instead of the actual ID. no:
Default. Use the WWIDs shown in the
|
blacklist |
Specifies the list of device names to ignore as non-multipathed devices, such as cciss, fd, hd, md, dm, sr, scd, st, ram, raw, loop. |
For an example, see Section 7.4.5.4, “Blacklisting Non-Multipathed Devices in /etc/multipath.conf”. |
blacklist_exceptions |
Specifies the list of device names to treat as multipath devices even if they are included in the blacklist. |
For an example, see the
|
failback |
Specifies whether to monitor the failed path recovery, and indicates the timing for group failback after failed paths return to service. When the failed path recovers, the path is added back into the multipath enabled path list based on this setting. Multipath evaluates the priority groups, and changes the active priority group when the priority of the primary path exceeds the secondary group. |
immediate: When a path recovers, enable the path immediately. n (> 0):
When the path recovers, wait manual: (Default) The failed path is not monitored for recovery. The administrator runs the multipath command to update enabled paths and priority groups. |
getuid |
The default program and arguments to call to obtain a unique path identifier. Should be specified with an absolute path. |
/sbin/scsi_id -g -u -s This is the default location and arguments. Example: getuid "/sbin/scsi_id -g -u -d /dev/%n" |
no_path_retry |
Specifies the behaviors to use on path failure. |
n (> 0): Specifies the number of retries until multipath stops the queuing and fails the path. Specify an integer value greater than 0. fail: Specified immediate failure (no queuing). queue. : Never stop queuing (queue forever until the path comes alive). |
path_grouping_policy |
Specifies the path grouping policy for a multipath device hosted by a given controller. |
failover: One path is assigned per priority group so that only one path at a time is used. multibus: (Default) All valid paths are in one priority group. Traffic is load-balanced across all active paths in the group. group_by_prio: One priority group exists for each path priority value. Paths with the same priority are in the same priority group. Priorities are assigned by an external program. group_by_serial: Paths are grouped by the SCSI target serial number (controller node WWN). group_by_node_name:
One priority group is assigned per target node name. Target node
names are fetched in
|
path_checker |
Determines the state of the path. |
directio:
(Default in readsector0:
(Default in tur:
Issues a SCSI test unit ready command to the device. This is the
preferred setting if the LUN supports it. On failure, the
command does not fill up Some SAN vendors provide custom path_checker options: |
path_selector |
Specifies the path-selector algorithm to use for load balancing. |
round-robin 0: (Default) The load-balancing algorithm used to balance traffic across all active paths in a priority group. Beginning in SUSE Linux Enterprise Server 11, the following additional I/O balancing options are available: least-pending: Provides a least-pending-I/O dynamic load balancing policy for bio based device mapper multipath. This load balancing policy considers the number of unserviced requests pending on a path and selects the path with least count of pending service requests. This policy is especially useful when the SAN environment has heterogeneous components. For example, when there is one 8 GB HBA and one 2 GB HBA connected to the same server, the 8 GB HBA could be utilized better with this algorithm. length-load-balancing: A dynamic load balancer that balances the number of in-flight I/O on paths similar to the least-pending option. service-time: A service-time oriented load balancer that balances I/O on paths according to the latency. |
pg_timeout |
Specifies path group timeout handling. |
NONE (internal default) |
prio_callout
Multipath prio_callouts are located in shared libraries in
|
Specifies the program and arguments to use to determine the layout of the multipath map. When queried by the multipath command, the specified mpath_prio_* callout program returns the priority for a given path in relation to the entire multipath layout. When it is used with the path_grouping_policy of group_by_prio, all paths with the same priority are grouped into one multipath group. The group with the highest aggregate priority becomes the active group. When all paths in a group fail, the group with the next highest aggregate priority becomes active. Additionally, a failover command (as determined by the hardware handler) might be send to the target. The mpath_prio_* program can also be a custom script created by a vendor or administrator for a specified setup.
A %n in the command line expands to the device name in the
A %b expands to the device number in
A %d expands to the device ID in the
If devices are hot-pluggable, use the %d flag instead of %n. This addresses the short time that elapses between the time when devices are available and when udev creates the device nodes. |
If no /bin/true: Use this value when the group_by_priority is not being used.
The prioritizer programs generate path
priorities when queried by the multipath
command. The program names must begin with
mpath_prio_alua %n: Generates path priorities based on the SCSI-3 ALUA settings. mpath_prio_balance_units: Generates the same priority for all paths. mpath_prio_emc %n: Generates the path priority for EMC arrays. mpath_prio_hds_modular %b: Generates the path priority for Hitachi HDS Modular storage arrays. mpath_prio_hp_sw %n: Generates the path priority for Compaq/HP controller in active/standby mode. mpath_prio_netapp %n: Generates the path priority for NetApp arrays. mpath_prio_random %n: Generates a random priority for each path. mpath_prio_rdac %n: Generates the path priority for LSI/Engenio RDAC controller. mpath_prio_tpc %n: You can optionally use a script created by a vendor or administrator that gets the priorities from a file where you specify priorities to use for each path. mpath_prio_spec.sh %n: Provides the path of a user-created script that generates the priorities for multipathing based on information contained in a second data file. (This path and filename are provided as an example. Specify the location of your script instead.) The script can be created by a vendor or administrator. The script’s target file identifies each path for all multipathed devices and specifies a priority for each path. For an example, see Section 7.6.3, “Using a Script to Set Path Priorities”. |
rr_min_io |
Specifies the number of I/O transactions to route to a path
before switching to the next path in the same path group, as
determined by the specified algorithm in the
| |
rr_weight |
Specifies the weighting method to use for paths. |
uniform: Default. All paths have the same round-robin weights. priorities: Each path’s weight is determined by the path’s priority times the rr_min_io setting. |
All paths are active. I/O is configured for some number of seconds or some number of I/O transactions before moving to the next open path in the sequence.
A single path with the highest priority (lowest value setting) is active for traffic. Other paths are available for failover, but are not used unless failover occurs.
Multiple paths with the same priority fall into the active group. When all paths in that group fail, the device fails over to the next highest priority group. All paths in the group share the traffic load in a round-robin load balancing fashion.
You can create a script that interacts with Device Mapper Multipath (DM-MPIO) to provide priorities for paths to the LUN when set as a resource for the prio_callout setting.
First, set up a text file that lists information about each device and
the priority values you want to assign to each path. For example, name
the file /usr/local/etc/primary-paths
. Enter one
line for each path in the following format:
host_wwpn target_wwpn scsi_id priority_value
Return a priority value for each path on the device. Make sure that the variable FILE_PRIMARY_PATHS resolves to a real file with appropriate data (host wwpn, target wwpn, scsi_id and priority value) for each device.
The contents of the primary-paths
file for a
single LUN with eight paths each might look like this:
0x10000000c95ebeb4 0x200200a0b8122c6e 2:0:0:0 sdb 3600a0b8000122c6d00000000453174fc 50
0x10000000c95ebeb4 0x200200a0b8122c6e 2:0:0:1 sdc 3600a0b80000fd6320000000045317563 2
0x10000000c95ebeb4 0x200200a0b8122c6e 2:0:0:2 sdd 3600a0b8000122c6d0000000345317524 50
0x10000000c95ebeb4 0x200200a0b8122c6e 2:0:0:3 sde 3600a0b80000fd6320000000245317593 2
0x10000000c95ebeb4 0x200300a0b8122c6e 2:0:1:0 sdi 3600a0b8000122c6d00000000453174fc 5
0x10000000c95ebeb4 0x200300a0b8122c6e 2:0:1:1 sdj 3600a0b80000fd6320000000045317563 51
0x10000000c95ebeb4 0x200300a0b8122c6e 2:0:1:2 sdk 3600a0b8000122c6d0000000345317524 5
0x10000000c95ebeb4 0x200300a0b8122c6e 2:0:1:3 sdl 3600a0b80000fd6320000000245317593 51
To continue the example mentioned in
Table 7.5, “Multipath Attributes”, create a script
named /usr/local/sbin/path_prio.sh
. You can use
any path and filename. The script does the following:
On query from multipath, grep the device and its path from the
/usr/local/etc/primary-paths
file.
Return to multipath the priority value in the last column for that entry in the file.
The mpath_prio_alua(8) command is used as a priority callout for the Linux multipath(8) command. It returns a number that is used by DM-MPIO to group SCSI devices with the same priority together. This path priority tool is based on ALUA (Asynchronous Logical Unit Access).
mpath_prio_alua [-ddirectory
] [-h] [-v] [-V]device
[device
...]
SCSI devices.
directory
Specifies the Linux directory path where the listed device node
names can be found. The default directory is
/dev
. When you use this option, specify the
device node name only (such as sda
) for the
device or devices you want to manage.
Displays help for this command, then exits.
Turns on verbose output to display status in human-readable format. Output includes information about which port group the specified device is in and its current state.
Displays the version number of this tool, then exits.
device
[device
...] Specifies the SCSI device (or multiple devices) that you want to manage. The device must be a SCSI device that supports the Report Target Port Groups (sg_rtpg(8)) command. Use one of the following formats for the device node name:
The full Linux directory path, such as
/dev/sda
. Do not use with the -d option.
The device node name only, such as sda
.
Specify the directory path by using the -d option.
The major and minor number of the device separated by a colon (:)
with no spaces, such as 8:0
. This creates a
temporary device node in the /dev
directory
with a name in the format of
tmpdev-<major>:<minor>-<pid>
.
For example, /dev/tmpdev-8:0-<pid>
.
On success, returns a value of 0 and the priority value for the group. Table 7.6, “ALUA Priorities for Device Mapper Multipath” shows the priority values returned by the mpath_prio_alua command.
Table 7.6. ALUA Priorities for Device Mapper Multipath¶
Values are widely spaced because of the way the multipath command handles them. It multiplies the number of paths in a group with the priority value for the group, then selects the group with the highest result. For example, if a non-optimized path group has six paths (6 x 10 = 60) and the optimized path group has a single path (1 x 50 = 50), the non-optimized group has the highest score, so multipath chooses the non-optimized group. Traffic to the device uses all six paths in the group in a round-robin fashion.
On failure, returns a value of 1 to 5 indicating the cause for the command’s failure. For information, see the man page for mpath_prio_alua.
Use the SCSI Report Target Port Groups (sg_rtpg(8)) command. For information, see the man page for sg_rtpg(8).
When using multipath I/O, you want any host bus adapter (HBA) failure or cable failures to be reported faster than when multipathing is not in use. Configure time-out settings for your HBA to disable failover at the HBA level to allow the failure to propagate up to the multipath I/O layer as fast as possible where the I/O can be redirected to another, healthy path.
To disable the HBA handling of failover, modify the driver’s options
in the /etc/modprobe.conf.local
file. Refer to the
HBA vendor’s documentation for information about how to disable
failover settings for your driver.
For example, for the QLogic qla2xxx family of host bus adapters, the following setting is recommended:
options qla2xxx qlport_down_retry=1
![]() | |
In the SUSE Linux Enterprise Server 10 SP1 initial release and earlier,
the root partition (
DM-MPIO is now available and supported for |
The multipathd daemon is not automatically active during the system installation. You can start it by using the
option in the YaST partitioner.During the install on the YaST2 Installation Settings page, click on
to open the YaST partitioner.Select
.Select the
main icon, click the button, then select .Start multipath.
YaST2 starts to rescan the disks and shows available multipath
devices (such as
/dev/mapper/3600a0b80000f4593000012ae4ab0ae65
).
This is the device that should be used for all further processing.
Click
to continue with the installation.The multipathd daemon is not automatically active during the system installation. You can start it by using the
option in the YaST partitioner.During the install on the YaST2 Installation Settings page, click on
to open the YaST partitioner.Select
.Select the
main icon, click the button, then select .Start multipath.
YaST2 starts to rescan the disks and shows available multipath
devices (such as
/dev/mapper/3600a0b80000f4593000012ae4ab0ae65
).
This is the device that should be used for all further processing.
Write down the device path and UUID; you need it later.
Click
to continue with the installation.After all settings are done and the installation finished, YaST2 starts to write the boot loader information, and displays a countdown to restart the system. Stop the counter by clicking the
button and press CTRL+ALT+F5 to access a console.
Use the console to determine if a passive path was entered in the
/boot/grub/device.map
file for the
hd0
entry.
This is necessary because the installation does not distinguish between active and passive paths.
Mount the root device to /mnt
by entering
mount /dev/mapper/UUID_part2 /mnt
For example, enter
mount /dev/mapper/3600a0b80000f4593000012ae4ab0ae65_part2 /mnt
Mount the boot device to /mnt/boot
by entering
mount /dev/mapper/UUID_part1 /mnt/boot
For example, enter
mount /dev/mapper/3600a0b80000f4593000012ae4ab0ae65_part1 /mnt/boot
Open /mnt/boot/grub/device.map
file by
entering
less /mnt/boot/grub/device.map
In the /mnt/boot/grub/device.map
file,
determine if the hd0
entry points to a passive
path, then do one of the following:
If the hd0 entry points to a passive path, change the configuration and reinstall the boot loader:
At the console, enter the following commands at the console prompt:
mount -o bind /dev /mnt/dev mount -o bind /sys /mnt/sys mount -o bind /proc /mnt/proc chroot
At the console, run multipath -ll, then check the output to find the active path.
Passive paths are flagged as ghost
.
In the /mnt/boot/grub/device.map
file, change
the hd0
entry to an active path, save the
changes, and close the file.
In case the selection was to boot from MBR,
/etc/grub.conf
should look like the following:
setup --stage2=/boot/grub/stage2 (hd0) (hd0,0) quit
Reinstall the boot loader by entering
grub < /etc/grub.conf
Enter the following commands:
exit umount /mnt/* umount /mnt
Return to the YaST graphical environment by pressing CTRL+ALT+F7.
Click
to continue with the installation reboot.
Install Linux with only a single path active, preferably one where
the by-id
symlinks are listed in the
partitioner.
Mount the devices by using the /dev/disk/by-id
path used during the install.
After installation, add dm-multipath to
/etc/sysconfig/kernel:INITRD_MODULES
.
For System Z, before running mkinitrd, edit the
/etc/zipl.conf
file to change the by-path
information in zipl.conf
with the same by-id
information that was used in the /etc/fstab
.
Re-run /sbin/mkinitrd to update the
initrd
image.
For System Z, after running mkinitrd, run zipl.
Reboot the server.
Ideally, you should configure multipathing for devices before you use them as components of a software RAID device. If you add multipathing after creating any software RAID devices, the DM-MPIO service might be starting after the multipath service on reboot, which makes multipathing appear not to be available for RAIDs. You can use the procedure in this section to get multipathing running for a previously existing software RAID.
For example, you might need to configure multipathing for devices in a software RAID under the following circumstances:
If you create a new software RAID as part of the Partitioning settings during a new install or upgrade.
If you did not configure the devices for multipathing before using them in the software RAID as a member device or spare.
If you grow your system by adding new HBA adapters to the server or expanding the storage subsystem in your SAN.
![]() | |
The following instructions assume the software RAID device is
|
Open a terminal console, then log in as the
root
user or equivalent.
Except where otherwise directed, use this console to enter the commands in the following steps.
If any software RAID devices are currently mounted or running, enter the following commands for each device to dismount the device and stop it.
umount /dev/mapper/mpath0
mdadm --misc --stop /dev/mapper/mpath0
Stop the boot.md service by entering
/etc/init.d/boot.md stop
Start the boot.multipath
and
multipathd
services by entering the following
commands:
/etc/init.d/boot.multipath start
/etc/init.s/multipathd start
After the multipathing services are started, verify that the software
RAID’s component devices are listed in the
/dev/disk/by-id
directory. Do one of the
following:
Devices Are Listed:
The device names should now have symbolic links to their Device
Mapper Multipath device names, such as
/dev/dm-1
.
Devices Are Not Listed: Force the multipath service to recognize them by flushing and rediscovering the devices.
To do this, enter the following commands:
multipath -F
multipath -v0
The devices should now be listed in
/dev/disk/by-id
, and have symbolic links to
their Device Mapper Multipath device names. For example:
lrwxrwxrwx 1 root root 10 Jun 15 09:36 scsi-mpath1 -> ../../dm-1
Restart the boot.md
service and the RAID device
by entering
/etc/init.d/boot.md start
Check the status of the software RAID by entering
mdadm --detail /dev/mapper/mpath0
The RAID’s component devices should match their Device Mapper
Multipath device names that are listed as the symbolic links of
devices in the /dev/disk/by-id
directory.
Make a new initrd
to ensure that the Device
Mapper Multipath services are loaded before the RAID services on
reboot. Enter
mkinitrd -f multipath
Reboot the server to apply these post-install configuration settings.
Verify that the software RAID array comes up properly on top of the multipathed devices by checking the RAID status. Enter
mdadm --detail /dev/mapper/mpath0
For example:
Number Major Minor RaidDevice State
|
0 253 0 0 active sync /dev/dm-0
|
1 253 1 1 active sync /dev/dm-1
|
2 253 2 2 active sync /dev/dm-2
|
If your system has already been configured for multipathing and you later need to add more storage to the SAN, you can use the rescan-scsi-bus.sh script to scan for the new devices. By default, this script scans all HBAs with typical LUN ranges.
rescan-scsi-bus.sh [options] [host [host ...]]
You can specify hosts on the command line (deprecated), or use the
--hosts=LIST
option (recommended).
For most storage subsystems, the script can be run successfully without options. However, some special cases might need to use one or more of the following parameters for the rescan-scsi-bus.sh script:
Use the following procedure to scan the devices and make them available to multipathing without rebooting the system.
On the storage subsystem, use the vendor’s tools to allocate the device and update its access control settings to allow the Linux system access to the new storage. Refer to the vendor’s documentation for details.
Scan all targets for a host to make its new device known to the middle layer of the Linux kernel’s SCSI subsystem. At a terminal console prompt, enter
rescan-scsi-bus.sh [options]
Check for scanning progress in the system log (the
/var/log/messages
file). At a terminal console
prompt, enter
tail -30 /var/log/messages
This command displays the last 30 lines of the log. For example:
# tail -30 /var/log/messages . . . Feb 14 01:03 kernel: SCSI device sde: 81920000 Feb 14 01:03 kernel: SCSI device sdf: 81920000 Feb 14 01:03 multipathd: sde: path checker registered Feb 14 01:03 multipathd: sdf: path checker registered Feb 14 01:03 multipathd: mpath4: event checker started Feb 14 01:03 multipathd: mpath5: event checker started Feb 14 01:03:multipathd: mpath4: remaining active paths: 1 Feb 14 01:03 multipathd: mpath5: remaining active paths: 1
Repeat Step 2 through Step 3 to add paths through other HBA adapters on the Linux system that are connected to the new device.
Run the multipath command to recognize the devices for DM-MPIO configuration. At a terminal console prompt, enter
multipath
You can now configure the new device for multipathing.
Use the example in this section to detect a newly added multipathed LUN without rebooting.
Open a terminal console, then log in as the
root
user.
Scan all targets for a host to make its new device known to the middle layer of the Linux kernel’s SCSI subsystem. At a terminal console prompt, enter
rescan-scsi-bus.sh [options]
For syntax and options information for the
rescan-scsi-bus-sh
script, see
Section 7.10, “Scanning for New Devices without Rebooting”.
Verify that the device is seen (the link has a new time stamp) by entering
ls -lrt /dev/dm-*
Verify the new WWN of the device appears in the log by entering
tail -33 /var/log/messages
Use a text editor to add a new alias definition for the device in the
/etc/multipath.conf
file, such as
oradata3
.
Create a partition table for the device by entering
fdisk /dev/dm-8
Trigger udev by entering
echo 'add' > /sys/block/dm-8/uevent
This generates the device-mapper devices for the partitions on
dm-8
.
Create a file system and label for the new partition by entering
mke2fs -j /dev/dm-9
tune2fs -L oradata3
/dev/dm-9
Restart DM-MPIO to let it read the aliases by entering
/etc/init.d/multipathd restart
Verify that the device is recognized by multipathd by entering
multipath -ll
Use a text editor to add a mount entry in the
/etc/fstab
file.
At this point, the alias you created in
Step 5 is not yet in the
/dev/disk/by-label
directory. Add the mount entry
the /dev/dm-9
path, then change the entry before
the next time you reboot to
LABEL=oradata3
Create a directory to use as the mount point, then mount the device by entering
md /oradata3
mount /oradata3
Querying the multipath I/O status outputs the current status of the multipath maps.
The multipath -l option displays the current path status as of the last time that the path checker was run. It does not run the path checker.
The multipath -ll option runs the path checker, updates the path information, then displays the current status information. This option always the displays the latest information about the path status.
At a terminal console prompt, enter
multipath -ll
This displays information for each multipathed device. For example:
3600601607cf30e00184589a37a31d911 [size=127 GB][features="0"][hwhandler="1 emc"]
\_ round-robin 0 [active][first] \_ 1:0:1:2 sdav 66:240 [ready ][active] \_ 0:0:1:2 sdr 65:16 [ready ][active]
\_ round-robin 0 [enabled] \_ 1:0:0:2 sdag 66:0 [ready ][active] \_ 0:0:0:2 sdc 8:32 [ready ][active]
For each device, it shows the device’s ID, size, features, and hardware handlers.
Paths to the device are automatically grouped into priority groups on device discovery. Only one priority group is active at a time. For an active/active configuration, all paths are in the same group. For an active/passive configuration, the passive paths are placed in separate priority groups.
The following information is displayed for each group:
Scheduling policy used to balance I/O within the group, such as round-robin
Whether the group is active, disabled, or enabled
Whether the group is the first (highest priority) group
Paths contained within the group
The following information is displayed for each path:
The physical address as
host:bus:target:lun
, such as 1:0:1:2
Device node name, such as sda
Major:minor numbers
Status of the device
You might need to configure multipathing to queue I/O if all paths fail concurrently by enabling queue_if_no_path. Otherwise, I/O fails immediately if all paths are gone. In certain scenarios, where the driver, the HBA, or the fabric experience spurious errors, DM-MPIO should be configured to queue all I/O where those errors lead to a loss of all paths, and never propagate errors upward.
When you use multipathed devices in a cluster, you might choose to disable queue_if_no_path. This automaticall fails the path instead of queuing the I/O, and escalates the I/O error to cause a failover of the cluster resources.
Because enabling queue_if_no_path leads to I/O being queued indefinitely unless a path is reinstated, make sure that multipathd is running and works for your scenario. Otherwise, I/O might be stalled indefinitely on the affected multipathed device until reboot or until you manually return to failover instead of queuing.
To test the scenario:
In a terminal console, log in as the root
user.
Activate queuing instead of failover for the device I/O by entering:
dmsetup message device_ID
0 queue_if_no_path
Replace the device_ID
with the ID for your
device. For example, enter:
dmsetup message 3600601607cf30e00184589a37a31d911 0 queue_if_no_path
Return to failover for the device I/O by entering:
dmsetup message device_ID
0 fail_if_no_path
This command immediately causes all queued I/O to fail.
Replace the device_ID
with the ID for your
device. For example, enter:
dmsetup message 3600601607cf30e00184589a37a31d911 0 fail_if_no_path
To set up queuing I/O for scenarios where all paths fail:
In a terminal console, log in as the root
user.
Open the /etc/multipath.conf
file in a text
editor.
Uncomment the defaults section and its ending bracket, then add the
default_features
setting, as follows:
defaults { default_features "1 queue_if_no_path" }
After you modify the /etc/multipath.conf
file,
you must run mkinitrd to re-create the
initrd
on your system, then reboot in order for
the changes to take effect.
When you are ready to return over to failover for the device I/O, enter:
dmsetup message mapname
0 fail_if_no_path
Replace the mapname
with the mapped alias
name or the device ID for the device.
This command immediately causes all queued I/O to fail and propagates the error to the calling application.
If all paths fail concurrently and I/O is queued and stalled, do the following:
Enter the following command at a terminal console prompt:
dmsetup message mapname
0 fail_if_no_path
Replace
with the
correct device ID or mapped alias name for the device. This causes all
queued I/O to fail and propagates the error to the calling
application.
mapname
Reactivate queueing by entering the following command at a terminal console prompt:
dmsetup message mapname
0 queue_if_no_path
For more information about configuring and using multipath I/O on SUSE Linux Enterprise Server, see the following additional resources in the Novell Support Knowledgebase:
Troubleshooting SLES Multipathing (MPIO) Problems (Technical Information Document 3231766)
Dynamically Adding Storage for Use with Multipath I/O (Technical Information Document 3000817)
DM MPIO Device Blacklisting Not Honored in multipath.conf (Technical Information Document 3029706)
Static Load Balancing in Device-Mapper Multipathing (DM-MP) (Technical Information Document 3858277)
Troubleshooting SCSI (LUN) Scanning Issues (Technical Information Document 3955167)
If you want to use software RAIDs, create and configure them before you create file systems on the devices. For information, see the following: