Contents
One of the central tasks in computer centers and when operating servers is providing hard disk capacity for server systems. Fibre Channel is often used for this purpose. iSCSI (Internet SCSI) solutions provide a lower-cost alternative to Fibre Channel that can leverage commodity servers and Ethernet networking equipment. Linux iSCSI provides iSCSI initiator and target software for connecting Linux servers to central storage systems.
iSCSI is a storage networking protocol that facilitates data transfers of SCSI packets over TCP/IP networks between block storage devices and servers. iSCSI target software runs on the target server and defines the logical units as iSCSI target devices. iSCSI initiator software runs on different servers and connects to the target devices to make the storage devices available on that server.
![]() | |
It is not supported to run iSCSI target software and iSCSI initiator software on the same server in a production environment. |
The iSCSI target and initiator servers communicate by sending SCSI packets at the IP level in your LAN. When an application running on the initiator server starts an inquiry for an iSCSI target device, the operating system produces the necessary SCSI commands. The SCSI commands are then embedded in IP packets and encrypted as necessary by software that is commonly known as the iSCSI initiator. The packets are transferred across the internal IP network to the corresponding iSCSI remote station, called the iSCSI target.
Many storage solutions provide access over iSCSI, but it is also possible to run a Linux server that provides an iSCSI target. In this case, it is important to set up a Linux server that is optimized for file system services. The iSCSI target accesses block devices in Linux. Therefore, it is possible to use RAID solutions to increase disk space as well as a lot of memory to improve data caching. For more information about RAID, also see Chapter 8, Software RAID Configuration.
YaST includes entries for iSCSI Target and iSCSI Initiator software, but the packages are not installed by default.
![]() | |
It is not supported to run iSCSI target software and iSCSI initiator software on the same server in a production environment. |
Install the iSCSI target software on the server where you want to create iSCSI target devices.
Open YaST, and log in as the root
user.
Select
When you are prompted to install the iscsitarget
package, click .
Follow the on-screen install instructions, and provide the installation media as needed.
When the installation is complete, YaST opens to the iSCSI Target Overview page with the
tab selected.Continue with Section 13.2, “Setting Up an iSCSI Target”.
Install the iSCSI initiator software on each server where you want to access the target devices that you set up on the iSCSI target server.
Open YaST, and log in as the root
user.
Select
When you are prompted to install the open-iscsi
package, click .
Follow the on-screen install instructions, and provide the installation media as needed.
When the installation is complete, YaST opens to the iSCSI Initiator Overview page with the
tab selected.Continue with Section 13.3, “Configuring iSCSI Initiator”.
SUSE Linux Enterprise Server comes with an open source iSCSI target solution that evolved from the Ardis iSCSI target. A basic setup can be done with YaST, but to take full advantage of iSCSI, a manual setup is required.
The iSCSI target configuration exports existing block devices to iSCSI initiators. You must prepare the storage space you want to use in the target devices by setting up unformatted partitions or devices by using the Partitioner in YaST, or by partitioning the devices from the command line.
![]() | |
After you set up a device or partition for use as an iSCSI target, you never access it directly via its local path. Do not specify a mount point for it when you create it. |
Log in as the root
user, then open YaST.
Select
.Click
to continue through the warning about using the Partitioner.Click
to create a partition, but do not format it, and do not mount it.iSCSI targets can use unformatted partitions with Linux, Linux LVM, or Linux RAID file system IDs.
Select
, then click .Specify the amount of space to use, then click
.Select
, then specify the file system ID type.Select
.Click
.Repeat Step 4 for each area that you want to use later as an iSCSI LUN.
Click
to keep your changes, then close YaST.You can use a Xen guest server as the iSCSI target server. You must assign the storage space you want to use for the iSCSI storage devices to the guest virtual machine, then access the space as virtual disks within the guest environment. Each virtual disk can be a physical block device, such as an entire disk, partition, or volume, or it can be a file-backed disk image where the virtual disk is a single image file on a larger physical disk on the Xen host server. For the best performance, create each virtual disk from a physical disk or a partition. After you set up the virtual disks for the guest virtual machine, start the guest server, then configure the new blank virtual disks as iSCSI target devices by following the same process as for a physical server.
File-baked disk images are created on the Xen host server, then assigned to the Xen guest server. By default, Xen stores file-backed disk images in the /var/lib/xen/images/
directory, where vm_name
is the name of the virtual machine.vm_name
For example, if you want to create the disk image /var/lib/xen/images/vm_one/xen-0
with a size of 4 GB, first make sure that the directory is there, then create the image itself.
Log in to the host server as the root
user.
At a terminal console prompt, enter the following commands
mkdir -p /var/lib/xen/images/vm_one dd if=/dev/zero of=/var/lib/xen/images/vm_one/xen-0 seek=1M bs=4096 count=1
Assign the file system image to the guest virtual machine in the Xen configuration file.
Log in as the root
user on the guest server, then use YaST to set up the virtual block device by using the process in Section 13.2.1.1, “Partitioning Devices”.
To configure the iSCSI target, run the Chapter 12, iSNS for Linux.
module in YaST. The configuration is split into three tabs. In the tab, select the start mode and the firewall settings. If you want to access the iSCSI target from a remote machine, select . If an iSNS server should manage the discovery and access control, activate and enter the IP address of your iSNS server. You cannot use hostnames, but must use the IP address. For more about iSNS, readThe
tab provides settings for the iSCSI server. The authentication set here is used for the discovery of services, not for accessing the targets. If you do not want to restrict the access to the discovery, use .If authentication is needed, there are two possibilities to consider. One is that an initiator must prove that it has the permissions to run a discovery on the iSCSI target. This is done with RFC 3720.
. The other is that the iSCSI target must prove to the initiator that it is the expected target. Therefore, the iSCSI target can also provide a username and password. This is done with . Find more information about authentication inThe targets are defined in the
tab. Use to create a new iSCSI target. The first dialog asks for information about the device to export.The
line has a fixed syntax that looks like the following:
iqn.yyyy-mm.<reversed domain name>:unique_id
It always starts with iqn. yyyy-mm is the format of the date when this target is activated. Find more about naming conventions in RFC 3722.
The
is freely selectable. It should follow some scheme to make the whole system more structured.It is possible to assign several LUNs to a target. To do this, select a target in the
tab, then click . Then, add new LUNs to an existing target.Add the path to the block device or file system image to export.
The next menu configures the access restrictions of the target. The configuration is very similar to the configuration of the discovery authentication. In this case, at least an incoming authentication should be setup.
finishes the configuration of the new target, and brings you back to the overview page of the tab. Activate your changes by clicking .
To create a target device:
Open YaST, and log in as the root
user.
Select
YaST opens to the iSCSI Target Overview page with the
tab selected.![]() |
In the
area, select one of the following:If you are using iSNS for target advertising, select the
check box, then type the IP address.If desired, open the firewall ports to allow access to the server from remote computers.
Select the
check box.Specify the network interfaces where you want to open the port by clicking
, selecting the check box next to a network interface to enable it, then clicking to accept the settings.If authentication is required to connect to target devices you set up on this server, select the
tab, deselect to enable authentication, then specify the necessary credentials for incoming and outgoing authentication.The
option is enabled by default. For a more secure configuration, you can specify authentication for incoming, outgoing, or both incoming and outgoing. You can also specify multiple sets of credentials for incoming authentication by adding pairs of usernames and passwords to the list under .![]() |
Configure the iSCSI target devices.
Select the
tab.If you have not already done so, select and delete the example iSCSI target from the list, then confirm the deletion by clicking
.Click
to add a new iSCSI target.The iSCSI target automatically presents an unformatted partition or block device and completes the Target and Identifier fields.
You can accept this, or browse to select a different space.
You can also subdivide the space to create LUNs on the device by clicking
and specifying sectors to allocate to that LUN. If you need additional options for these LUNs, select .Click
Repeat Step 7.c to Step 7.e for each iSCSI target device you want to create.
(Optional) On the
tab, click to export the information about the configured iSCSI targets to a file.This makes it easier to later provide this information to consumers of the resources.
Click
to create the devices, then click to restart the iSCSI software stack.Configure an iSCSI target in /etc/ietd.conf
. All parameters in this file before the first Target declaration are global for the file. Authentication information in this portion has a special meaning—it is not global, but is used for the discovery of the iSCSI target.
If you have access to an iSNS server, you should first configure the file to tell the target about this server. The address of the iSNS server must always be given as an IP address. You cannot specify the DNS name for the server. The configuration for this functionality looks like the following:
iSNSServer 192.168.1.111 iSNSAccessControl no
This configuration makes the iSCSI target register itself with the iSNS
server, which in turn provides the discovery for initiators. For more about iSNS, see Chapter 12, iSNS for Linux. The access control for the iSNS discovery is not supported. Just keep iSNSAccessControl no
.
All direct iSCSI authentication can be done in two directions. The iSCSI target can require the iSCSI initiator to authenticate with the IncomingUser
, which can be added multiple times. The iSCSI initiator can also require the iSCSI target to authenticate. Use OutgoingUser
for this. Both have the same syntax:
IncomingUser <username> <password> OutgoingUser <username> <password>
The authentication is followed by one or more target definitions. For each target, add a Target
section. This section always starts with a Target
identifier followed, by definitions of logical unit numbers:
Target iqn.yyyy-mm.<reversed domain name>[:identifier] Lun 0 Path=/dev/mapper/system-v3 Lun 1 Path=/dev/hda4 Lun 2 Path=/var/lib/xen/images/xen-1,Type=fileio
In the Target
line, yyyy-mm
is the date when this target is activated, and identifier
is freely selectable. Find more about naming conventions in RFC 3722. Three different block devices are exported in this example. The first block device is a logical volume (see also Chapter 4, LVM Configuration), the second is an IDE partition, and the third is an image available in the local file system. All these look like block devices to an iSCSI initiator.
Before activating the iSCSI target, add at least one IncomingUser
after the Lun
definitions. It does the authentication for the use of this target.
To activate all your changes, restart the iscsitarget daemon with rcopen-iscsi restart
. Check your configuration in the /proc
file system:
cat /proc/net/iet/volume tid:1 name:iqn.2006-02.com.example.iserv:systems lun:0 state:0 iotype:fileio path:/dev/mapper/system-v3 lun:1 state:0 iotype:fileio path:/dev/hda4 lun:2 state:0 iotype:fileio path:/var/lib/xen/images/xen-1
There are many more options that control the behavior of the iSCSI target. For more information, see the man page of ietd.conf
.
Active sessions are also displayed in the /proc
file system. For each connected initiator, an extra entry is added to /proc/net/iet/session
:
cat /proc/net/iet/session tid:1 name:iqn.2006-02.com.example.iserv:system-v3 sid:562949957419520 initiator:iqn.2005-11.de.suse:cn=rome.example.com,01.9ff842f5645 cid:0 ip:192.168.178.42 state:active hd:none dd:none sid:281474980708864 initiator:iqn.2006-02.de.suse:01.6f7259c88b70 cid:0 ip:192.168.178.72 state:active hd:none dd:none
When changes to the iSCSI target configuration are necessary, you must always restart the target to activate changes that are done in the configuration file. Unfortunately, all active sessions are interrupted in this process. To maintain an undisturbed operation, the changes should be done in the main configuration file /etc/ietd.conf
, but also made manually to the current configuration with the administration utility ietadm.
To create a new iSCSI target with a LUN, first update your configuration file. The additional entry could be:
Target iqn.2006-02.com.example.iserv:system2 Lun 0 Path=/dev/mapper/system-swap2 IncomingUser joe secret
To set up this configuration manually, proceed as follows:
Create a new target with the command ietadm --op new --tid=2 --params Name=iqn.2006-02.com.example.iserv:system2.
Add a logical unit with ietadm --op new --tid=2 --lun=0 --params Path=/dev/mapper/system-swap2.
Set the username and password combination on this target with ietadm --op new --tid=2 --user --params=IncomingUser=joe,Password=secret.
Check the configuration with cat /proc/net/iet/volume.
It is also possible to delete active connections. First, check all active connections with the command cat /proc/net/iet/session. This might look like:
cat /proc/net/iet/session tid:1 name:iqn.2006-03.com.example.iserv:system sid:281474980708864 initiator:iqn.1996-04.com.example:01.82725735af5 cid:0 ip:192.168.178.72 state:active hd:none dd:none
To delete the session with the session ID 281474980708864, use the command ietadm --op delete --tid=1 --sid=281474980708864 --cid=0. Be aware that this makes the device inaccessible on the client system and processes accessing this device are likely to hang.
ietadm can also be used to change various configuration parameters. Obtain a list of the global variables with ietadm --op show --tid=1 --sid=0. The output looks like:
InitialR2T=Yes ImmediateData=Yes MaxConnections=1 MaxRecvDataSegmentLength=8192 MaxXmitDataSegmentLength=8192 MaxBurstLength=262144 FirstBurstLength=65536 DefaultTime2Wait=2 DefaultTime2Retain=20 MaxOutstandingR2T=1 DataPDUInOrder=Yes DataSequenceInOrder=Yes ErrorRecoveryLevel=0 HeaderDigest=None DataDigest=None OFMarker=No IFMarker=No OFMarkInt=Reject IFMarkInt=Reject
All of these parameters can be easily changed. For example, if you want to change the maximum number of connections to two, use
ietadm --op update --tid=1 --params=MaxConnections=2.
In the file /etc/ietd.conf
, the associated line should look like MaxConnections 2
.
![]() | |
The changes that you make with the ietadm utility are not permanent for the system. These changes are lost at the next reboot if they are not added to the |
There are several more options available for the ietadm utility. Use ietadm -h to find an overview. The abbreviations there are target ID (tid), session ID (sid), and connection ID (cid). They can also be found in /proc/net/iet/session
.
The iSCSI initiator, also called an iSCSI client, can be used to connect to any iSCSI target. This is not restricted to the iSCSI target solution explained in Section 13.2, “Setting Up an iSCSI Target”. The configuration of iSCSI initiator involves two major steps: the discovery of available iSCSI targets and the setup of an iSCSI session. Both can be done with YaST.
The iSCSI Initiator Overview in YaST is divided into three tabs:
Service: The tab can be used to enable the iSCSI initiator at boot time. It also offers to set a unique and an iSNS server to use for the discovery. The default port for iSNS is 3205.
Connected Targets: The tab gives an overview of the currently connected iSCSI targets. Like the tab, it also gives the option to add new targets to the system.
On this page, you can select a target device, then toggle the start-up setting for each iSCSI target device:
Automatic: This option is used for iSCSI targets that are to be connected when the iSCSI service itself starts up. This is the typical configuration.
Onboot: This option is used for iSCSI targets that are to be connected during boot; that is, when root (/
) is on iSCSI. As such, the iSCSI target device will be evaluated from the initrd on server boots.
Discovered Targets: provides the possibility of manually discovering iSCSI targets in the network.
Open YaST, and log in as the root
user.
Select
YaST opens to the iSCSI Initiator Overview page with the
tab selected.![]() |
In the
area, select one of the following:Specify or verify the
.Specify a well-formed iSCSI qualified name (IQN) for the iSCSI initiator on this server. The initiator name must be globally unique on your network. The IQN uses the following general format:
iqn.yyyy-mm.com.mycompany:n1:n2
where n1 and n2 are alphanumeric characters. For example:
iqn.1996-04.de.suse:01:9c83a3e15f64
The /etc/iscsi/initiatorname.iscsi
file on the server.
If the server has iBFT (iSCSI Boot Firmware Table) support, the
is completed with the corresponding value in the IBFT, and you are not able to change the initiator name in this interface. Use the BIOS Setup to modify it instead.The iBFT is a block of information containing various parameters useful to the iSCSI boot process, including iSCSI target and initiator descriptions for the server.Use either of the following methods to discover iSCSI targets on the network.
iSNS: To use iSNS (Internet Storage Name Service) for discovering iSCSI targets, continue with Section 13.3.1.2, “Discovering iSCSI Targets by Using iSNS”.
Discovered Targets: To discover iSCSI target devices manually, continue with Section 13.3.1.3, “Discovering iSCSI Targets Manually”.
Before you can use this option, you must have already installed and configured an iSNS server in your environment. For information, see Chapter 12, iSNS for Linux.
Repeat the following process for each of the iSCSI target servers that you want to access from the server where you are setting up the iSCSI initiator.
In YaST, select
, then select the tab.Click
to open the dialog.Enter the IP address and change the port if needed.
The default port is 3260.
If authentication is required, deselect
, then specify the credentials the or authentication.Click
to start the discovery and connect to the iSCSI target server.If credentials are required, after a successful discovery, use
to activate the target.You are prompted for authentication credentials to use the selected iSCSI target.
Click
to finish the configuration.If everything went well, the target now appears in
.The virtual iSCSI device is now available.
On the iSCSI Initiator Overview page, click
to save and apply your changes.You can find the local device path for the iSCSI target device by using the lsscsi command:
lsscsi [1:0:0:0] disk IET VIRTUAL-DISK 0 /dev/sda
In YaST, select
, then select the tab to view a list of the iSCSI target devices that are currently connected to the server.Select the iSCSI target device that you want to manage.
Click
to modify the setting:Automatic: This option is used for iSCSI targets that are to be connected when the iSCSI service itself starts up. This is the typical configuration.
Onboot: This option is used for iSCSI targets that are to be connected during boot; that is, when root (/
) is on iSCSI. As such, the iSCSI target device will be evaluated from the initrd on server boots.
Click
to save and apply your changes.Both the discovery and the configuration of iSCSI connections require a running iscsid. When running the discovery the first time, the internal database of the iSCSI initiator is created in the directory /var/lib/open-iscsi
.
If your discovery is password protected, provide the authentication information to iscsid. Because the internal database does not exist when doing the first discovery, it cannot be used at this time. Instead, the configuration file /etc/iscsid.conf
must be edited to provide the information. To add your password information for the discovery, add the following lines to the end of /etc/iscsid.conf
:
discovery.sendtargets.auth.authmethod = CHAP discovery.sendtargets.auth.username = <username> discovery.sendtargets.auth.password = <password>
The discovery stores all received values in an internal persistent database. In addition, it displays all detected targets. Run this discovery with the command iscsiadm -m discovery --type=st --portal=<targetip>
. The output should look like:
149.44.171.99:3260,1 iqn.2006-02.com.example.iserv:systems
To discover the available targets on a iSNS
server, use the command iscsiadm --mode discovery --type isns --portal <targetip>
For each target defined on the iSCSI target, one line appears. For more information about the stored data, see Section 13.3.3, “The iSCSI Client Databases”.
The special --login
option of iscsiadm creates all needed devices:
iscsiadm -m node -n iqn.2006-02.com.example.iserv:systems --login
The newly generated devices show up in the output of lsscsi and can now be accessed by mount.
All information that was discovered by the iSCSI initiator is stored in two database files that reside in /var/lib/open-iscsi
. There is one database for the discovery of targets and one for the discovered nodes. When accessing a database, you first must select if you want to get your data from the discovery or from the node database. Do this with the -m discovery
and -m node
parameters of iscsiadm. Using iscsiadm just with one of these parameters gives an overview of the stored records:
iscsiadm -m discovery 149.44.171.99:3260,1 iqn.2006-02.com.example.iserv:systems
The target name in this example is iqn.2006-02.com.example.iserv:systems
. This name is needed for all actions that relate to this special data set. To examine the content of the data record with the ID iqn.2006-02.com.example.iserv:systems
, use the following command:
iscsiadm -m node --targetname iqn.2006-02.com.example.iserv:systems node.name = iqn.2006-02.com.example.iserv:systems node.transport_name = tcp node.tpgt = 1 node.active_conn = 1 node.startup = manual node.session.initial_cmdsn = 0 node.session.reopen_max = 32 node.session.auth.authmethod = CHAP node.session.auth.username = joe node.session.auth.password = ******** node.session.auth.username_in = <empty> node.session.auth.password_in = <empty> node.session.timeo.replacement_timeout = 0 node.session.err_timeo.abort_timeout = 10 node.session.err_timeo.reset_timeout = 30 node.session.iscsi.InitialR2T = No node.session.iscsi.ImmediateData = Yes ....
To edit the value of one of these variables, use the command iscsiadm with the update
operation. For example, if you want iscsid to log in to the iSCSI target when it initializes, set the variable node.startup
to the value automatic
:
iscsiadm -m node -n iqn.2006-02.com.example.iserv:systems --op=update --name=node.startup --value=automatic
Remove obsolete data sets with the delete
operation If the target iqn.2006-02.com.example.iserv:systems
is no longer a valid record, delete this record with the following command:
iscsiadm -m node -n iqn.2006-02.com.example.iserv:systems --op=delete
![]() | |
Use this option with caution because it deletes the record without any additional confirmation prompt. |
To get a list of all discovered targets, run the iscsiadm -m node command.
In SLES 10, you could add the hotplug
option to your device in the /etc/fstab
file to mount iSCSI targets. For example:
/dev/disk/by-uuid-blah /oracle/db ext3 hotplug,rw 0 2
For SLES 11, the hotplug
option no longer works. Use the nofail
option instead. For example:
/dev/sdb1 /mnt/mountpoint ext3 acl,user,nofail 0 0
For information, see TID 7004427: /etc/fstab entry does not mount iSCSI device on boot up .
A firewall might drop packets if it gets to busy. The default for the SUSE Firewall is to drop packets after three minutes. If you find that iSCSI traffic packets are being dropped, you can consider configuring the SUSE Firewall to queue packets instead of dropping them when it gets too busy.
Use the troubleshooting tips in this section when using LVM on iSCSI targets.
When you set up the iSCSI Initiator, make sure to enable discovery at boot time so that udev can discover the iSCSI devices at boot time and set up the devices to be used by LVM.
When using LVM on iSCSI target devices, make sure that openISCSI is loaded at boot time and before LVM in the boot sequence.
In SLES 11, when using iSCSI volumes with LVM, make sure you have configured udev rules for LVM. For information, see TID 7003988: How to Use iSCSI Volumes with LVM on Boot.
In SLES 11 SP1, the LVM rules are found in /lib/udev/rules.d/64-lvm2.rules
.
Remember that udev
provides the default setup for devices in SLES 11. Make sure that all of the applications that create devices have a Runlevel setting to run at boot so that udev can recognize and assign devices for them at system startup. If the application or service is not started until later, udev does not create the device automatically as it would at boot time.
You can check your runlevel settings for LVM2 and iSCSI in
by going to . The following services should be eanbled at boot (B):boot.lvm |
boot.open-iscsi |
open-iscsi |
The iSCSI protocol has been available for several years. There are many reviews and additional documentation comparing iSCSI with SAN solutions, doing performance benchmarks, or just describing hardware solutions. Important pages for more information about open-iscsi are:
There is also some online documentation available. See the man pages for iscsiadm, iscsid, ietd.conf
, and ietd and the example configuration file /etc/iscsid.conf
.