Subject: lstape: fix output with SCSI lin_tape and multiple paths to same tape/changer
From: Steffen Maier <maier@linux.ibm.com>

Description:  lstape, lsluns: handle non-zfcp; lin_tape multiple paths

Symptom:      lstape shows unexpected additional Device suffix numbers in
              excess output columns for each additional path of the same
              tape/changer driven by the IBM lin_tape device driver
              (independent of actual path failover enablement in lin_tape).
              It also shows a wrong number of found devices in the header
              (without --scsi-only).

              lstape prints error about "Unexpected extra argument:" and the
              usage for "sg_inq" along with wrong tabular output.

              lsluns prints ENOENT error text for "cat" on SCSI device sysfs
              attributes hba_id, wwpn, and fcp_lun.

              lstape with --verbose option prints ENOENT error text for
              "cat" on SCSI device sysfs attributes hba_id and wwpn.

              lstape man page: Description of --type and <devbusid> filter
              for channel tapes is incomplete. SCSI output description is
              incomplete.

              lstape shows "N/A" instead of the HBA device bus-ID with
              virtio-scsi-ccw.

Problem:      s390-tools-1.8.0 before the first upstream commit b627b8d8e1ab
              ("Initial s390-tools-2.0.0 import") introduced SCSI
              tape/changer output for lstape. It used the SCSI device serial
              number as lookup key to find a match in IBM lin_tape device
              driver proc-fs output for a given SCSI device name. Multiple
              paths to the same tape/changer have the same serial number.
              Multiple matches cause excess arguments to printf. Explaining
              the resulting output, the bash man page says: "The format is
              reused as necessary to consume all of the arguments." This
              also causes a wrong number of found devices.

              The default bash settings have nullglob disabled so if
              $SCSI_DEV/scsi_generic* aka
              /sys/bus/scsi/devices/*:*:*:*/scsi_generic* does not match
              anything, it leaves the glob pattern unmodified and
              SG_DEV=$(basename $SG_DEV/*) results in the literal "*". If
              $SG_INQ exists, it invoked sg_inq with more than the one
              allowed positional argument for a SCSI generic device node
              "sg_inq /dev/*". Causing error messages and the usage of
              sg_inq to land in $TAPE_SERIAL.

              lsluns iterates SCSI generic devices and unconditionally
              reads zfcp-specific SCSI device sysfs attributes hba_id, wwpn,
              and fcp_lun.

              lstape --verbose unconditionally reads zfcp-specific SCSI
              device sysfs attributes hba_id and wwpn.

              <devbusid> filter missing from synopsis. <device-type> example
              at wrong place with <devbusid> filter. <devbusid> filter
              option description is a duplicate of <device-type> filter
              option description. SCSI output description misses fields.

              Lstape only used the zfcp-specific sysfs attribute hba_id.

Solution:     Prefer sysfs to find lin_tape device name for SCSI device.
              Fallback: The lin_tape proc-fs output format has changed over
              the years. The HBA device driver string can contain whitespace
              (e.g. "Virtio SCSI HBA") and breaks the field numbers with
              tokenized parsing. Grep for the SCSI device name as word (to
              skip names with same substring, such as 0:0:1:1 also matching
              0:0:1:10) and cut the first field 'Number' (lin_tape device
              name suffix). If there is no SCSI column at all [lin_tape
              before v2.2.0] (and no SCSI LLDD or other column with a name
              accidentally matching an existing SCSI device name), we get no
              match and better bail out with the initialized "N/A" for the
              lstape column "Device".

              To not have to rely on the nullglob setting, explicitly check
              for the existence of $SCSI_DEV/scsi_generic before evaluating
              SG_DEV=$(basename $SG_DEV/*). Also handle availability of
              sg_inq but absence of scsi_generic individually to provide the
              user with a hint if only sg is missing.

              Simply skip non-zfcp SCSI devices, such as iSCSI or
              virtio-scsi-ccw, to not erroneously access absent attributes.

              Assume "N/A" for HBA and WWPN of non-zfcp SCSI devices, such
              as iSCSI or virtio-scsi-ccw, to not erroneously access absent
              zfcp-specific sysfs attributes.

              Add <devbusid> filter to synopsis. Move <device-type> example
              to <device-type> option. Replace <devbusid> filter option
              description. Move existing SCSI output description to a new
              subsection and add description of missing fields.

              Also search sysfs for an ancestor with subsystem ccw.

Reproduction: Attach more than one path to the same SCSI tape or changer and
              load the IBM lin_tape device driver.

              Unload sg kernel module.

              Attach non-zfcp SCSI devices such as iSCSI or virtio-scsi-ccw.

              Attach non-zfcp SCSI devices such as iSCSI or virtio-scsi-ccw.

              man lstape

              Attach SCSI tape or changer with virtio-scsi-ccw to a KVM
              guest and run "lstape --verbose".

Upstream-ID:  ef4dc7a45b1b80c58076a0a317245569d84b2754
Problem-ID:   170633

Upstream-Description:

              lstape: fix output with SCSI lin_tape and multiple paths to same tape/changer

              https://www.ibm.com/developerworks/linux/linux390/s390-tools-1.8.0.html
              before the first upstream commit b627b8d8e1ab ("Initial s390-tools-2.0.0
              import") introduced SCSI tape/changer output for lstape. It used the SCSI
              device serial number as lookup key to find a match in IBM lin_tape device
              driver proc-fs output for a given SCSI device name. Since multiple paths
              to the same tape/changer have the same serial number, multiple matches can
              occur and distort the output:

              % lstape
              ...
              SCSI tape devices (found 12):
              ...
              Generic Device        Target       Vendor   Model            Type     State
              sg0     IBMtape0      3            0:0:0:0  IBM              ULT3580-TD6 tapedrv
              running
              sg1     IBMchanger0   1            2        3                0:0:0:1  IBM
              3573-TL changer       running
              ...

              There are unexpected additional Device suffix numbers for each additional
              path of the same tape/changer (independent of actual path failover
              enablement in lin_tape). The multiple matches above cause excess arguments
              to printf. Explaining the resulting output, the bash man page says:
              "The format is reused as necessary to consume all of the arguments."
              This also causes a wrong number of found devices.

              Current lin_tape (v3.0.31) uses the printf format string
              "%-8d%-12s%-25s%-16s%-16s%-10s" for fixed column proc-fs output.
              This causes the following example input to lstape:

              ==> /proc/scsi/IBMchanger <==
              lin_tape version: 3.0.31
              lin_tape major number: 251
              Attached Changer Devices:
              Number  model       SN                       HBA             SCSI            FO Path
              0       3573-TL     00L4U78W6497_LL0         Virtio SCSI HBA 0:0:0:1         Primary
              1       3573-TL     00L4U78W6497_LL0         Virtio SCSI HBA 0:0:1:1         Alternate
              2       3573-TL     00L4U78W6497_LL0         Virtio SCSI HBA 1:0:1:1         Alternate
              3       3573-TL     00L4U78W6497_LL0         Virtio SCSI HBA 1:0:0:1         Alternate

              ==> /proc/scsi/IBMtape <==
              lin_tape version: 3.0.31
              lin_tape major number: 251
              Attached Tape Devices:
              Number  model       SN                       HBA             SCSI            FO Path
              0       ULT3580-TD6 10WT037733               Virtio SCSI HBA 0:0:0:0         Primary
              1       ULT3580-TD6 10WT037701               Virtio SCSI HBA 0:0:1:0         Primary
              2       ULT3580-TD6 10WT037701               Virtio SCSI HBA 1:0:1:0         Alternate
              3       ULT3580-TD6 10WT037733               Virtio SCSI HBA 1:0:0:0         Alternate

              If that was reliable and stable, we could use fixed column parsing:
              			IBM_IDX=$(
              				awk 'BEGIN {
              					FIELDWIDTHS="8 12 25 16 16 10"
              				}
              				gensub(" ","","g",$5) == "'$SCSI_ID'"{
              					print gensub(" ","","g",$1)
              				}' /proc/scsi/$DEV_NAME
              			)
              However, we can hardly guarantee a field value always fits into the left-
              adjusted columns (printf won't cap to field width!). Also, the fixed column
              format changed between lin_tape v3.0.20 and v3.0.18 from
              "%-8d%-12s%-18s%-16s%-16s%-10s" with less width for the 'SN' field.
              The column format also changed between v2.2.0 and v1.37.0 from
              "%-8d%-12s%-18s%-32s%-10s" where the 'SCSI' field did not even exist.
              Likely that was the reason for the original Serial lookup in lstape.

              As can be seen in the example, at least the HBA device driver string can
              contain whitespace and breaks the field numbers with tokenized parsing.
              We want to match column 5 (SCSI) but above we would match against the
              string 'SCSI' as part of "Virtio SCSI HBA" instead of e.g. '0:0:0:0'.

              The following works currently knowing that the last two columns do not
              contain white space. But it can also fail if the value of field
              'HBA' does not fit into the left-adjusted columns so there is no white
              space between columns 'HBA' and 'SCSI' making them collapse:
              awk '$(NF-1) == "'$SCSI_ID'" { print $1 }' /proc/scsi/$DEV_NAME

              Grep for the SCSI device name as word (to skip names with same substring,
              such as 0:0:1:1 also matching 0:0:1:10) and cut the first field 'Number'.
              If there is no SCSI column at all (and no SCSI LLDD or other column with a
              name accidentally matching an existing SCSI device name), we get no match
              and better bail out with the initialized "N/A" for the lstape column
              "Device". If the SCSI column exists, the SCSI device names are unique and
              we always get at most one match.

              Signed-off-by: Steffen Maier <maier@linux.ibm.com>
              Reviewed-by: Hendrik Brueckner <brueckner@linux.ibm.com>
              Reviewed-by: Jens Remus <jremus@linux.ibm.com>
              Signed-off-by: Jan Höppner <hoeppner@linux.ibm.com>


Signed-off-by: Steffen Maier <maier@linux.ibm.com>
---
 zconf/lstape   |   11 ++++-------
 zconf/lstape.8 |    7 ++-----
 2 files changed, 6 insertions(+), 12 deletions(-)

--- a/zconf/lstape
+++ b/zconf/lstape
@@ -2,7 +2,7 @@
 #
 # lstape - Tool to show information about tape devices
 #
-# Copyright IBM Corp. 2003, 2017
+# Copyright IBM Corp. 2003, 2018
 #
 # s390-tools is free software; you can redistribute it and/or modify
 # it under the terms of the MIT license. See LICENSE for details.
@@ -55,7 +55,7 @@ function PrintVersion()
 {
 	cat <<-EOD
 	$CMD: version %S390_TOOLS_VERSION%
-	Copyright IBM Corp. 2003, 2017
+	Copyright IBM Corp. 2003, 2018
 	EOD
 }
 
@@ -292,16 +292,13 @@ function SysfsCreateListSCSI()
 				TAPE_DEV=$CHG_IDX
 			fi
 		elif [ -r /proc/scsi/$DEV_NAME ]; then
-			if [ "$TAPE_SERIAL" != "NO/INQ" ]; then
 				IBM_IDX=$(
-					awk '$3 == "'$TAPE_SERIAL'"{
-						print $1
-					}' /proc/scsi/$DEV_NAME
+					grep -wF "$SCSI_ID" /proc/scsi/$DEV_NAME |
+					cut -d ' ' -f 1
 				)
 				if [ "$IBM_IDX" != "" ]; then
 					TAPE_DEV=$DEV_NAME$IBM_IDX
 				fi
-			fi
 		fi
 
 		printf "$SCSIFORMAT" \
--- a/zconf/lstape.8
+++ b/zconf/lstape.8
@@ -1,8 +1,8 @@
-.\" Copyright 2017 IBM Corp.
+.\" Copyright 2017, 2018 IBM Corp.
 .\" s390-tools is free software; you can redistribute it and/or modify
 .\" it under the terms of the MIT license. See LICENSE for details.
 .\"
-.TH LSTAPE 8 "Jul 2007" "s390-tools"
+.TH LSTAPE 8 "Jun 2018" "s390-tools"
 
 .SH NAME
 lstape \- list tape devices.
@@ -34,9 +34,6 @@ lstape command tries to find out which o
 and changer driver the device names start with "st" or "sch", while for the
 IBM tape driver this would be "IBMtape" or "IBMchanger". If "N/A" is shown,
 the correct driver could not be obtained.
-This happens for example if there is no sg_inq command installed which is
-required to read the drive's serial number which in turn is used to find out
-the device number of the IBM tape driver.
 
 The serial number of a SCSI tape can be displayed with the --verbose option. If
 there is no sg_inq command available "NO/INQ" is shown as the tape's serial.
