StoreEver Tape Storage
1748178 Members
4215 Online
108758 Solutions
New Discussion юеВ

San tape drives disappearing

 
RMC_2
Advisor

San tape drives disappearing

Hi guys,

a client of mine has the following problem:

randomly some san tape drives just disappear, that problem occurs only in a couple of linux machines (2.6.9-78.0.5.ELsmp), unfortunately, I have no access to the machines right now, and my client just sent some info.

The relevant info AFAIK is in the messages log:

------------------------------------- cut ---------------------------------------
Jun 22 09:53:38 S1S-CAEF-BD1 kernel: scsi0 (0,0,1) : reservation conflict
Jun 22 09:53:38 S1S-CAEF-BD1 kernel: st1: Error 18 (sugg. bt 0x0, driver bt 0x0, host bt 0x0).
Jun 22 10:09:14 S1S-CAEF-BD1 kernel: scsi0 (0,0,1) : reservation conflict
Jun 22 10:09:14 S1S-CAEF-BD1 kernel: st1: Error 18 (sugg. bt 0x0, driver bt 0x0, host bt 0x0).
------------------------------------- cut ---------------------------------------
Jun 23 12:59:05 S1S-CAEF-BD1 kernel: scsi0 (0,0,1) : reservation conflict
Jun 23 12:59:05 S1S-CAEF-BD1 kernel: st1: Error 18 (sugg. bt 0x0, driver bt 0x0, host bt 0x0).
Jun 23 12:59:05 S1S-CAEF-BD1 kernel: scsi1 (0,0,1) : reservation conflict
Jun 23 12:59:05 S1S-CAEF-BD1 kernel: st4: Error 18 (sugg. bt 0x0, driver bt 0x0, host bt 0x0).
------------------------------------- cut ---------------------------------------

The problem occurs just when another backup is running, if not, the situation looks normal (it must be 8 tapes), here the cat /proc/scsi/scsi shows:
Attached devices:
Host: scsi0 Channel: 00 Id: 00 Lun: 00
Vendor: HP Model: Ultrium 3-SCSI Rev: G65W
Type: Sequential-Access ANSI SCSI revision: 03
Host: scsi0 Channel: 00 Id: 00 Lun: 01
Vendor: HP Model: Ultrium 3-SCSI Rev: G65W
Type: Sequential-Access ANSI SCSI revision: 03
Host: scsi0 Channel: 00 Id: 05 Lun: 00
Vendor: HP Model: Ultrium 3-SCSI Rev: G65W
Type: Sequential-Access ANSI SCSI revision: 03
Host: scsi0 Channel: 00 Id: 05 Lun: 01
Vendor: HP Model: Ultrium 3-SCSI Rev: G65W
Type: Sequential-Access ANSI SCSI revision: 03
Host: scsi1 Channel: 00 Id: 00 Lun: 01
Vendor: HP Model: Ultrium 3-SCSI Rev: G65W
Type: Sequential-Access ANSI SCSI revision: 03
Host: scsi1 Channel: 00 Id: 00 Lun: 02
Vendor: HP Model: Ultrium 3-SCSI Rev: G65W
Type: Sequential-Access ANSI SCSI revision: 03
Host: scsi1 Channel: 00 Id: 06 Lun: 01
Vendor: HP Model: Ultrium 3-SCSI Rev: G65W
Type: Sequential-Access ANSI SCSI revision: 03
Host: scsi1 Channel: 00 Id: 06 Lun: 02
Vendor: HP Model: Ultrium 3-SCSI Rev: G65W
Type: Sequential-Access ANSI SCSI revision: 03

A test:

mt -f /dev/st1 status
SCSI 2 tape drive:
File number=-1, block number=-1, partition=0.
Tape block size 0 bytes. Density code 0x0 (default).
Soft error count since last status=0
General status bits on (50000):
DR_OPEN IM_REP_EN


Addtional info:

Data Protector version 6.0, CM,IS and clients

Licences in Data Protector:


Category Number of Licenses
Cell Manager for all platforms 2
Cell Manager for Windows / Linux 0
Tape drive for SAN / all platforms 8
Direct attached tape drive for Windows / NetWare / Linux 0
Multi-Drive Server for UNIX 0
Multi-Drive Server for Windows / NetWare 0
On-line Extension for UNIX 0
On-line Extension for Windows 0
Manager-of-Managers Extension for all platforms 0
Manager-of-Managers Extension for Windows / Linux 0
61-250 Slot Libraries Extension for UNIX 0
61-250 Slot Libraries Extension for Windows 0
Unlimited Slot Libraries Extension for UNIX 0
Unlimited Slot Libraries Extension for Windows 0
EMC Split Mirror Extension 0
HP XP Split Mirror Extension 0
Single Server Edition for all platforms 0
Single Server Edition for Windows / Linux 0
On-line Extension for ONE UNIX system 7
On-line Extension for ONE Windows / Linux system 0
Extension for ONE 61-250 Slot Library 0
Extension for ONE 61-250 Slot Library for Windows 0
Extension for ONE Unlimited Slot Library 0
Extension for ONE Unlimited Slot Library for Windows 0
Zero Downtime Backup Extension for ONE EMC Symmetrix 0
Zero Downtime Backup Extension for ONE HP StorageWorks XP 0
Extension for ONE NDMP Server 0
Zero Downtime Backup for 1 TB EMC Symmetrix / DMX 0
Zero Downtime Backup for 1 TB HP StorageWorks XP 0
Zero Downtime Backup for 1 TB HP StorageWorks EVA/VA 0
Instant Recovery for 1 TB HP StorageWorks XP 0
Instant Recovery for 1 TB HP StorageWorks EVA/VA 0
Direct Backup for 1 TB HP StorageWorks XP or compatible 0
Direct Backup using NDMP for 1 TB 0
Direct Backup for 1 TB HP StorageWorks VA 0
Zero Downtime Backup for 1 TB HP StorageWorks Modular SAN Array 1000 0
Instant Recovery for 1 TB HP StorageWorks Modular SAN Array 1000 0
Advanced backup to disk for 1 TB 0

Switches "licenseshow"
licenseshow:
/fabos/cliexec/licenseshow :
bQ9Qcz999bcRARdU:
Unknown1 license
cSbRdSRcQddTcSe5:
Unknown2 license
zddyRzbQS0eezSp:
Fabric license
b9yy9dzze9cQzzAN:
First Ports on Demand license - additional 8 port upgrade license
SycRbbyzSRTzdSce:
Extended Fabric license
b9yy9dzze9cA3zAB:
Second Ports on Demand license - additional 8 port upgrade license

My question is, Could someone point out where to start searching for a solution?
6 REPLIES 6
TTr
Honored Contributor

Re: San tape drives disappearing

>The problem occurs just when another backup is running...

What does the environment look like? Where is the backup running from? Another server (the DP server)? And why do these Linux boxes have access to the same tape drives? Are they media servers?
RMC_2
Advisor

Re: San tape drives disappearing

Thnx for your reply TTr,

Environment: Extended SAN, 2 fabrics, 8 Tape drive for fabric, each machine connected to the SAN must see 8 drives (4 from fabric 1 and 4 from fabric 2)

The backup is running from the CM (a 2 nodes active/passive cluster in fact)

They have a lot of machines without connection to the SAN, so they use the ones connected as media servers.

Greetings,

R.
TTr
Honored Contributor

Re: San tape drives disappearing

I have not used DP since it was called OBII3.5 but I use netbackup a lot. I assume that the Linux machines that have the scsi reservation problem are DP media servers. If so, check if there is a setting in DP to enable the scsi reserve. Also check if they are properly licensed for the tape drives.
If they Linux servers are NOT part of the DP environment and you share the drives with them, I think then that this is normal.
Another thing to check is OS patches for those two servers.
Steven Clementi
Honored Contributor

Re: San tape drives disappearing

How is the zoning configured on the san switches?

Does each server that can see the tape drive have it's own zone?

assuming there are 3 servers, you should than have at least 3 zones..

Zone1: Server1, Tape1-Tape8
Zone2: Server2, Tape1-Tape8
Zone3: Server3, Tape1-Tape8

What does it look like?


Steven
Steven Clementi
HP Master ASE, Storage, Servers, and Clustering
MCSE (NT 4.0, W2K, W2K3)
VCP (ESX2, Vi3, vSphere4, vSphere5, vSphere 6.x)
RHCE
NPP3 (Nutanix Platform Professional)
RMC_2
Advisor

Re: San tape drives disappearing

Steve, thnx for ur reply, S1P_CAEF_BD1 is one of the servers, an extract of the cfg:

/fabos/cliexec/cfgshow :
Defined configuration:

---------------------cut-----------------------------
zone: BCK_S1P_CAEF_BD1
S1P_CAEF_BD1; NSR_UP_S1; NSR_UP_S2
---------------------cut-----------------------------

Effective configuration:
---------------------cut-----------------------------
zone: BCK_S1P_CAEF_FS1
10,21
10,12
11,5
---------------------cut-----------------------------
TTr
Honored Contributor

Re: San tape drives disappearing

> 10,21
> 10,12
> 11,5

It would be nice if you provided some explanation here. The effective config is what count but all we see is that 3 ports, ports 21 and 12 in switch10 and port 5 in switch5 are zoned together. What are these ports for? You have 2 servers with problems, more media servers and 8 drives that are connected to the fiber switches. How are they zoned?