FSC 3200 on MSL

[Dval]

Hi all,

I'm not used to posting here, but i'll try to describe my issue as much as I can.

I have the following configuration:
MSL5052 (master) - 2 SDLT 320 drives & e1200

MSL5026 (Slave 0) - 2 SDLT 320 drives & e1200

MSL5026 (Slave 1) - 2 SDLT 320 drives & e1200

MSL6052 (Slave 2) - no drives (library controller piggy backed on Slave 2-e1200)

I have the passthrough connecting them all.

The libraries are on FW 5.16 and e1200 are 5.6.87 (not the latest, but previous)

I think I need a young priest and an old one.

The hosts that are connected to this config through fiber are win 2000 (2) and win 2003 (6)

A mix of HP rebranded emulex and qlogic HBAs.

The issue is that after the library has been backing up for "awhile" (2-9 hours) it produces FSC 3200 and can't unload tapes. Once, I reboot the library through the web interface (or manually), it's good for X period of time. I'm using Commvault Galaxy 6.1 (SP2). Honestly I don't think the software has anything to do with it.

I've done the standard things to the config(hosts):
-Disabled RSM
-Disabled HP Fiber array mgmt
-On the emulex HBAs set the driver to ResetTPRLO to 1
-Disabled (on the qlogics) reset target
-On W2K3 hosts, I added the registry Autorun to 0 for the hpqdlt service (forget the KB article at this point)
-Persistently bound devices (e1200s) on the respective HBAs
-On the e1200s mapped the proper devices, controller first, drives, then AF (E1200 controller)
-Devices are properly seen on the Hosts OSes
-One of the W2K hosts uses the same HBA to connect to an MSA 1000, (which isn't recommended), however I disabled the port on the switch to test, and I still got the 3200 error.

The fiber switches are HP rebranded silkworms. All ports are forced to 2GB.

HP keeps sending onsite techs (case# 3215517037 & others), which is nice, but I think there's some setting somewhere (HBAs/e1200) causing the issue. The picker was changed multiple times, worked "ok". Then the master library wasn't reading labels properly; I have 6 char labels (label setting OK), it was adding * (characters) only on the master library to the begining and end of the label (not good the the software). The reader was changed and now i've got the 3200 errors.

I'm starting to run out of ideas and ...Any fresh ideas are welcome...

Thanks.

dval

[Torsten.]

The errors means "shuttle jammed".

Is the error related to only one of the boxes?

[Dval]

The error appears in the master "trace" log, however any of the drive can experience this unload error (in the software) the result is the FSC.

Yes, 3200 is shuttle jammed.

[Torsten.]

The message should appear in the front panel too. If this occurs on the same box everytime, I would suspect the shuttle there.
If it occurs on different boxes, probably the elevator is the culprit.

[Dval]

The message should appear in the front panel too. If this occurs on the same box everytime, I would suspect the shuttle there.
If it occurs on different boxes, probably the elevator is the culprit.


Hi Torsten,

Yes that's the issue, the error occurs on the master everytime. The shuttle/picker has been replaced 3 times. When is comes back up, the calibration is done/tested. Then the cartridge cycle is done, which tests the elevator, drives, pickers, etc. We can leave this test run for 5 mins or 12 hours, and no errors will appear. But when the fiber ports are enabled and jobs start, this is when the error is experienced. I really don't think it's a hardware (library) issue. I think something is resetting the bus/etc and cacking out the jobs. I'm trying to find out what, but honestly I think it's a 1 or 0 in the registry or driver somewhere causing this issue. I just can't figure out where.

Thanks for your help.

Darren