MSL5030 Load/Unload Problems on SAN

[Derek_31]

I have a new MSL5030L1 tape library connected to my SAN, with about 9 servers running Windows Server 2003 Enterprise.

I am running CommVault Galaxy, all with media agents, so all of my servers see the 2 LTO tape drives for backups via the SAN.

I'm having a problem running the Library and Tape tools program. I can run individual LTO drive tests and they pass, but library exercising always fails.

During the first tape drive unload pass I get a soft error 200F, removal prevented. In the test log I get an I/O exception move media.

The consistent load/unload problems are causing Galaxy all kinds of fits.

The servers have PSP 7.00, latest HP tape driver bundle, and I have zoning in effect (storage and backup zones).

Can anyone help me out here?

[Derek_31]

Forgot to mention that the library and drives all have the latest firmware versions (MSL 4.21 and LTO drive v38).

[David Ruska]

> I'm having a problem running the Library and Tape tools program. I can run individual LTO drive tests and they pass, but library exercising always fails.

Can you post the EventLog.ltt (from the logs folder of LTT)? If the failure happened the last time you ran LTT, just post that file. If it happened on a previous run, zip up all the EventLog files and send them to ltt_team@hp.com.

> During the first tape drive unload pass I get a soft error 200F, removal prevented. In the test log I get an I/O exception move media.

Was you backup application shut down? I wonder if it is leaving the drives with media removal prevented.

> The consistent load/unload problems are causing Galaxy all kinds of fits.

Are the galaxy problems happening independent of LTT?

If so, do you have compaq insight manager 6.XX running on any of the servers, or are they all up to 7.00 with PSP 7.00?

Which routers are you running with, and what level of firmware?

[Derek_31]

I shut down all of the backup services, so those shouldn't have been interfering. The tape mount/dismount problem also happens while the backup jobs are running.

Both Galaxy and LTT have mount/dismount problems.

All servers have PSP 7.0. I'm using the e1200 storage router in the MSL, and it's at 0319 firmware (going from memory here, but it's the latest I saw).

I should also mention that this is the 2nd MSL library on this SAN that has had this problem. The last MSL died because of a bad touch screen.

I'll ship off the logs tomorrow after I get into work.

[David Ruska · Accepted solution]

> I'm using the e1200 storage router in the MSL, and it's at 0319 firmware (going from memory here, but it's the latest I saw).

The NSR firmware numbering uses a 6 digit numbering scheme, but unfortunately only 4 digits are viewable in the standard SCSI inquiry.

I think the 0319 you're referring to is 4.03.19. Since then, 5.01.04 and 5.3.0b have been released. Right now, the 5.3.0b is only available for the serial/telnet/ftp method of updating (not LTT).

Here's the image:
ftp://ftp.hp.com/pub/softlib/software2/COL3566/co-12621-3/5.3.0b_fw_winston_x.dlx

Have you worked with the HP call centers to get a resolution to the unload problems?

There was a known issue in CIM 6.40 that could cause robotic hangs. Are you still having failures while running CIM 7.00 (which I understand just released)?

[Curtis Ballard]

The soft error 200F for an MSL tape library will only be reported when some application that has access to a drive on the library has issued a "Prevent Media Removal" command. There is nothing the library can do in this case when an unload request is received.

The most common cause of that is a computer on the SAN that has visibility to the tape drive running the Microsoft removable storage manager (RSM). Note that the computer does not need to be one of your backup servers. If the drives are visible to any windows computers running RSM then the RSM software will claim and lock the drive.

The other possibility might be that Galaxy is issuing the reserve. I have never seen a reserve issued by version 4.1 but a newer version migh. You don't say what version of CommVault Galaxy you are using.

There are a lot of excellent logs stored by Galaxy that could be checked if you can't resolve the problem but if you are getting an error code 200F reported by Galaxy then you won't be able to learn much more from the logs. If only L&TT is reporting the 200F then checking the Galaxy logs would be a good idea.

[Derek_31]

I have RSM disabled via group policy for all servers on the network. I did the MSR firmware update to the 5.3.0b up from 4.03.19.

Due to the previous MSL library dying on me, I had to basically re-install all of the Galaxy 4.2 agents. I'm up to 5 out of my 9 computers. I ran two test backups, and so far I haven't had load/unload problems.

I'll finish up the rest of the installs today and see how it goes.

[Derek_31]

I ran several backups across all machines and numerous library verification tests and all mounts and dismounts work fine. So I think whatever the problem was is now gone.

[Curtis Ballard]

Hope it is gone for good. If you have any further problems the LmsLibraryXX_XX_X.log file should help.