Disabling Secure Manager Feature because a management path could not be found for the following devi

[MO_4]

I have HP ESL322 Tape Library with 18 LTO5 drives. In use since November 2010.

Interface Manager Firmware Rev = I270
Library Firmware Rev = 7.60.10
LTO5 Drives Firmware = I3BW
HP NS E2400FC-4G firmware = 5963
CVTL v2.7.00 build 82

Backup Software = HP Data Protector v6.11 with latest patches running on HP-UX 11.23.

SAN switches brocade.

Been getting this message on ramdom drives:
"Disabling Secure Manager Feature because a management path could not be found for the following devices"

In order to get drive working again we need to move stuck tape via CVTL, reboot drive via CVTL.

Been working with HP L1/L2, field engineer and just recently escalated to L3 for over 2.5 months but still don't have fix. Been supplying s-tickets, supportshows, 2 tape drives replaced, cabinet control board replaced to no avail.

Has anyone had this problem on their EML or ESL libraries?

Need to know if this problems is library, backup software or SAN swith/port related?

Thanks,
MO.

[Marino Meloni_1 · Accepted solution]

The Secure manager is the application running in the management side of the library that is demanded to map the devices to the Hosts.
another part of the management is the IM (Interface Manager) that should control all the various components of the library and report any issue to the operator via the CVTL running remotely on the LAN in another PC.
The IM is supervising all the components and get informed if something is not working, like a fan or a robot movement for example.
The IM also check constantly that each active component is present and available, this is done throught the management internal private LAN.
In your case,
every 15 minutes the IM try to ping the IFC (Interface Controller) and check it it get a response.
If during this burt of ping, the IM do not get any response from the IFC, it assume the path to that component is momentary broken.
That mean there is no management path to this component.
That mean the secure manager cannot modify any configuration (map) as there is actually no way to update any change in teh IFC map.

This is the reson why you are getting such an error, this indicate you cannot actually use secure manager to change anything.

Now, this situation usually recover by itself, this because the next check from IM to IFC, ususally goes right, and the situation is correctly recovered when the IM see again the IFC.

One of the main reason of this random communicatin failure (ping not answered by one IFC) is due to the way the IFC code is build. The IFC always give priority to the SAN data flow over the management, so if the IFC is highly loaded with routing data packets to the Drive, he will delay the answer to any command with lower priority, and then the management answers will be delayed, ending up in time out in the IM, and ending up with the message that the management path is not available, and ending up with the Secure Manager on hold.

You do not have to worry about this until it do not occurs when no backups are running, or if it is a permanent error.

One of the way you can reduce the traffic in the specific IFC is using bigger block size if they are not already at 256K or creating backup task that use drives connected to different IFC when running concurent backups, for example using two drives on IFC 1 and two drives on IFC2.
These kind of work around may reduce the frequence of these messages that are only cosmetics at the moment for what I can understand in your description

[Curtis Ballard]

The information posted above is useful for LTO-3 and older drives but not for LTO-4 and LTO-5 as there is no IFC and the communication from the interface manager card in the ESL-e to the drive is over an Ethernet link.

Is there any particular time when this happens? Randomly during operation? During power-up? While idle?

You mention HP-UX 11.23 hosts running DP 6.11 - how many hosts are mapped to the drive?

If this is random I suspect a software issue somewhere in the Interface manager hardware and Drive Ethernet communication link. There must be some characteristic of the configuration or event though that is causing your configuration to run into that issue as it isn't a common problem with the LTO-5 drives.

[Torsten.]

If I'm not wrong, there are still interface controllers installed, but only one is used to allow access to the robotics (hope i don't mix up with eml here).
The drives are direct connected to a switch via FC, but communicate with the library via an internal ethernet network. So the interface controllers should be out of question here, right?

[Curtis Ballard]

Torsten is correct - there is one interface controller for library connectivity on both the EML-e and ESL-e with LTO-5 but that interface controller is not used for drive communications so would not be involved in this issue.

[Marino Meloni_1]

oh, sure, Curtis, I didn't noticed Lto5 where involved, I just saw the FW of the components and assumed the drives where connected to the NSR.
in this case it would be interesting to review the IM log, where you may probably see what is the component where the path is broken, and then troubleshoot that component connection (usually the LAN cable). If the problem occurs randomly on different LTO drives, it could be you have a lot of Hosts zoned to that drive and sending polling or similar unsollecited scsi commands, that cause the drives to be overloaded routing backup data and responding those polling scsi commands

[MO_4]

Thank you all for your responses.

@ Curtis: The drives lose connection always during backups. Drive(s) could be engaged during backup for many hours and then we get error on backup that coincides with CVTL "Disabling Secure Manager" All 18 LTO5 drives are mapped to Data Protector Cell Manager (HP-UX) and all drives are mapped with 1-2 in some cases 3 Linux hosts. I do not have Windows hosts in my SAN. Is it random? Yes. It's mostly during my FULL backups on the weekend but it can happen to drive 1 this week and next week drive 15.

I don't want to rule out HP Data protector or rogue polling from host on SAN and that's why I wanted to know if other HP ESL customers were having same or similar issues. I have collected s-tickets on offending drive, IM, IFC, picker plus SAN Switch supportsaves after these occurrences but HP has not been able to come up with a solution yet.

Once again thanks for your responses.
Much appreciated.

MO.

[Curtis Ballard]

Thanks for the additional information. Very interesting that there aren't a lot of hosts and that you are getting errors in the backup application as well.

Do you have a case number?

[MO_4]

CASE # 4627510489

[Colette Howe 1]

We definately need support tickets to debug this issue.
After the daughter board was replaced there have been no issues. The ESL-e files showed that the CCB could not write to the data base. Need to replace daughter card first and possibly the CCB if it did not fix. Drives have been going off line randomly.