EVA VSS hardware provider - slow/delayed startup

[Steven Fletcher_1]

Hi all,

We are attempting to use the hp EVA VSS hardware provider with Netbackup and MS SQL server to perform off-host snapshot backups.

We are having transient problems with snapshot timeouts, which I suspect are due to the amount of time that the "hpEVA VSS Hardware Provider" service takes to start on the SQL server which is being backed up, which appears to be down to unrelated DR Groups on the EVA.

We've installed all of the hotfixes mentioned in the EVA VSS documentation so now I'm a bit stumped on where to go next.

For example:

At 02:30:04 this morning, MSSQLSERVER freezes the I/O on our SQL database as part of the VSS process.

At 02:30:12 the Volume Shadow Copy service starts.

At 02:30:21 the hpEVA VSS Hardware Provider service starts according to the NT System Event Log.

However it is not until over a minute later, at 02:31:37 that the "HP EVA Hardware Provider loaded successfully" appears in the NT Application Event Log, at which time the snapshot is prepared, but it doesn't appear to complete in time.

At 02:31:50, VSS and VolSnap errors are logged basically saying that the writes canot be held any longer. The backup is abandoned and the snapshot is deleted.

So from what I can see, there appears to be a delay of 1 minute 16 seconds from the HP EVA VSS hardware provider starting and then declaring itself as started - this does not include any of the time taken for an attempt to create a snapshot on the EVA.

I do not have any such delays when using the HP EVA VSS configuration tool on the servers in question, nor do I have any issues when using the Command View web interfaces, so I can't understand why it is taking such a long time to start.

After turning on extensive logging on the Command View server, I saw that the VSS client was making a number of calls to NscManager::scGetVdiskInfo and NsaDrmGroup::getMemberProperties for Vdisks in that had nothing to do with the Vdisks being backed up, however all were part of a DR Group. None of these disks are presented to either the SQL server or the Backup server. This exchange of apparently irrelevant information went on for over a minute, with the VSS provider looking at the same disks many times over, for no apparent reason.

Has anyone else seen strange delays like this with the startup of the HP EVA VSS Hardware Provider when using DR Groups?

I may alter the configuration of our EVA to remove the DR Groups to test the behavior, but this would be a drastic last resort.

We're running version 4.03.00 of the HP EVA VSS Provider, chiefly on Windows Server 2003 x64 machines.

Many thanks

Steven