Beware of the latest versions of HP Version Control Agent because of a critical bug!

[Tri-Media]

Hello,

Beware!

The versions 7.2.0.0 (cp017734.exe) and 7.2.2.0 (cp020132.exe) of HP Version Control Agent for Windows x64 contain a critical bug which would result in random BSODs with an error 0xD1 and multiple iScsiPrt Error events (IDs 7, 29, 42) in the event log.

The configuration:

Server: HP ProLiant DL360 G5

OS: Windows Server 2008 R2, Windows Server 2012 R2

Roles and Features: Hyper-V, Failover Cluster Hyper-V

Environment: iSCSI cluster shared volume(s) provided by Windows Server 2008 Storage Edition

Sympoms: random BSODs, Error Events

Event 42, iScsiPrt

Target sent an invalid status sequence number for a connection. Dump data contains Expected Status Sequence number followed by the given status sequence number.

Details: ... 0000040F 0F040000

Event 29, iScsiPrt

Target rejected an iSCSI PDU sent by the initiator. Dump data contains the rejected PDU.

Event 7, iScsiPrt

The initiator could not send an iSCSI PDU. Error status is given in the dump data.

The versions of HP VCA 7.1.2.0 and 7.1.0.3 are OK.

That's a nasty bug, which I discovered when was adding an additional node to our Hyper-V Failover Cluster. The previous nodes were configured a couple of years ago and were not touched. For the new node I installed the whole fresh stuff from one of HP ProLiant service packs, and after migration there of some virtual machines got a lot of iSCSI error events, virtual machines crashes and finally several BSODs:

Operating System failure (Windows bug check, STOP: 0x000000D1 (0x0000000000000010, 0x0000000000000002, 0x0000000000000000, 0xFFFFF8800137EFA8))

As one could know, each machine in a Hyper-V cluster should have pretty the same configuration, especially for the CPUs, and the new machine was worked as a webserver just perfectly for years. So I was sure that was a software bug. But I would never ever consider that the cause was Version Control Agent! Finally I started to install HP stuff one-by-one of the same versions as at other machines of the cluster. However for VCA for some very strange reason I made an exclusion and installed the latest one! Maybe the Gods of sysadmins helped me there. Machine crashed almost immediately when VCA was installed, so I got it pinned.

I tried to contact HP support, but as the server is beyond of warranty, they ignored me pretty much.

I suspect the bug would produce itself on any G5, G6, G7 or Gen8 server in the similar environment...

So leaving my warning there.

Best luck

Sincerely

Sergey Nudnov