B140i raid controller driver spamming vmkernel logs

[Akın_AKALAN]

Hello All,

In my environment looks like hpdsa driver spamming some crash messages in vmkernel.log. I can't find any solution to stop these messages. Any help will appreciate.

- server model and BIOS version
HP ProLiant DL380 Gen9 | BIOS: P89 | Date: 04/25/2017

- VMware ESXi 6.0.0 build-6921384

- HPE vib installed 
Vendor Name Summary Version Acceptance Level
------ ---- ------- ------- ----------------
Hewlett-Packard scsi-hpdsa hpdsa: scsi driver for VMware ESX 5.5.0.52-1OEM.550.0.0.1331820 PartnerSupported
Hewlett-Packard scsi-hpvsa hpvsa: scsi driver for VMware ESX 5.5.0.100-1OEM.550.0.0.1331820 PartnerSupported
HPE amshelper Agentless Management Service for ESXi 6.0 600.10.5.0-24.2494585 PartnerSupported
HPE conrep Conrep Utility 6.0.0.01-02.00.1.2494585 PartnerSupported
HPE hpe-ilo HPE Channel Interface Driver for VMware ESXi 6.0 600.10.0.0.26-1OEM.600.0.0.2494585 PartnerSupported
HPE hpe-cru HPE CRU Kernel Module for ESXi 6.0 6.0.8.6-1OEM.600.0.0.2494585 PartnerSupported
HPE hpe-build HPE Custom Image Build Information for ESXi 6.0.0 600.9.7.0.17-2494585 PartnerSupported
HPE hpe-esxi-fc-enablement HPE Fiber Channel Enablement Package for ESXi 6.0.0 600.2.6.20-2494585 PartnerSupported
HPE hpe-nmi HPE NMI Sourcing module for ESXi 6.0 600.2.4.16-2494575 PartnerSupported
HPE hpbootcfg HPE ProLiant Boot Configuration Utility for ESXi 6.0.0.02-02.00.6.2494585 PartnerSupported
HPE hponcfg HPE ProLiant Lights-Out Configuration Utility for ESXi 6.0.0.4.4-2.4.2494585 PartnerSupported
HPE hptestevent HPE ProLiant Test Event Utility for ESXi 6.0.0.01-01.00.5.2494585 PartnerSupported
HPE ssacli Smart Storage Administrator CLI for ESXi 2.60.18.0-6.0.0.2494585 PartnerSupported
HPE hpe-smx-provider HPE Insight Management WBEM Providers for ESXi 6.0 600.03.11.00.9-2768847 VMwareAccepted

Driver versions listed below

- Host Bus Adapter 

vmhba1 Intel Corporation HP Dynamic Smart Array B140i RAID controller - 6 port
vmhba2 Intel Corporation HP Dynamic Smart Array B140i RAID controller - 4 port
vmhba3 QLogic Corp 2600 Series 16Gb Fibre Channel to PCI Express HBA
vmhba4 QLogic Corp 2600 Series 16Gb Fibre Channel to PCI Express HBA
vmhba5 QLogic Corp 2600 Series 16Gb Fibre Channel to PCI Express HBA
vmhba6 QLogic Corp 2600 Series 16Gb Fibre Channel to PCI Express HBA
-----------------------------------------------------------------------------------------------------------------
HBA driver name VID:DID SVID:SDID driver version
------ ----------- --------- --------- --------------
vmhba0 hpdsa 103c:193f 103c:3381 5.5.0.52-1OEM
vmhba3 qlnativefc 1077:2031 103c:17e8 2.1.50.0-1vmw.600.3.57.5050593
vmhba4 qlnativefc 1077:2031 103c:17e8 2.1.50.0-1vmw.600.3.57.5050593
vmhba5 qlnativefc 1077:2031 103c:17e8 2.1.50.0-1vmw.600.3.57.5050593
vmhba6 qlnativefc 1077:2031 103c:17e8 2.1.50.0-1vmw.600.3.57.5050593

Spam messages in vmkernel log.

2017-11-21T19:33:00.604Z cpu43:33602)0x4391da11b6b0:[0x418027d99789]SubmitATAWrite@<None>#<None>+0x151 stack: 0x246, 0x4307f7afaa2c, 0x4
2017-11-21T19:33:00.604Z cpu43:33602)0x4391da11b700:[0x418027d96cb0]PAL_AHCI_SendRequest@<None>#<None>+0x6d4 stack: 0x4391da11b9a4, 0x43
2017-11-21T19:33:00.604Z cpu43:33602)0x4391da11b770:[0x418027d9cfb6]PAL_AHCI_SendRequest_VPAL@<None>#<None>+0x16 stack: 0x4391da11b7e0,
2017-11-21T19:33:00.604Z cpu43:33602)0x4391da11b780:[0x418027d73e12]PAL_VM_SendRequest@<None>#<None>+0x26 stack: 0x57d1493418074, 0x417f
2017-11-21T19:33:00.604Z cpu43:33602)0x4391da11b790:[0x418027d50de8]queue_drive_request@<None>#<None>+0x438 stack: 0x19f8, 0x198, 0x4000
2017-11-21T19:33:00.604Z cpu43:33602)0x4391da11b7f0:[0x418027d481f6]queue_elevator_request@<None>#<None>+0x192 stack: 0x4391da11b870, 0x
2017-11-21T19:33:00.604Z cpu43:33602)0x4391da11b840:[0x418027d51f78]queue_all_requests@<None>#<None>+0x204 stack: 0x4307f9a81f3c, 0x4391
2017-11-21T19:33:00.604Z cpu43:33602)0x4391da11b880:[0x418027d8a827]sched_one_stripe_raid1@<None>#<None>+0x167 stack: 0x430100000002, 0x
2017-11-21T19:33:00.604Z cpu43:33602)0x4391da11b910:[0x418027d8afa6]multiblock_mapper_raid1@<None>#<None>+0x302 stack: 0x430289cbcc00, 0
2017-11-21T19:33:00.604Z cpu43:33602)0x4391da11bdb0:[0x418027e2d9ba]logical_read_write@<None>#<None>+0x4ca stack: 0x100100, 0x200200, 0x
2017-11-21T19:33:00.604Z cpu43:33602)0x4391da11be10:[0x418027e2dc30]execute_logical_request@<None>#<None>+0x68 stack: 0x4391da11bea0, 0x
2017-11-21T19:33:00.604Z cpu43:33602)0x4391da11be30:[0x418027e2dfe6]Parse_Task@<None>#<None>+0x26e stack: 0x1, 0x4307e5e67180, 0x0, 0x43
2017-11-21T19:33:00.604Z cpu43:33602)0x4391da11beb0:[0x418027564c30]kthread@com.vmware.driverAPI#9.2+0x158 stack: 0x0, 0x4307e5e60030, 0
2017-11-21T19:33:00.604Z cpu43:33602)0x4391da11bf40:[0x41802756233f]LinuxStartFunc@com.vmware.driverAPI#9.2+0x8f stack: 0x0, 0x418027562
2017-11-21T19:33:00.604Z cpu43:33602)0x4391da11bf90:[0x418026e9c75b]vmkWorldFunc@vmkernel#nover+0x3b stack: 0x418026e9c756, 0x0, 0x4391d
2017-11-21T19:33:00.604Z cpu43:33602)0x4391da11bfd0:[0x41802701746e]CpuSched_StartWorld@vmkernel#nover+0xa2 stack: 0x0, 0x0, 0x0, 0x0, 0
2017-11-21T19:33:00.609Z cpu43:33602)DEBUG ASSERT: pxfer->sg[ ulCount ].host_address.h < 0x100 file=vmkdrivers/src_9/drivers/hpdsa/hpvsa/pal_ahci/ahcisat.c function=SetPRDT line=2817
2017-11-21T19:33:00.609Z cpu43:33602)Backtrace for current CPU #43, worldID=33602, rbp=0x4391da11b6a0
2017-11-21T19:33:00.609Z cpu43:33602)0x4391da11b640:[0x418026e9837e]vmk_LogBacktraceMessage@vmkernel#nover+0x22 stack: 0x4391da11b6a0, 0
2017-11-21T19:33:00.609Z cpu43:33602)0x4391da11b660:[0x418027d98f22]SetPRDT@<None>#<None>+0x1a2 stack: 0x4307e5e8f148, 0x4307f7afb9cc, 0
2017-11-21T19:33:00.609Z cpu43:33602)0x4391da11b6b0:[0x418027d99789]SubmitATAWrite@<None>#<None>+0x151 stack: 0x246, 0x4307f7afaa2c, 0x4
2017-11-21T19:33:00.609Z cpu43:33602)0x4391da11b700:[0x418027d96cb0]PAL_AHCI_SendRequest@<None>#<None>+0x6d4 stack: 0x4391da11b9a4, 0x43
2017-11-21T19:33:00.609Z cpu43:33602)0x4391da11b770:[0x418027d9cfb6]PAL_AHCI_SendRequest_VPAL@<None>#<None>+0x16 stack: 0x4391da11b7e0,
2017-11-21T19:33:00.609Z cpu43:33602)0x4391da11b780:[0x418027d73e12]PAL_VM_SendRequest@<None>#<None>+0x26 stack: 0x57d1494678601, 0x417f
2017-11-21T19:33:00.609Z cpu43:33602)0x4391da11b790:[0x418027d50de8]queue_drive_request@<None>#<None>+0x438 stack: 0x1150, 0x110, 0x4000
2017-11-21T19:33:00.609Z cpu43:33602)0x4391da11b7f0:[0x418027d481f6]queue_elevator_request@<None>#<None>+0x192 stack: 0x4391da11b870, 0x
2017-11-21T19:33:00.609Z cpu43:33602)0x4391da11b840:[0x418027d51f78]queue_all_requests@<None>#<None>+0x204 stack: 0x4307f9a81f3c, 0x4391
2017-11-21T19:33:00.609Z cpu43:33602)0x4391da11b880:[0x418027d8a827]sched_one_stripe_raid1@<None>#<None>+0x167 stack: 0x430100000001, 0x
2017-11-21T19:33:00.609Z cpu43:33602)0x4391da11b910:[0x418027d8afa6]multiblock_mapper_raid1@<None>#<None>+0x302 stack: 0x430289cbcd80, 0
2017-11-21T19:33:00.609Z cpu43:33602)0x4391da11bdb0:[0x418027e2d9ba]logical_read_write@<None>#<None>+0x4ca stack: 0x100100, 0x200200, 0x
2017-11-21T19:33:00.609Z cpu43:33602)0x4391da11be10:[0x418027e2dc30]execute_logical_request@<None>#<None>+0x68 stack: 0x4391da11bea0, 0x
2017-11-21T19:33:00.609Z cpu43:33602)0x4391da11be30:[0x418027e2dfe6]Parse_Task@<None>#<None>+0x26e stack: 0x1, 0x4307e5e67180, 0x0, 0x43
2017-11-21T19:33:00.609Z cpu43:33602)0x4391da11beb0:[0x418027564c30]kthread@com.vmware.driverAPI#9.2+0x158 stack: 0x0, 0x4307e5e60030, 0
2017-11-21T19:33:00.609Z cpu43:33602)0x4391da11bf40:[0x41802756233f]LinuxStartFunc@com.vmware.driverAPI#9.2+0x8f stack: 0x0, 0x418027562
2017-11-21T19:33:00.609Z cpu43:33602)0x4391da11bf90:[0x418026e9c75b]vmkWorldFunc@vmkernel#nover+0x3b stack: 0x418026e9c756, 0x0, 0x4391d
2017-11-21T19:33:00.609Z cpu43:33602)0x4391da11bfd0:[0x41802701746e]CpuSched_StartWorld@vmkernel#nover+0xa2 stack: 0x0, 0x0, 0x0, 0x0, 0
2017-11-21T19:33:00.609Z cpu43:33602)DEBUG ASSERT: pxfer->sg[ ulCount ].host_address.h < 0x100 file=vmkdrivers/src_9/drivers/hpdsa/hpvsa/pal_ahci/ahcisat.c function=SetPRDT line=2817
2017-11-21T19:33:00.609Z cpu43:33602)Backtrace for current CPU #43, worldID=33602, rbp=0x4391da11b6a0
2017-11-21T19:33:00.609Z cpu43:33602)0x4391da11b640:[0x418026e9837e]vmk_LogBacktraceMessage@vmkernel#nover+0x22 stack: 0x4391da11b6a0, 0
2017-11-21T19:33:00.609Z cpu43:33602)0x4391da11b660:[0x418027d98f22]SetPRDT@<None>#<None>+0x1a2 stack: 0x4307e5e8f148, 0x4307f7afc374, 0
2017-11-21T19:33:00.609Z cpu43:33602)0x4391da11b6b0:[0x418027d99789]SubmitATAWrite@<None>#<None>+0x151 stack: 0x246, 0x4307f7afaa2c, 0x4
2017-11-21T19:33:00.609Z cpu43:33602)0x4391da11b700:[0x418027d96cb0]PAL_AHCI_SendRequest@<None>#<None>+0x6d4 stack: 0x4391da11b9a4, 0x43
2017-11-21T19:33:00.609Z cpu43:33602)0x4391da11b770:[0x418027d9cfb6]PAL_AHCI_SendRequest_VPAL@<None>#<None>+0x16 stack: 0x4391da11b7e0,
2017-11-21T19:33:00.609Z cpu43:33602)0x4391da11b780:[0x418027d73e12]PAL_VM_SendRequest@<None>#<None>+0x26 stack: 0x57d1494678601, 0x417f
2017-11-21T19:33:00.609Z cpu43:33602)0x4391da11b790:[0x418027d50de8]queue_drive_request@<None>#<None>+0x438 stack: 0x19f8, 0x198, 0x4000
2017-11-21T19:33:00.609Z cpu43:33602)0x4391da11b7f0:[0x418027d481f6]queue_elevator_request@<None>#<None>+0x192 stack: 0x4391da11b870, 0x
2017-11-21T19:33:00.609Z cpu43:33602)0x4391da11b840:[0x418027d51f78]queue_all_requests@<None>#<None>+0x204 stack: 0x4307f9a81f3c, 0x4391
2017-11-21T19:33:00.609Z cpu43:33602)0x4391da11b880:[0x418027d8a827]sched_one_stripe_raid1@<None>#<None>+0x167 stack: 0x430100000001, 0x
2017-11-21T19:33:00.609Z cpu43:33602)0x4391da11b910:[0x418027d8afa6]multiblock_mapper_raid1@<None>#<None>+0x302 stack: 0x430289cbcd80, 0
2017-11-21T19:33:00.609Z cpu43:33602)0x4391da11bdb0:[0x418027e2d9ba]logical_read_write@<None>#<None>+0x4ca stack: 0x100100, 0x200200, 0x
2017-11-21T19:33:00.609Z cpu43:33602)0x4391da11be10:[0x418027e2dc30]execute_logical_request@<None>#<None>+0x68 stack: 0x4391da11bea0, 0x
2017-11-21T19:33:00.609Z cpu43:33602)0x4391da11be30:[0x418027e2dfe6]Parse_Task@<None>#<None>+0x26e stack: 0x1, 0x4307e5e67180, 0x0, 0x43
2017-11-21T19:33:00.609Z cpu43:33602)0x4391da11beb0:[0x418027564c30]kthread@com.vmware.driverAPI#9.2+0x158 stack: 0x0, 0x4307e5e60030, 0
2017-11-21T19:33:00.609Z cpu43:33602)0x4391da11bf40:[0x41802756233f]LinuxStartFunc@com.vmware.driverAPI#9.2+0x8f stack: 0x0, 0x418027562
2017-11-21T19:33:00.609Z cpu43:33602)0x4391da11bf90:[0x418026e9c75b]vmkWorldFunc@vmkernel#nover+0x3b stack: 0x418026e9c756, 0x0, 0x4391d
2017-11-21T19:33:00.609Z cpu43:33602)0x4391da11bfd0:[0x41802701746e]CpuSched_StartWorld@vmkernel#nover+0xa2 stack: 0x0, 0x0, 0x0, 0x0, 0

[Akın_AKALAN]

Thanks for moving it to correct forum.

Anyone?

[Akın_AKALAN]

that is really shame, no one replying from HPE?

This is a serious problem and I saw at least 3 friends having this issue. No one has answer?

[Dennis Handly]

> no one replying from HPE?

This is a peer to peer forum, not an official HPE support site.
Have you tried contacting the HPESC?
https://support.hpe.com/hpesc/home

[Akın_AKALAN]

I already opened an SR and they said use grep to find what you are looking for... This is unacceptable.

Go to VMware forums and you can find lots of folks from VMware replying guys and taking actions..

Nevermind. I'll change the hosts with a different vendor.

[Akın_AKALAN]

anyone having this issue, don't waste your time with HPE... Use Dell servers..