Server DL380 G5 shutdown unexprectly

[khanhhoa]

Dear all,
We have one HP DL380 G5 server that is running Windows 2003 Enterprise edition. But lately it was frequently shut down unexpected. When the server was restarted, there is error in system log as follow :

Event Type: Error
Event Source: System Error
Event Category: (102)
Event ID: 1003
Date: 4/22/2008
Time: 9:24:27 AM
User: N/A
Computer: CHEMICAL_SVR
Description:
Error code 0000000a, parameter1 00000000, parameter2 d0000002, parameter3 00000001, parameter4 80830126.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
Data:
0000: 53 79 73 74 65 6d 20 45 System E
0008: 72 72 6f 72 20 20 45 72 rror Er
0010: 72 6f 72 20 63 6f 64 65 ror code
0018: 20 30 30 30 30 30 30 30 0000000
0020: 61 20 20 50 61 72 61 6d a Param
0028: 65 74 65 72 73 20 30 30 eters 00
0030: 30 30 30 30 30 30 2c 20 000000,
0038: 64 30 30 30 30 30 30 32 d0000002
0040: 2c 20 30 30 30 30 30 30 , 000000
0048: 30 31 2c 20 38 30 38 33 01, 8083
0050: 30 31 32 36 0126

And also in "HP Proliant Intergrated Management Log viewer" show some error log :

"POST Error: 1785-Drive Array not Configured",POST Messages,Caution,1,12/19/2007 7:10 AM,12/19/2007 7:10 AM
"Network Adapter Link Down (Slot 0, Port 2)",Network Adapter,Critical,3,3/6/2008 1:24 PM,1/8/2008 7:37 AM
"ASR Detected by System ROM",Automatic Server Recovery,Repaired,1,4/1/2008 9:21 AM,3/7/2008 2:28 PM
"Blue Screen Trap (BugCheck, STOP: 0x0000000A (0x00000000, 0x000000FF, 0x00000001, 0x80833CD6))",Operating System,Repaired,1,4/1/2008 9:21 AM,3/12/2008 1:41 AM
"Blue Screen Trap (BugCheck, STOP: 0x0000000A (0x00000000, 0x000000FF, 0x00000001, 0x80833CD6))",Operating System,Repaired,1,4/1/2008 9:21 AM,3/12/2008 6:55 AM
"Blue Screen Trap (BugCheck, STOP: 0x0000000A (0x00000000, 0x000000FF, 0x00000001, 0x80833CD6))",Operating System,Repaired,1,4/1/2008 9:21 AM,3/12/2008 7:24 AM
"Blue Screen Trap (BugCheck, STOP: 0x00000024 (0x0019033D, 0xB856E6EC, 0xB856E3E8, 0x00000000))",Operating System,Repaired,1,4/1/2008 9:21 AM,3/21/2008 6:24 AM
"ASR Detected by System ROM",Automatic Server Recovery,Repaired,1,4/1/2008 9:21 AM,3/27/2008 10:19 PM
"Blue Screen Trap (BugCheck, STOP: 0x0000000A (0x00000000, 0x000000FF, 0x00000001, 0x80833CD6))",Operating System,Repaired,1,4/1/2008 9:21 AM,3/29/2008 4:53 PM
"Blue Screen Trap (BugCheck, STOP: 0x0000008E (0xC0000005, 0x80833CD6, 0xB8028CF0, 0x00000000))",Operating System,Critical,1,4/4/2008 12:35 PM,4/4/2008 12:35 PM
"Blue Screen Trap (BugCheck, STOP: 0x0000000A (0x00000000, 0x000000FF, 0x00000001, 0x80833CD6))",Operating System,Critical,1,4/7/2008 12:43 AM,4/7/2008 12:43 AM
"Blue Screen Trap (BugCheck, STOP: 0x0000000A (0x40000024, 0x000000FF, 0x00000001, 0x80833CD6))",Operating System,Critical,1,4/14/2008 11:19 PM,4/14/2008 11:19 PM
"Blue Screen Trap (BugCheck, STOP: 0x0000000A (0x00000000, 0x000000FF, 0x00000001, 0x80833CD6))",Operating System,Critical,1,4/17/2008 12:43 AM,4/17/2008 12:43 AM
"ASR Detected by System ROM",Automatic Server Recovery,Critical,1,4/21/2008 9:51 PM,4/21/2008 9:51 PM
"Blue Screen Trap (BugCheck, STOP: 0x0000000A (0x00000000, 0xD0000002, 0x00000001, 0x80830126))",Operating System,Critical,1,4/22/2008 2:36 AM,4/22/2008 2:36 AM

So anyone please help me to solve this problem, it is very urgent
Thank you so much

[James ~ Happy Dude]

Hello,

For Stop code 0xA .. refer:
http://support.microsoft.com/kb/836049
you can request a hotfix.

:) Regards.

[KarloChacon]

hi

check this

http://support.microsoft.com/kb/836049

regards

[KarloChacon]

oh man James

almost at the same time jejje

[khanhhoa]

Dear all,
Thank you for your reply.
But our server is using Windows 2003 SP1 and the hotfix that your provide is just released for Windows 2k3 SP1.
How should I do ?

[KarloChacon]

hi

so your server applies right?


"But our server is using Windows 2003 SP1 and the hotfix that your provide is just released for Windows 2k3 SP1."


???

regards

[khanhhoa]

Yes, it was applied most hotfix and service patch by using window update service

[James ~ Happy Dude]

Hi,
If you are not ready to install the hotfix;
Then Collect the memory dump files & call HP Support; They should guide you after analysis.

;) Regards.




____________________________________________
Yo Karlo !
____________________________________________

[khanhhoa]

Dear all,
I analys my dump file and is says as below, please tell me what should I do ?
Thanks

Microsoft (R) Windows Debugger Version 6.9.0003.113 X86
Copyright (c) Microsoft Corporation. All rights reserved.


Loading Dump File [C:\WINDOWS\Minidump\Mini051808-01.dmp]
Mini Kernel Dump File: Only registers and stack trace are available

Symbol search path is: C:\WINDOWS\Symbols\*.*
Executable search path is:
Windows Server 2003 Kernel Version 3790 (Service Pack 2) MP (4 procs) Free x86 compatible
Product: Server, suite: Enterprise TerminalServer SingleUserTS
Built by: 3790.srv03_sp2_rtm.070216-1710
Kernel base = 0x80800000 PsLoadedModuleList = 0x808af9c8
Debug session time: Sun May 18 16:37:54.368 2008 (GMT+7)
System Uptime: 1 days 5:16:14.343
Loading Kernel Symbols
............................................................................................................
Loading User Symbols
Loading unloaded module list
.......
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************

Use !analyze -v to get detailed debugging information.

BugCheck A, {0, ff, 1, 80833cd6}

Probably caused by : ntkrnlmp.exe ( nt!KiServiceExit+a2 )

Followup: MachineOwner
---------

3: kd> !analyze -v
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************

IRQL_NOT_LESS_OR_EQUAL (a)
An attempt was made to access a pageable (or completely invalid) address at an
interrupt request level (IRQL) that is too high. This is usually
caused by drivers using improper addresses.
If a kernel debugger is available get the stack backtrace.
Arguments:
Arg1: 00000000, memory referenced
Arg2: 000000ff, IRQL
Arg3: 00000001, bitfield :
bit 0 : value 0 = read operation, 1 = write operation
bit 3 : value 0 = not an execute operation, 1 = execute operation (only on chips which support this level of status)
Arg4: 80833cd6, address which referenced memory

Debugging Details:
------------------


WRITE_ADDRESS: 00000000

CURRENT_IRQL: 0

FAULTING_IP:
nt!KiServiceExit+a2
80833cd6 0f84c1000000 je nt!KiSystemCallExit3+0x60 (80833d9d)

CUSTOMER_CRASH_COUNT: 1

DEFAULT_BUCKET_ID: DRIVER_FAULT_SERVER_MINIDUMP

BUGCHECK_STR: 0xA

PROCESS_NAME: FrameworkServic

TRAP_FRAME: b984acf0 -- (.trap 0xffffffffb984acf0)
ErrCode = 00000002
eax=00000000 ebx=86fe7b40 ecx=00000001 edx=ffffffff esi=86fe7b40 edi=b984ad64
eip=80833cd6 esp=b984ad64 ebp=b984ad64 iopl=0 nv up di pl nz na pe nc
cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000 efl=00010006
nt!KiServiceExit+0xa2:
80833cd6 0f84c1000000 je nt!KiSystemCallExit3+0x60 (80833d9d) [br=0]
Resetting default scope

LAST_CONTROL_TRANSFER: from 80833cd6 to 80836de5

STACK_TEXT:
b984acf0 80833cd6 badb0d00 ffffffff 00000001 nt!KiTrap0E+0x2a7
b984ad64 7c8285ec badb0d00 02cdfefc 00000000 nt!KiServiceExit+0xa2
WARNING: Frame IP not in any known module. Following frames may be wrong.
02cdff5c 00000000 00000000 00000000 00000000 0x7c8285ec


STACK_COMMAND: kb

FOLLOWUP_IP:
nt!KiServiceExit+a2
80833cd6 0f84c1000000 je nt!KiSystemCallExit3+0x60 (80833d9d)

SYMBOL_STACK_INDEX: 1

SYMBOL_NAME: nt!KiServiceExit+a2

FOLLOWUP_NAME: MachineOwner

MODULE_NAME: nt

IMAGE_NAME: ntkrnlmp.exe

DEBUG_FLR_IMAGE_TIMESTAMP: 45d6a0af

FAILURE_BUCKET_ID: 0xA_W_nt!KiServiceExit+a2

BUCKET_ID: 0xA_W_nt!KiServiceExit+a2

Followup: MachineOwner
---------

3: kd> .trap 0xffffffffb984acf0
ErrCode = 00000002
eax=00000000 ebx=86fe7b40 ecx=00000001 edx=ffffffff esi=86fe7b40 edi=b984ad64
eip=80833cd6 esp=b984ad64 ebp=b984ad64 iopl=0 nv up di pl nz na pe nc
cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000 efl=00010006
nt!KiServiceExit+0xa2:
80833cd6 0f84c1000000 je nt!KiSystemCallExit3+0x60 (80833d9d) [br=0]

[KarloChacon]

hi

previous post in thread you say server has SP1

I would start update to SP2, after that I would reapply HP proliant support pack 8.0 again and test

I think blue screens have too any different flavors (stop codes)

so try that, something else let us know the server configuration I mean
number of Processors, memory dimms, any PCI card, attached to something (tape, storage, ...), power supplies.

thanks

[KarloChacon]

oh sorry

it says SP2 already, so let us know the other thing I asked you

something else you don't mention if you have latest drivers and firmware from HP

PSP from HP 8.0
http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareDescription.jsp?lang=en&cc=us&prodTypeId=15351&prodSeriesId=1121516&prodNameId=3288134&swEnvOID=1005&swLang=8&mode=2&taskId=135&swItem=MTX-9602c6c78829482d89b1432e46


latest firmware 8.1
http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareDescription.jsp?lang=en&cc=us&prodTypeId=15351&prodSeriesId=1121516&prodNameId=3288134&swEnvOID=1005&swLang=8&mode=2&taskId=135&swItem=MTX-2c5987b8362348dbb3e80d277b

regards

[khanhhoa]

Hi Karlo,
Thanks for your replied.
My server had been used PSP 7.90 and firmware 7.91, it is not the lastest version but I think that's ok because it was deliverred with server.
My server's specification show as below, please help me show this problem.
if you need further information please let me know. Thank you very much

System
Product Name ProLiant DL380 G5
Serial Number SGH802DPAK
Processor Package 1 (Socket 1, Core = 4) Intel Xeon Processor @ 2.00 Ghz
Operating system environment Microsoft(R) Windows(R) Server 2003, Enterprise Edition, Service Pack 2 (Build 3790)
Total memory - 2048 Mbytes
Board 0
DIMM 1A ( DDR2-FBD ) 1024 Mbytes (667 MHz)
DIMM 2C Not Installed
DIMM 3A ( DDR2-FBD ) 1024 Mbytes (667 MHz)
DIMM 4C Not Installed
DIMM 5B Not Installed
DIMM 6D Not Installed
DIMM 7B Not Installed
DIMM 8D Not Installed
ECC memory installed Yes
Hardware PAE enabled No
Serial presence detect (SPD) information - DIMM 1
Memory type DDR2 SDRAM FB-DIMM
DIMM Description
Spare Part Number
Single-Bit Threshold Exceeded Status No DIMM errors detected
Multi-Bit Status No DIMM errors detected
Single-Bit Threshold Count 0
Multi-Bit Count 0
Serial presence detect (SPD) information - DIMM 3
Memory type DDR2 SDRAM FB-DIMM
DIMM Description
Spare Part Number
Single-Bit Threshold Exceeded Status No DIMM errors detected
Multi-Bit Status No DIMM errors detected
Single-Bit Threshold Count 0
Multi-Bit Count 0
Communication
Serial Port 1 (Address 3F8h)
USB Device 1 USB UHCI Root Hub
USB Device 2 USB Keyboard & Mouse
USBDevice OnPort 2 generic usb hub
USB Device 4 USB UHCI Root Hub
USB Device 5 USB UHCI Root Hub
USB Device 6 USB UHCI Root Hub
USB Device 7 USB UHCI Root Hub
USB Device 8 USB EHCI Root Hub
USB Device 9 Standard Universal PCI to USB Host Controller
USB Device 10 Standard Universal PCI to USB Host Controller
USB Device 11 Standard Universal PCI to USB Host Controller
USB Device 12 Standard Universal PCI to USB Host Controller
USB Device 13 Standard Enhanced PCI to USB Host Controller
USB Device 14 Standard Universal PCI to USB Host Controller
Network Controller 1, Slot 0 NC373i PCI Express Multifunction Gigabit Server Adapter
Network Controller 2, Slot 0 NC373i PCI Express Multifunction Gigabit Server Adapter
Storage
Optical Drive 1 TEAC DV-28E-V at Primary/Master
Physical Hard Drive 1, Controller Slot 1 146.8 Gbytes 10K RPM - HP DG146ABAB4
Physical Hard Drive 2, Controller Slot 1 146.8 Gbytes 10K RPM - HP DG146ABAB4
Physical Hard Drive 3, Controller Slot 1 146.8 Gbytes 10K RPM - HP DG146ABAB4
Physical Hard Drive 4, Controller Slot 1 146.8 Gbytes 10K RPM - HP DG146ABAB4
Logical Disks
C: Logical Disk 1 ~ 52.43GB
D: Logical Disk 2 ~ 184.33GB
E: Logical Disk 3 ~ 56.80GB
PCI Slots
PCI-E Slot 1 In Use
PCI-E Slot 2 Available
PCI-E Slot 3 Available
PCI-E Slot 4 Available
PCI-E Slot 5 Available

[KarloChacon]

hi

like I told you since those are too many different blue screens stop codes my recommendation is update to the latest if not it's going to be difficult to know what is causing those blue screen so.....


regards

[khanhhoa]

Hi,
I'm sorry for my late reply because need few days to know result.
I've already update PSP and firmware to lastest version but the problem still happen.
What should I do next ?
Thanks

[KarloChacon]

hi

have you contacted microsoft?

have you think this is a windows issue " Probably caused by : ntkrnlmp.exe ( nt!KiServiceExit+a2 ) "???

regards