topic Re: Parstatus command hangs in 2 npars in sd3200 with hp-ux 11.11 in Operating System - HP-UX

Parstatus command hangs in 2 npars in sd3200 with hp-ux 11.11

likid0 — Thu, 04 Aug 2011 12:44:58 GMT

Hi,

I have a sd3200, with 2 npars, since yesterday when I launch the parstatus command it doesn't finish it just hangs...

I imagine is a communication problem between the npars and the gsp but there are a lot of things in the middle

I have tried giving the GSP a reset, but it didn't help, I checked SEL and FPL for errors couldn't find anything very helpfull(there is a power fault in cell4,but thats another problem..), i will attach the logs.

Nothing in syslog.log either.

Any idea what can solve this problem, Will a npar(reboot/reset(RS) help?

I also thought of reseting the gsp bus with the RU command, but need hp for that in the sd3200, no support anymore ..

I also attached the output of tusc command when launching parstatus.

Re: Parstatus command hangs in 2 npars in sd3200 with hp-ux 11.11

likid0 — Thu, 04 Aug 2011 12:46:57 GMT

it didn't let me attach the files

Re: Parstatus command hangs in 2 npars in sd3200 with hp-ux 11.11

Torsten. — Thu, 04 Aug 2011 12:49:30 GMT

When you say "since yesterday" - something changed? Maybe something related to the cimserver, any providers, etc ...?

Re: Parstatus command hangs in 2 npars in sd3200 with hp-ux 11.11

Dennis Handly — Thu, 04 Aug 2011 13:20:32 GMT

>it didn't let me attach the files

Did you give them a suffix like .txt?

Re: Parstatus command hangs in 2 npars in sd3200 with hp-ux 11.11

likid0 — Thu, 04 Aug 2011 14:19:38 GMT

yes the files have a .log

Re: Parstatus command hangs in 2 npars in sd3200 with hp-ux 11.11

Dennis Handly — Thu, 04 Aug 2011 14:25:24 GMT

>the files have a .log

You may have to use .txt.

Re: Parstatus command hangs in 2 npars in sd3200 with hp-ux 11.11

likid0 — Thu, 04 Aug 2011 14:30:59 GMT

They told me about parstatus, and it was hanging since the 2nd of august, but looking further into the problem, i found out icod_stat commands also hanging since the 13th of june:

root 16290     1 0 Jun 13 ?         0:00 /usr/sbin/icod_stat
    root 16564     1 0 Jun 13 ?         0:00 /usr/sbin/icod_stat
    root   323   197 0 Jun 18 ?         0:00 icod_stat
    root 28488     1 0 Jun 19 ?         0:00 /usr/sbin/icod_stat
    root 4699 4553 0 Jun 22 ?         0:00 icod_stat
    root 19416     1 0 Jun 15 ?         0:00 /usr/sbin/icod_stat
    root 1064   942 0 Jun 15 ?         0:00 icod_stat
    root 8939     1 0 Jun 21 ?         0:00 /usr/sbin/icod_stat
    root 4501     1 0 Jun 14 ?         0:00 /usr/sbin/icod_stat
    root 11044 10904 0 Jun 14 ?         0:00 icod_stat
..................................

And that corresponds with the a problem we had in the cpd, with overtemperature, and the complex shutdown

30 PM   0     *5 0x582008516100406e 0x00006f050d07000a INLET_OVERTEMP
131 PM   0     *5 0x582008516100406e 0x00006f050d07000d INLET_OVERTEMP
132 PM   0     *6 0x582008616100406f 0x00006f050d070b09 INLET_OVERTEMP
133 PM   0     *14 0x5c2008e16100406d 0x00006f050d071011 INLET_OVERTEMP
134 PM   0     *2 0x5c20082944ff302f 0x00006f050d071012 CABPWR_OFF
135 PDHC 0,3 *14 0x246014e39201404f 0x00ffff01ffffff91 POWER_FAULT
135 PDHC 0,3 *14 0x58601c0000004040 0x00006f050d071012 06/13/2011 07:16:18

So it looks that once the npars booted again, they where not able to speak with the gsp.

And that brings us also to the power_fault we have in cell 0,3:

HW status for Cell 3 in cabinet 0: FAILURE DETECTED
Power status: on, 1.6V BRICK 1 UNDER VOLTAGE FAULT

does brick 1 refer to the power brick in the backplane HBPB1 ?

could be related, we have to fix it also..

On the other hand Torsten, no we didn't modify anithing cim related, and are very stable we hardly ever touch them

Re: Parstatus command hangs in 2 npars in sd3200 with hp-ux 11.11

likid0 — Thu, 04 Aug 2011 14:32:42 GMT

triying with .txt

Re: Parstatus command hangs in 2 npars in sd3200 with hp-ux 11.11

Torsten. — Thu, 04 Aug 2011 17:34:09 GMT

I would start now to check the components with GSP "ps" command.

Re: Parstatus command hangs in 2 npars in sd3200 with hp-ux 11.11

likid0 — Fri, 05 Aug 2011 06:54:37 GMT

Yes, In the ps output I have once cell with a power problem:

    B - Cabinet (UGUY)
    C - Cell
    G - GSP
    I - Core IO
        Select Device: c

    Enter cabinet number: 0
    Enter slot number: 3

HW status for Cell 3 in cabinet 0: FAILURE DETECTED

Power status: on, 1.6V BRICK 1 UNDER VOLTAGE FAULT
Boot is blocked; PDH memory is shared
Cell Attention LED is off
RIO cable status: not connected
RIO cable connection physical location: cannot be determined
Core cell is INVALID

PDH status LEDs: _*__
                              CPUs
                            0 1 2 3
          Populated         * * * *
          Over temperature

DIMMs populated:
+----- A -----+ +----- B -----+ +----- C -----+ +----- D -----+
0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7
* * * *         * * * *         * * * *         * * * *

I have also seen this error in the df output:

ERROR: There has been an error retrieving FRU CPU: Location 3, 1
    RtnCodeUsbDevNotPresent

176 of 176 FRU IDs were retrieved and valid

------------------------------------------------------------------------------
Fru Name         Part Name   Loc Serial Num   Art Eng Scan R Fru Spec.
Manf Test Hist. 0 Manf Test Hist. 1 Manf Test Hist. 2 CC V FR
Manf Test Hist. 3 Manf Test Hist. 4 Manf Test Hist. 5 Spare
------------------------------------------------------------------------------
SBC Board        A1150-2094                             0x2020 B
202020202020202020 202020202020202020 202020202020202020 a7 Y A
202020202020202020 202020202020202020 202020202020202020 2020
             BPS AXXXX-XXXXX 0    NOT VALID        000E 0x3030              0x1
000000000000000000 000000000000000000 000000000000000000 3b Y A
000000000000000000 000000000000000000 000000000000000000 0000
             BPS AXXXX-XXXXX 1    NOT VALID        000F 0x3030              0x2
000000000000000000 000000000000000000 000000000000000000 3d Y A
000000000000000000 000000000000000000 000000000000000000 0000
             BPS AXXXX-XXXXX 2    NOT VALID        000E 0x3030              0x1
000000000000000000 000000000000000000 000000000000000000 3b Y A