HPE 3PAR StoreServ Storage

3Par 7200 -Interface Card 0 Failed (Interface Card Firmware Unknown {0x0})

 
SOLVED
Go to solution
Luke_Y
Advisor

3Par 7200 -Interface Card 0 Failed (Interface Card Firmware Unknown {0x0})

Hello all.

i have checked showinventory command and could only see one controller.

i have replaced the controller but still the status is the same- can only see one controller .

i can see in the 3Par management console that i have an error:

Interface Card 0 Failed (Interface Card Firmware Unknown {0x0})
in the back of the new controller the light is solid green.

any ideas?

 

 

 

9 REPLIES 9
sbhat09
HPE Pro

Re: 3Par 7200 -Interface Card 0 Failed (Interface Card Firmware Unknown {0x0})

Hello @Luke_Y,

Is that Node 0 or Node 1 which you replaced?

How many Nodes you see when you run the command 'showsys' or 'showsys -d' ?

How about the paths and devices connected to the node which is not showing up in 'showinventory'? Are those paths and devices looking active?

Regards,
Srinivas Bhat

If you feel this was helpful please click the KUDOS! thumb below!
Note: All of my comments are my own and are not any official representation of HPE.


I am an HPE Employee

Accept or Kudo

Luke_Y
Advisor

Re: 3Par 7200 -Interface Card 0 Failed (Interface Card Firmware Unknown {0x0})

hello @sbhat09 .

it was Node 0 (bottom one).

when running the commands showsys i see only one node.

where can i check if the paths are active.inactive?

 

 

sbhat09
HPE Pro

Re: 3Par 7200 -Interface Card 0 Failed (Interface Card Firmware Unknown {0x0})

Hello @Luke_Y,

You can run 'showhost -pathsum' to check the host paths.

After replacing the node, have you completed all the processes? Please run the command 'admithw -checkonly' to see any hardwares added are pending for admission.

Regards,
Srinivas Bhat

If you feel this was helpful please click the KUDOS! thumb below!
Note: All of my comments are my own and are not any official representation of HPE.


I am an HPE Employee

Accept or Kudo

Luke_Y
Advisor

Re: 3Par 7200 -Interface Card 0 Failed (Interface Card Firmware Unknown {0x0})

Hello @sbhat09 
"After replacing the node, have you completed all the processes?"

We have just replaced the node , we didnt do any other process after that.

this is the output of  admithw -checkonly

Checking nodes...

Some nodes appear to be missing from the system:

Node 0

 

Checking volumes...

 

Checking system LDs...

The following system LDs are not valid:

Id Name   RAID -Detailed_State- Own SizeMB UsedMB Use Lgct LgId WThru MapV

6 log0.0    1 not_started      -/-  61440      0 log    0  ---     N    N

--------------------------------------------------------------------------

1                                   61440      0

 

After an upgrade from 2.2.4 or earlier, if sufficient space is present,

admithw will automatically remove and recreate the preserved data LDs

with a larger set size to improve their availability.

 

Checking ports...

 

Checking state of disks...

Checking for drive table upgrade packages

The following disks are NOT in an acceptable state:

Id CagePos Type -State-- --------------------Detailed_State-------------------- -SedState--

  0 0:0:0   FC   degraded missing_B_port                                         not_capable

  1 0:1:0   FC   degraded missing_B_port                                         not_capable

  2 0:2:0   FC   degraded missing_B_port                                         not_capable

  3 0:3:0   FC   degraded missing_B_port                                         not_capable

  4 0:4:0   FC   degraded missing_B_port                                         not_capable

  5 0:5:0   FC   degraded missing_B_port                                         not_capable

  6 0:6:0   FC   degraded missing_B_port                                         not_capable

  7 0:7:0   FC   degraded missing_B_port                                         not_capable

  8 6:14:0  FC   degraded missing_B_port                                         not_capable

  9 0:9:0   FC   degraded missing_B_port                                         not_capable

10 3:2:0   FC   degraded missing_B_port                                         not_capable

11 3:3:0   FC   degraded missing_B_port                                         not_capable

12 1:13:0  NL   degraded missing_B_port                                         not_capable

13 3:5:0   FC   degraded missing_B_port                                         not_capable

14 3:6:0   FC   degraded missing_B_port                                         not_capable

15 3:7:0   FC   degraded missing_B_port                                         not_capable

17 3:9:0   FC   degraded missing_B_port                                         not_capable

18 1:0:0   NL   degraded missing_B_port                                         not_capable

19 1:1:0   NL   degraded missing_B_port                                         not_capable

20 1:2:0   NL   degraded missing_B_port                                         not_capable

21 1:4:0   NL   degraded missing_B_port                                         not_capable

22 1:5:0   NL   degraded missing_B_port                                         not_capable

23 1:6:0   NL   degraded missing_B_port                                         not_capable

24 1:8:0   NL   degraded missing_B_port                                         not_capable

25 1:9:0   NL   degraded missing_B_port                                         not_capable

26 1:10:0  NL   degraded missing_B_port                                         not_capable

27 1:12:0  NL   degraded missing_B_port                                         not_capable

28 1:3:0   NL   degraded missing_B_port                                         not_capable

29 1:14:0  NL   degraded missing_B_port                                         not_capable

30 1:16:0  NL   degraded missing_B_port                                         not_capable

31 1:17:0  NL   degraded missing_B_port                                         not_capable

32 1:20:0  NL   degraded missing_B_port                                         not_capable

33 1:21:0  NL   degraded missing_B_port                                         not_capable

34 6:0:0   FC   degraded missing_B_port                                         not_capable

35 6:1:0   FC   degraded missing_B_port                                         not_capable

36 6:2:0   FC   degraded missing_B_port                                         not_capable

37 6:3:0   FC   degraded missing_B_port                                         not_capable

38 6:4:0   FC   degraded missing_B_port                                         not_capable

39 6:5:0   FC   degraded missing_B_port                                         not_capable

40 6:6:0   FC   degraded missing_B_port                                         not_capable

41 6:7:0   FC   degraded missing_B_port                                         not_capable

42 6:8:0   FC   degraded missing_B_port                                         not_capable

43 6:9:0   FC   degraded missing_B_port                                         not_capable

44 5:2:0   FC   degraded missing_B_port                                         not_capable

45 5:3:0   FC   degraded missing_B_port                                         not_capable

46 5:4:0   FC   degraded missing_B_port                                         not_capable

47 5:5:0   FC   degraded missing_B_port                                         not_capable

48 5:6:0   FC   degraded missing_B_port                                         not_capable

49 5:7:0   FC   degraded missing_B_port                                         not_capable

50 5:8:0   FC   degraded missing_B_port                                         not_capable

51 5:9:0   FC   degraded missing_B_port                                         not_capable

52 3:4:0?  FC   failed   vacated,missing,invalid_media,no_valid_ports,servicing unknown

53 1:7:0   NL   degraded missing_B_port                                         not_capable

54 4:22:0  NL   degraded missing_B_port                                         not_capable

55 4:18:0  NL   degraded missing_B_port                                         not_capable

56 1:18:0  NL   degraded missing_B_port                                         not_capable

57 4:3:0   NL   degraded missing_B_port                                         not_capable

58 1:22:0  NL   degraded missing_B_port                                         not_capable

59 4:7:0   NL   degraded missing_B_port                                         not_capable

60 4:0:0   NL   degraded missing_B_port                                         not_capable

61 4:1:0   NL   degraded missing_B_port                                         not_capable

62 4:2:0   NL   degraded missing_B_port                                         not_capable

63 4:4:0   NL   degraded missing_B_port                                         not_capable

64 4:5:0   NL   degraded missing_B_port                                         not_capable

65 4:6:0   NL   degraded missing_B_port                                         not_capable

66 4:8:0   NL   degraded missing_B_port                                         not_capable

67 4:9:0   NL   degraded missing_B_port                                         not_capable

68 4:10:0  NL   degraded missing_B_port                                         not_capable

69 4:12:0  NL   degraded missing_B_port                                         not_capable

70 4:13:0  NL   degraded missing_B_port                                         not_capable

71 4:14:0  NL   degraded missing_B_port                                         not_capable

72 4:16:0  NL   degraded missing_B_port                                         not_capable

73 4:17:0  NL   degraded missing_B_port                                         not_capable

74 4:20:0  NL   degraded missing_B_port                                         not_capable

75 4:21:0  NL   degraded missing_B_port                                         not_capable

76 3:0:0   FC   degraded missing_B_port                                         not_capable

77 3:1:0   FC   degraded missing_B_port                                         not_capable

78 3:10:0  FC   degraded missing_B_port                                         not_capable

79 3:11:0  FC   degraded missing_B_port                                         not_capable

80 3:12:0  FC   degraded missing_B_port                                         not_capable

81 3:13:0  FC   degraded missing_B_port                                         not_capable

82 3:14:0  FC   degraded missing_B_port                                         not_capable

83 3:15:0  FC   degraded missing_B_port                                         not_capable

84 5:0:0   FC   degraded missing_B_port                                         not_capable

85 5:1:0   FC   degraded missing_B_port                                         not_capable

86 5:10:0  FC   degraded missing_B_port                                         not_capable

87 5:11:0  FC   degraded missing_B_port                                         not_capable

88 5:12:0  FC   degraded missing_B_port                                         not_capable

89 5:13:0  FC   degraded missing_B_port                                         not_capable

90 5:14:0  FC   degraded missing_B_port                                         not_capable

91 5:15:0  FC   degraded missing_B_port                                         not_capable

92 0:10:0  FC   degraded missing_B_port                                         not_capable

93 0:11:0  FC   degraded missing_B_port                                         not_capable

94 0:12:0  FC   degraded missing_B_port                                         not_capable

95 0:13:0  FC   degraded missing_B_port                                         not_capable

96 0:14:0  FC   degraded missing_B_port                                         not_capable

97 0:15:0  FC   degraded missing_B_port                                         not_capable

98 6:10:0  FC   degraded missing_B_port                                         not_capable

99 6:11:0  FC   degraded missing_B_port                                         not_capable

100 6:12:0  FC   degraded missing_B_port                                         not_capable

101 6:13:0  FC   degraded missing_B_port                                         not_capable

102 3:8:0   FC   degraded missing_B_port                                         not_capable

103 6:15:0  FC   degraded missing_B_port                                         not_capable

104 1:11:0  NL   degraded missing_B_port                                         not_capable

105 1:15:0  NL   degraded missing_B_port                                         not_capable

106 1:19:0  NL   degraded missing_B_port                                         not_capable

107 1:23:0  NL   degraded missing_B_port                                         not_capable

108 4:11:0  NL   degraded missing_B_port                                         not_capable

109 4:15:0  NL   degraded missing_B_port                                         not_capable

110 4:19:0  NL   degraded missing_B_port                                         not_capable

111 4:23:0  NL   degraded missing_B_port                                         not_capable

112 0:8:0   FC   degraded missing_B_port                                         not_capable

--------------------------------------------------------------------------------------------

112 total

 

Checking cabling...

 

Found one or more discrepancies when examining cage connections:

 

Component ---------Description---------- Qty

Cabling   Missing node for cabling check   1

 

-Identifier- -- ------------------Description-------------------

node0        -- No cabling data for node0. Check status of node0

 

Checking cage firmware...

6 cages require firmware upgrades.

This will require approximately 60-120 minutes.

 

Checking system health...

Checking alert

Checking ao

Checking cabling

Checking cage

Checking cert

Checking dar

Checking date

Checking file

Checking fs

Checking host

Checking ld

Checking license

Checking network

Checking node

Checking pd

Checking pdch

Checking port

Checking qos

Checking rc

Checking snmp

Checking task

Checking vlun

Checking vv

Checking sp

Component ------------------------Summary Description-------------------------  Qty

Alert     New alerts                                                            327

Cabling   Missing node for cabling check                                          1

Cage      Cages missing B loop                                                    6

Cage      Cages not on current firmware                                           6

File      sr_mnt not mounted                                                      1

host      Hosts not seen by multiple nodes                                        6

LD        LDs in write through mode                                             324

LD        LDs without backup                                                    327

LD        Failed or degraded LDs                                                  1

LD        Preserved data storage space does not equal total node's Data memory    1

License   Licenses which have expired                                             1

Network   Too few working admin network connections                               1

Node      Nodes that are not online                                               1

PD        Unbalanced PD types in Cages                                           11

PD        Magazines that are being serviced                                       1

PD        PDs that are degraded                                                 111

QoS       Unable to check QoS                                                     1

Task      Failed Tasks                                                            2

VV        CPGs below growth increment threshold                                   1

-----------------------------------------------------------------------------------

       19 total                                                                1130

 

 

admithw has completed.

Luke_Y
Advisor

Re: 3Par 7200 -Interface Card 0 Failed (Interface Card Firmware Unknown {0x0})

Hello.

i have put admithw -checkonly output.

what are the steps to add the new Node to the cluster?

Luke_Y
Advisor

Re: 3Par 7200 -Interface Card 0 Failed (Interface Card Firmware Unknown {0x0})

i replaced with another node that has the same revision

when booting it i  get :Remote SSN mismatch node 1 expected SSN 18b0ab but got SSN 1934d8”

any ideas?

 

sbhat09
HPE Pro
Solution

Re: 3Par 7200 -Interface Card 0 Failed (Interface Card Firmware Unknown {0x0})

Hello @Luke_Y,

This is the ideal scenario when the new node is sourced from another system with different configuration or node id mismatch. The node you are trying to add has different serial number. So it is not joining the cluster with the existing node.

Enter CTRL+W to enter whack terminal. Or follow this procedure - step 1-4 https://support.hpe.com/hpesc/public/docDisplay?docId=sf000045573en_us&docLocale=en_US

In whack prompt, enter the command to set the 7 digit serial number as below ; 

set perm sys_serial=<7_digit_3PAR_serial> 

the serial number is set, type 'reset' at whack prompt and the node will reboot and join cluster.

But, I advise you to get it done by HPE support.

Regards,
Srinivas Bhat

If you feel this was helpful please click the KUDOS! thumb below!
Note: All of my comments are my own and are not any official representation of HPE.


I am an HPE Employee

Accept or Kudo

Luke_Y
Advisor

Re: 3Par 7200 -Interface Card 0 Failed (Interface Card Firmware Unknown {0x0})

hello @sbhat09 
thank you for the help 

i opened a ticket with support .

 

now everything is fine

the process was 

boot the node Ctrl+W to get into Whack
prom edit 
change the SN 
change the node (to 0 it was 1)
reset
after the reboot it still show the other
got into whack again 
ide erase 
when it booted again 
from node 1 startnoderescue -node 0
hope this will help to somebody in the future.

sbhat09
HPE Pro

Re: 3Par 7200 -Interface Card 0 Failed (Interface Card Firmware Unknown {0x0})

Great to know that the issue resolved @Luke_Y !!

Thanks for coming back and posting the final steps. I will tag the keywords so that it will be available for anyone else facing the same problem.

Regards,
Srinivas Bhat

If you feel this was helpful please click the KUDOS! thumb below!
Note: All of my comments are my own and are not any official representation of HPE.


I am an HPE Employee

Accept or Kudo