RP8420 MEM problem

[Lexxx]

I have rp8420 server with 32 CPUs and 128G memories. with two partitions
I checked IN-ALL that all are configured & active. in both partition 0 and 1.
After boot up I tried to do top command and check dmesg for partion 1 was 16 CPUs but only 32MB. But for partition0 it was 16CPUs and64G.
Is there a way how to enable the missing 32G. I check it and it was really in configured and active. please help.

[Murat SULUHAN]

Hi Lexxx

could you submit vparstatus and vparstatus -v command outputs

I assume you are using vPar

Best Regards
Murat

[Torsten.]

If you don't have vPars running, execute and post this:

# parstatus

[Lexxx]

i checked it via parstatus and grepping memory
twsmp8:/root#parstatus
Note: No action specified. Default behavior is display all.
[Complex]
Complex Name : Server1
Complex Capacity
Compute Cabinet (4 cell capable) : 1
Active MP Location : cabinet 0
Original Product Name : server rp8420
Original Serial Number : DEH4736E3F
Current Product Order Number : A6912A
OEM Manufacturer :
Complex Profile Revision : 1.0
The total number of partitions present : 2



[Cabinet]
Cabinet I/O Bulk Power Backplane
Blowers Fans Supplies Power Boards
OK/ OK/ OK/ OK/
Cab Failed/ Failed/ Failed/ Failed/
Num Cabinet Type N Status N Status N Status N Status MP
=== ============ ========= ========= ========== ============ ======
0 4 cell slot 21/0/N+ 6/0/N+ 6/0/N+ - Active

Notes: N+ = There are one or more spare items (fans/power supplies).
N = The number of items meets but does not exceed the need.
N- = There are insufficient items to meet the need.
? = The adequacy of the cooling system/power supplies is unknown.
HO = Housekeeping only; The power is in a standby state.
NA = Not Applicable.


[Cell]
CPU Memory Use
OK/ (GB) Core On
Hardware Actual Deconf/ OK/ Cell Next Par
Location Usage Max Deconf Connected To Capable Boot Num
========== ============ ======= ========= =================== ======= ==== ===
cab0,cell0 Active Core 8/0/8 32.0/0.0 cab0,bay0,chassis0 yes yes 0
cab0,cell1 Active Core 8/0/8 32.0/0.0 cab0,bay0,chassis1 yes yes 1
cab0,cell2 Active Base 8/0/8 32.0/0.0 - no yes 0
cab0,cell3 Active Base 8/0/8 32.0/0.0 - no yes 1

Notes: * = Cell has no interleaved memory.

twsmp8:/root#grep -i Physical /var/adm/syslog/syslog.log
Jan 2 23:31:18 twsmp8 vmunix: physical page size = 4096 bytes, logical page size = 4096 bytes
Jan 2 23:31:18 twsmp8 vmunix: Physical: 33534976 Kbytes, lockable: 27150492 Kbytes, available: 31169184 Kbytes
twsmp8:/root#

[Don Morris_1]

I hope you only used top for cpu counts, since it has no real relation to total physical memory.

OS version would be nice, as would the dmesg lines you believe give evidence of the problem.

Could you also report the result of:

parstatus -C
parstatus -V -p 0
parstatus -V -p 1

Thanks.

[Lexxx]

Looks like you have point as i see this interleave mem. Please let me know how to change this to 64. thanks a lot

twsmp8:/root#parstatus -C


[Cell]
CPU Memory Use
OK/ (GB) Core On
Hardware Actual Deconf/ OK/ Cell Next Par
Location Usage Max Deconf Connected To Capable Boot Num
========== ============ ======= ========= =================== ======= ==== ===
cab0,cell0 Active Core 8/0/8 32.0/0.0 cab0,bay0,chassis0 yes yes 0
cab0,cell1 Active Core 8/0/8 32.0/0.0 cab0,bay0,chassis1 yes yes 1
cab0,cell2 Active Base 8/0/8 32.0/0.0 - no yes 0
cab0,cell3 Active Base 8/0/8 32.0/0.0 - no yes 1

Notes: * = Cell has no interleaved memory.
twsmp8:/root#parstatus -V -p 1
[Partition]
Partition Number : 1
Partition Name : partition1
Status : Active
IP Address :
Primary Boot Path : 1/0/0/2/0.6.0
Alternate Boot Path : 1/0/0/3/0.6.0
HA Alternate Boot Path : 0/0/0/0/0/0/0/0.0
PDC Revision : 24.1
IODCH Version : 88e0
Cell Architecture : PA-RISC
CPU Compatibility : BCE-0
CPU Speed : 1100 MHz
Core Cell : cab0,cell1
Core Cell Choice [0] : cab0,cell1
Total Good Memory Size : 64.0 GB
Total Interleave Memory: 32.0 GB
Total Requested CLM : 32.0 GB
Total Allocated CLM : 32.0 GB


[Cell]
CPU Memory Use
OK/ (GB) Core On
Hardware Actual Deconf/ OK/ Cell Next Par
Location Usage Max Deconf Connected To Capable Boot Num
========== ============ ======= ========= =================== ======= ==== ===
cab0,cell1 Active Core 8/0/8 32.0/0.0 cab0,bay0,chassis1 yes yes 1
cab0,cell3 Active Base 8/0/8 32.0/0.0 - no yes 1

Notes: * = Cell has no interleaved memory.


[Chassis]
Core Connected Par
Hardware Location Usage IO To Num
=================== ============ ==== ========== ===
cab0,bay0,chassis1 Active yes cab0,cell1 1



twsmp8:/root#parstatus -V -p 0
[Partition]
Partition Number : 0
Partition Name : partition0
Status : Active
IP Address :
Primary Boot Path : 0/0/0/2/0.6.0
Alternate Boot Path : 0/0/0/3/0.6.0
HA Alternate Boot Path : 0/0/0/3/1.6.0
PDC Revision : 24.1
IODCH Version : 88e0
Cell Architecture : PA-RISC
CPU Compatibility : BCE-0
CPU Speed : 1100 MHz
Core Cell : cab0,cell0
Total Good Memory Size : 64.0 GB
Total Interleave Memory: 64.0 GB
Total Requested CLM : 0.0 GB
Total Allocated CLM : 0.0 GB


[Cell]
CPU Memory Use
OK/ (GB) Core On
Hardware Actual Deconf/ OK/ Cell Next Par
Location Usage Max Deconf Connected To Capable Boot Num
========== ============ ======= ========= =================== ======= ==== ===
cab0,cell0 Active Core 8/0/8 32.0/0.0 cab0,bay0,chassis0 yes yes 0
cab0,cell2 Active Base 8/0/8 32.0/0.0 - no yes 0

Notes: * = Cell has no interleaved memory.


[Chassis]
Core Connected Par
Hardware Location Usage IO To Num
=================== ============ ==== ========== ===
cab0,bay0,chassis0 Active yes cab0,cell0 0



twsmp8:/root#

[Lexxx]

Looks like you have point as i see this interleave mem. Please let me know how to change this to 64. thanks a lot
BTW both are HP.B.11.11

twsmp8:/root#parstatus -C


[Cell]
CPU Memory Use
OK/ (GB) Core On
Hardware Actual Deconf/ OK/ Cell Next Par
Location Usage Max Deconf Connected To Capable Boot Num
========== ============ ======= ========= =================== ======= ==== ===
cab0,cell0 Active Core 8/0/8 32.0/0.0 cab0,bay0,chassis0 yes yes 0
cab0,cell1 Active Core 8/0/8 32.0/0.0 cab0,bay0,chassis1 yes yes 1
cab0,cell2 Active Base 8/0/8 32.0/0.0 - no yes 0
cab0,cell3 Active Base 8/0/8 32.0/0.0 - no yes 1

Notes: * = Cell has no interleaved memory.
twsmp8:/root#parstatus -V -p 1
[Partition]
Partition Number : 1
Partition Name : partition1
Status : Active
IP Address :
Primary Boot Path : 1/0/0/2/0.6.0
Alternate Boot Path : 1/0/0/3/0.6.0
HA Alternate Boot Path : 0/0/0/0/0/0/0/0.0
PDC Revision : 24.1
IODCH Version : 88e0
Cell Architecture : PA-RISC
CPU Compatibility : BCE-0
CPU Speed : 1100 MHz
Core Cell : cab0,cell1
Core Cell Choice [0] : cab0,cell1
Total Good Memory Size : 64.0 GB
Total Interleave Memory: 32.0 GB
Total Requested CLM : 32.0 GB
Total Allocated CLM : 32.0 GB


[Cell]
CPU Memory Use
OK/ (GB) Core On
Hardware Actual Deconf/ OK/ Cell Next Par
Location Usage Max Deconf Connected To Capable Boot Num
========== ============ ======= ========= =================== ======= ==== ===
cab0,cell1 Active Core 8/0/8 32.0/0.0 cab0,bay0,chassis1 yes yes 1
cab0,cell3 Active Base 8/0/8 32.0/0.0 - no yes 1

Notes: * = Cell has no interleaved memory.


[Chassis]
Core Connected Par
Hardware Location Usage IO To Num
=================== ============ ==== ========== ===
cab0,bay0,chassis1 Active yes cab0,cell1 1



twsmp8:/root#parstatus -V -p 0
[Partition]
Partition Number : 0
Partition Name : partition0
Status : Active
IP Address :
Primary Boot Path : 0/0/0/2/0.6.0
Alternate Boot Path : 0/0/0/3/0.6.0
HA Alternate Boot Path : 0/0/0/3/1.6.0
PDC Revision : 24.1
IODCH Version : 88e0
Cell Architecture : PA-RISC
CPU Compatibility : BCE-0
CPU Speed : 1100 MHz
Core Cell : cab0,cell0
Total Good Memory Size : 64.0 GB
Total Interleave Memory: 64.0 GB
Total Requested CLM : 0.0 GB
Total Allocated CLM : 0.0 GB


[Cell]
CPU Memory Use
OK/ (GB) Core On
Hardware Actual Deconf/ OK/ Cell Next Par
Location Usage Max Deconf Connected To Capable Boot Num
========== ============ ======= ========= =================== ======= ==== ===
cab0,cell0 Active Core 8/0/8 32.0/0.0 cab0,bay0,chassis0 yes yes 0
cab0,cell2 Active Base 8/0/8 32.0/0.0 - no yes 0

Notes: * = Cell has no interleaved memory.


[Chassis]
Core Connected Par
Hardware Location Usage IO To Num
=================== ============ ==== ========== ===
cab0,bay0,chassis0 Active yes cab0,cell0 0



twsmp8:/root#

[Murat SULUHAN · Accepted solution]

Hi Lexx


Please look at following url
http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=1174660

Best Regards
Murat

[Don Morris_1]

Yup, I think that's the problem. ccNUMA (and hence Cell Local Memory) support wasn't until around 11.22 or so. I don't think the 11.11 kernel can see the CLM at all, you'll want to reconfigure whichever Cell(s) is/are configured with CLM to 100% ILM to run 11.11 on it.

Check:

parstatus -V -c 1
parstatus -V -c 3

to make sure of the configuration (if I read the parstatus output correctly, if either of those was 100% CLM it would be marked with an asterisk... so I think you'll see a 50% configuration in each Cell... but don't take my word for it...).

As noted in the other thread (thanks Murat):

parmodify -p 1 -m 0/1::y:ri:0
parmodify -p 1 -m 0/3::y:ri:0

shutdown -R

should do the trick, but it may not work (don't know why it didn't in the other thread, but they said it didn't and just deleted the Cell anyway... at which point it became rather moot). If it doesn't, shutdown partition 1 and delete/re-add the cells with the 0% CLM configuration.

[Lexxx]

Thanks for feedback looks like i need to delete and re-add the cells.
Can send me the command to reconfigure this.
I encounter an error.
twsmp8:/root#parmodify -p 1 -m 0/1::y:ri:0
Note: Using default absolute notation for the specified cell local memory value 0.
Warning: The target partition does not support cell local memory, therefore the specified cell local memory will be ignored.
twsmp8:/root#parmodify -p 1 -m 0/3::y:ri:0
Note: Using default absolute notation for the specified cell local memory value 0.
Warning: The target partition does not support cell local memory, therefore the specified cell local memory will be ignored.

twsmp8:/root#parstatus -V -p 1
[Partition]
Partition Number : 1
Partition Name : partition1
Status : Active
IP Address :
Primary Boot Path : 1/0/0/2/0.6.0
Alternate Boot Path : 1/0/0/3/0.6.0
HA Alternate Boot Path : 0/0/0/0/0/0/0/0.0
PDC Revision : 24.1
IODCH Version : 88e0
Cell Architecture : PA-RISC
CPU Compatibility : BCE-0
CPU Speed : 1100 MHz
Core Cell : cab0,cell1
Core Cell Choice [0] : cab0,cell1
Total Good Memory Size : 64.0 GB
Total Interleave Memory: 32.0 GB
Total Requested CLM : 32.0 GB
Total Allocated CLM : 32.0 GB


[Cell]
CPU Memory Use
OK/ (GB) Core On
Hardware Actual Deconf/ OK/ Cell Next Par
Location Usage Max Deconf Connected To Capable Boot Num
========== ============ ======= ========= =================== ======= ==== ===
cab0,cell1 Active Core 8/0/8 32.0/0.0 cab0,bay0,chassis1 yes yes 1
cab0,cell3 Active Base 8/0/8 32.0/0.0 - no yes 1

Notes: * = Cell has no interleaved memory.