ES80 (NUMA) and Pset Effect on Kernel/Tru64 OS threads/Resources.

[Chris Ellam]

I am trying to understand whether or not PSETs are really applicable for an ES80 and whether or not their usage can affect the OS behaviour etc...

We have an ES80 with 8 cpus (4 boxes), I/O is distributed between the 4 boxes, i.e. Ggabit is in box 0, Fibre channel in box 1 and 2, OS disk in box 1 etc... There are 2 psets, 0) cpu 0 & 1 and 2) cpu 2,3,4,5,6,7. The Os and application net comms run in pset 0, the application in pset 2. The application does a lot of calculations and also disk i/o both to the local disk and fibre disks. The apps net comms receives lots of data and puts this in shared memory mailboxes for the app. In turn the app put net response in shared memory mailboxes for the apps net comms task to send to the remote systems.

I do not see any kernel threads or Os daemon threads in pset 2. This makes me assume that for doing any i/o from pset 2 is that it must route this to the to a thread on cpu 0 or 1 that in turn will route the i/o back to the respective box.

Can you explain to me how this works and any experiences that might help.

Thanks, Chris

[Hein van den Heuvel]

Good questions Chris, but hard to answer without a ton more information.

I find the Pset solution generally too exclusive, and found more happyness in rad based runon or similar commands.

Was the decision to run with psets well documented? Re-read it!
Was the pset decision perhaps based on a Wildfire GS80 or GS160, and carried forward to the Marvel architecture based? It may no longer be relevant.

If this was my problem, then I would get ahold of Xmesh to evaluate the rad based CPU and memory usage, and inter-rad traffic.

http://h30097.www3.hp.com/manage/xmesh/

To answer a similar question for an other (Oracle) application I once wrote a tool to picture the memory pages usage and distribution for a process. That was posted in:
http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=644238

fwiw,
Hein.

[Hein van den Heuvel]

I just happened to run into a 'best practice' document which may help you some:

"Using a Processor Set to Reserve CPUs for a
Key Application"

http://h30097.www3.hp.com/docs/best_practices/BP_PSETS/psets_bp.pdf


fwiw,
Hein.

[Chris Ellam]

Hi Hein, many thanks for your answers. I'd forgotten about xmesh, what a pity that it doesn't allow the collection of the statistics too. The pset best practice is ok for the GS80/160/320 when we had QBBs. For the ES80/GS1280 I have the feeling that it's missing some details.