Simpler Navigation for Servers and Operating Systems
Completed: a much simpler Servers and Operating Systems section of the Community. We combined many of the older boards, so you won't have to click through so many levels to get at the information you need. Check the consolidated boards here as many sub-forums are now single boards.
Operating System - Tru64 Unix
cancel
Showing results for 
Search instead for 
Did you mean: 

ES80 (NUMA) and Pset Effect on Kernel/Tru64 OS threads/Resources.

Chris Ellam
Advisor

ES80 (NUMA) and Pset Effect on Kernel/Tru64 OS threads/Resources.

I am trying to understand whether or not PSETs are really applicable for an ES80 and whether or not their usage can affect the OS behaviour etc...

We have an ES80 with 8 cpus (4 boxes), I/O is distributed between the 4 boxes, i.e. Ggabit is in box 0, Fibre channel in box 1 and 2, OS disk in box 1 etc... There are 2 psets, 0) cpu 0 & 1 and 2) cpu 2,3,4,5,6,7. The Os and application net comms run in pset 0, the application in pset 2. The application does a lot of calculations and also disk i/o both to the local disk and fibre disks. The apps net comms receives lots of data and puts this in shared memory mailboxes for the app. In turn the app put net response in shared memory mailboxes for the apps net comms task to send to the remote systems.

I do not see any kernel threads or Os daemon threads in pset 2. This makes me assume that for doing any i/o from pset 2 is that it must route this to the to a thread on cpu 0 or 1 that in turn will route the i/o back to the respective box.

Can you explain to me how this works and any experiences that might help.

Thanks, Chris
3 REPLIES
Hein van den Heuvel
Honored Contributor

Re: ES80 (NUMA) and Pset Effect on Kernel/Tru64 OS threads/Resources.

Good questions Chris, but hard to answer without a ton more information.

I find the Pset solution generally too exclusive, and found more happyness in rad based runon or similar commands.

Was the decision to run with psets well documented? Re-read it!
Was the pset decision perhaps based on a Wildfire GS80 or GS160, and carried forward to the Marvel architecture based? It may no longer be relevant.

If this was my problem, then I would get ahold of Xmesh to evaluate the rad based CPU and memory usage, and inter-rad traffic.

http://h30097.www3.hp.com/manage/xmesh/

To answer a similar question for an other (Oracle) application I once wrote a tool to picture the memory pages usage and distribution for a process. That was posted in:
http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=644238

fwiw,
Hein.
Hein van den Heuvel
Honored Contributor

Re: ES80 (NUMA) and Pset Effect on Kernel/Tru64 OS threads/Resources.

I just happened to run into a 'best practice' document which may help you some:

"Using a Processor Set to Reserve CPUs for a
Key Application"

http://h30097.www3.hp.com/docs/best_practices/BP_PSETS/psets_bp.pdf


fwiw,
Hein.

Chris Ellam
Advisor

Re: ES80 (NUMA) and Pset Effect on Kernel/Tru64 OS threads/Resources.

Hi Hein, many thanks for your answers. I'd forgotten about xmesh, what a pity that it doesn't allow the collection of the statistics too. The pset best practice is ok for the GS80/160/320 when we had QBBs. For the ES80/GS1280 I have the feeling that it's missing some details.