Server Tuning/Config help

[David Poe_2]

I have an HP RP7400 8-way, 32 GB RAM running HP UX 11.0. I believe this server is poorly configured, however, I know very little about HP "best practive" configuration and could use some help. The server runs several applications on it including 6, very active, 64 bit Oracle DB's. Below is some basic config info. I would appreciate any insights/help anyone can provide. TIA!!!

swapinfo -at
Kb Kb Kb PCT START/ Kb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 4194304 0 4194304 0% 0 - 1 /dev/vg00/lvol2
dev 1048576 0 1048576 0% 0 - 1 /dev/vg00/swap01
reserve - 5242880 -5242880
memory 27571964 24925764 2646200 90%
total 32814844 30168644 2646200 92% - 0 -

sysdef
NAME VALUE BOOT MIN-MAX UNITS FLAGS
acctresume 4 - -100-100 -
acctsuspend 2 - -100-100 -
allocate_fs_swapmap 0 - - -
bufpages 102400 - 0- Pages -
create_fastlinks 0 - - -
dbc_max_pct 2 - - -
dbc_min_pct 2 - - -
default_disk_ir 0 - - -
dskless_node 0 - 0-1 -
eisa_io_estimate 768 - - -
eqmemsize 143 - - -
file_pad 10 - 0- -
fs_async 0 - 0-1 -
hpux_aes_override 0 - - -
maxdsiz 86016 - 0-655360 Pages -
maxdsiz_64bit 262144 - 256-1048576 Pages -
maxfiles 2048 - 30-2048 -
maxfiles_lim 2048 - 30-2048 -
maxssiz 2048 - 0-655360 Pages -
maxssiz_64bit 2048 - 256-1048576 Pages -
maxswapchunks 4096 - 1-16384 -
maxtsiz 16384 - 0-655360 Pages -
maxtsiz_64bit 262144 - 256-1048576 Pages -
maxuprc 4338 - 3- -
maxvgs 80 - - -
msgmap 1263468544 - 3- -
nbuf 90937 - 0- -
ncallout 4836 - 6- -
ncdnode 150 - - -
ndilbuffers 30 - 1- -
netisr_priority -1 - -1-127 -
netmemmax 0 - - -
nfile 74358 - 14- -
nflocks 4820 - 2- -
ninode 40608 - 14- -
no_lvm_disks 0 - - -
nproc 4820 - 10- -
npty 60 - 1- -
nstrpty 60 - - -
nswapdev 25 - 1-25 -
nswapfs 10 - 1-25 -
public_shlibs 1 - - -
remote_nfs_swap 0 - - -
rtsched_numpri 32 - - -
sema 0 - 0-1 -
semmap 631701504 - 4- -
shmem 0 - 0-1 -
shmmni 512 - 3-1024 -
streampipes 0 - 0- -
swapmem_on 1 - - -
swchunk 2048 - 2048-16384 kBytes -
timeslice 1 - -1-2147483648 Ticks -
unlockable_mem 131072 - 0- Pages -

[Geoff Wild]

Why do you think the server is porrly configured?

Rgds...Geoff

[B. Hulst]

Hi,

Monitor the server's behavior with Glance for a few days... either interactive or by logging everything.

You will get more details IF the server is poorly configured or not.

Regards,
Bob

[Geoff Wild]

Another thing you can do - download and install System Health Check:

http://software.hp.com/portal/swdepot/displayProductInfo.do?productNumber=SHCBASE01

The lite version is free - and will give you some indication of any issues...

Rgds...Geoff

[David Poe_2]

The load on this server has jumped up tremendously in the past few months and because of that we've ran into several problems, some of which were config related. Things like nproc too low (we've raised from 1800 to 4820), added swap space, added CPU. I'm lacking several hours of sleep now, otherwise I'm sure I would also recall others. I'm trying to be more proactive about these types of issues rather than just putting out fires all the time on a server that has very small and defined maint windows. Until we can get the new replacement servers up and running, I'm trying to nurse this one as long as I can. Unfortunately, I am not very versed in HP UX.

[A. Clay Stephenson]

This sysdef output looks very hokey. Sema, shmem are both set to zero which has the effect of turning off these facilities -- which Oracle must have.

The most glaring problem I see is that timeslice is set to 1. As the system begins to load, the box will do context switches and very little else. The '1' setting is a legacy of a state-of-the-art stupid tuned parameter set that has been carried over. Set timeslice to 10 and leave it there.

Kmtune is a much better tools than sysdef to display the tunables. Man kmtune for details.

[David Poe_2]

Thank you for the info. Since you mentioned it, I thought I would also drop in the kmtune output.

kmtune
Parameter Value
===============================================================================
NSTRBLKSCHED 2
NSTREVENT 50
NSTRPUSH 16
NSTRSCHED 0
STRCTLSZ 1024
STRMSGSZ 65535
acctresume 4
acctsuspend 2
aio_listio_max 256
aio_max_ops 2048
aio_physmem_pct 10
aio_prio_delta_max 20
allocate_fs_swapmap 0
alwaysdump 0
bootspinlocks 256
bufcache_hash_locks 128
bufpages 102400
chanq_hash_locks 256
create_fastlinks 0
dbc_max_pct 2
dbc_min_pct 2
default_disk_ir 0
desfree 0
disksort_seconds 0
dnlc_hash_locks 64
dontdump 0
dskless_node 0
dst 1
eisa_io_estimate 0x300
eqmemsize 15
fcp_large_config 0
file_pad 10
fs_async 0
ftable_hash_locks 64
hdlpreg_hash_locks 128
hfs_max_ra_blocks 8
hfs_ra_per_disk 64
hpux_aes_override 0
initmodmax 50
io_ports_hash_locks 64
iomemsize 40000
km_disable 0
ksi_alloc_max (NPROC*8)
ksi_send_max 32
lotsfree 0
max_async_ports 50
max_fcp_reqs 512
max_mem_window 0
max_thread_proc 64
maxdsiz 0X15000000
maxdsiz_64bit 0x0000000040000000
maxfiles 2048
maxfiles_lim 2048
maxqueuetime 0
maxssiz 0x00800000
maxssiz_64bit 0x00800000
maxswapchunks 4096
maxtsiz 0x04000000
maxtsiz_64bit 0x0000000040000000
maxuprc ((NPROC*9)/10)
maxusers 600
maxvgs 80
mesg 1
minfree 0
modstrmax 500
msgmap (MSGTQL+2)
msgmax 32768
msgmnb 65535
msgmni (NPROC)
msgseg 32767
msgssz 128
msgtql (NPROC*4)
nbuf 0
ncallout (16+NPROC)
ncdnode 150
nclist (100+16*MAXUSERS)
ncsize (NINODE+VX_NCSIZE)
ndilbuffers 30
netisr_priority -1
netmemmax 0
nfile (15*NPROC+2048)
nflocks (NPROC)
nhtbl_scale 0
ninode (8*NPROC+2048)
nkthread (((NPROC*7)/4)+16)
nni 2
no_lvm_disks 0
nproc ((MAXUSERS*8)+20)
npty 60
nstrpty 60
nstrtel (MAXUSERS)
nswapdev 25
nswapfs 10
nsysmap ((NPROC)>800?2*(NPROC):800)
nsysmap64 ((NPROC)>800?2*(NPROC):800)
num_tachyon_adapters 0
o_sync_is_o_dsync 0
page_text_to_local 0
pfdat_hash_locks 128
public_shlibs 1
region_hash_locks 128
remote_nfs_swap 0
rtsched_numpri 32
scroll_lines 100
scs_table_max_size 500
scs_table_size 10
scsi_max_qdepth 8
scsi_maxphys 1048576
sema 1
semaem 16384
semmap (SEMMNI+2)
semmni (NPROC*2)
semmns (SEMMNI*2)
semmnu (NPROC-4)
semmsl_override 2048
semume 64
semvmx 32768
sendfile_max 0
shmem 1
shmmax 0X40000000
shmmni 512
shmseg 32
st_ats_enabled 0
st_fail_overruns 0
st_large_recs 0
st_san_safe 0
streampipes 0
swapmem_on 1
swchunk 2048
sysv_hash_locks 128
tcphashsz 0
timeslice 1
timezone 420
unlockable_mem (MAXUSERS*10)
vnode_cd_hash_locks 128
vnode_hash_locks 128
vps_ceiling 16
vps_chatr_ceiling 65536
vps_pagesize 4
vx_fancyra_enable 1
vx_maxlink 32767
vx_ncsize 1024
vx_ninode 0
vx_noifree 0
vx_nothrottle 1
vxfs_max_ra_kbytes 1024
vxfs_ra_per_disk 1024

[A. Clay Stephenson · Accepted solution]

Okay, at least you have semaphores and shared memory. The VERY FIRST thing you need to do is increase timeslice to 10. Next, I would increase bufpages to at least 204800 (800 MB) and possibly as high as 409600 (1600 MB) --- if you are using cooked files for Oracle otherwise you current setting of 400MB is okay. I think you are going to see amazing improvements from the timeslice change.

[Tim D Fulford]

Clay may well have got you 90% there with timeslice (the forum is littered with this advice, usually a thread or more a week!!)

My concern is disks, LAN and general IO. You mention system lod has shot up recently. This may be because your IO is choking the system. what do you get during a relatively busy period when you do "sar -du 300 1" ? is tehre any evedence of network bottlenecks ? "/usr/sbin/lanadmin; lan; dis; ;quit;"

Regards

Tim

[Ranjith_5]

Hi,

Attached the performance cookbook. This will help you.

pls find the PDF


regards,
Syam