more memory & more processors, means slower night processing

[Jeff Hagstrom]

I have an RP 7410 running hpux 11.11. Recently we added 2 more processors for a total of 4 and 4 more gig of ram, totaling 10 gig. Since the addition, everything seems to run SLOWER. Besides just adding those parts, is there anything in SAM that needs to be updated? What would cause the system to be so slow? TOP shows 1 of the 4 processors getting any time, the runns about 755 and the other 3 are at 98%. The system isn't using it's resources.

[J. Bravo]

Hello:

Please, send us the output of swapinfo -tam, top, vmstat.

Regards;

J. Bravo.

[Geoff Wild]

What is dbc_max_pct and dbc_min_pct set to?

With 10 GB of ram - I would set to 6 or 7 for max and 5 for min.

Is this server running Oracle? If yes - did the SGA increase to take advantage of the extra memory?

Rgds...Geoff

[TwoProc]

I believe that there are optimal configurations for those cell boards. This means that ram can be put in them that does not follow the HP recommended pattern for installation. This means that while certain memory layout patterns may work, they may be sub-optimal for memory performance. Did you have a look at the proper "filling" matrix to see that 10G of ram is supported in the manner that you installed it, and did you check that it was also an optimal solution? Some combinations, while they may work, aren't supported at all, while others are listed as sub-optimal. I usually have my HP service tech vet all my memory layout plans for me before I start buying, to make sure that it conforms to his paperwork on supported and recommended configurations.

[Jeff Hagstrom]

vmstat

[Jeff Hagstrom]

top

[Jeff Hagstrom]

tam

[Jeff Hagstrom]

dbc_max_pct = 50
dbc_min_pct = 5

This isn't Oracle, it is PROGRESS 9

[Geoff Wild]

Your dbc_max_pct is still at default - fine if you only had 1 GB of ram - but not with 10 gb - means it can reserve 5GB of ram for buffer cache!

Rgds...Geoff

[Jeff Hagstrom]

What should my dbc_max_pct and dbc_min_pct be set to?

Also, my swap:
mbytes available 4096 of 4096
percent available 100%
mbytes used 0 of 4096
percent used 0%
current priority 1
enabled now/on boot
swap minimum 0
swap maximum 0

It doesn't look like swap is being used?

[Geoff Wild]

I would go with 7 for max and leave min at 5.

Rgds...Geoff

[A. Clay Stephenson · Accepted solution]

You don't want swap to be used. You do want to make certain that you have enough swap so that all of your memory can be used for processes. This means that the total of device and filesystem swap + 75% of your RAM (if pseudoswap is enabled) is at least equal to your total physical memory. This will the system to reserve process space for all of your physical memory.

On an 11.11 box will 10GiB of memory, I would pin the buffer cache at about 1200 MiB by setting bufpages=307200. This will convert you to a static buffer cache and overall system performance becomes more predictable.

At that point, you then need to gather more metrics and we can then determine where the bottlenecks lie. You may have made a classic mistake of adding CPU's and memory when those weren't the actual constraints on the system. Also, unless an application is coded to be a multi-threaded process, additional CPU's will not help at all. Of course, more CPU's will help distribute the general processing load but if almost all of your processing is being done by a single-threaded application then more CPU's won't help at all.

[Geoff Wild]

Thought I'd also post the kernel parms of one of my remaining 11.11 systems:

# kmtune
Parameter Current Dyn Planned Module Version
===============================================================================
NSTRBLKSCHED - - 2
NSTREVENT 50 - 50
NSTRPUSH 16 - 16
NSTRSCHED 0 - 0
STRCTLSZ 1024 - 1024
STRMSGSZ 65535 - 65535
acctresume 4 - 4
acctsuspend 2 - 2
aio_listio_max 256 - 256
aio_max_ops 2048 - 2048
aio_physmem_pct 10 - 10
aio_prio_delta_max 20 - 20
allocate_fs_swapmap 0 - 0
alwaysdump 1 - 1
bcvmap_size_factor 2 - 2
bootspinlocks - - 256
bufcache_hash_locks 128 - 128
bufpages 0 - (NBUF*2)
chanq_hash_locks 256 - 256
core_addshmem_read 0 Y 0
core_addshmem_write 0 Y 0
create_fastlinks 0 - 0
dbc_max_pct 5 - 5
dbc_min_pct 4 - 4
default_disk_ir 0 - 0
desfree - - 0
disksort_seconds 0 - 0
dmp_rootdev_is_vol 0 - 0
dmp_swapdev_is_vol 0 - 0
dnlc_hash_locks 512 - 512
dontdump 0 - 0
dskless_node - - 0
dst 1 - 1
effective_maxpid - - ((NPROC<=30000)?30000:(NPROC*5/4))
eisa_io_estimate - - 0x300
enable_idds 0 - 0
eqmemsize 15 - 15
executable_stack 0 - 0
fcp_large_config 0 - 0
file_pad - - 10
fs_async 0 - 0
ftable_hash_locks 64 - 64
gvid_no_claim_dev 0 - 0
hdlpreg_hash_locks 128 - 128
hfs_max_ra_blocks 8 - 8
hfs_max_revra_blocks 8 - 8
hfs_ra_per_disk 64 - 64
hfs_revra_per_disk 64 - 64
hp_hfs_mtra_enabled 1 - 1
hpux_aes_override - - 0
initmodmax 50 - 50
io_ports_hash_locks 64 - 64
iomemsize - - 40000
ksi_alloc_max 33600 - 33600
ksi_send_max 32 - 32
lotsfree - - 0
max_async_ports 50 - 50
max_fcp_reqs 512 - 512
max_mem_window 128 - 128
max_thread_proc 3000 - 3000
maxdsiz 0x40000000 - 0X40000000
maxdsiz_64bit 0x400000000 - 0X400000000
maxfiles 4096 - 4096
maxfiles_lim 4096 Y 4096
maxqueuetime - - 0
maxssiz 134217728 - 134217728
maxssiz_64bit 0x40000000 - 0X40000000
maxswapchunks 16384 - 16384
maxtsiz 0x4000000 Y 0X4000000
maxtsiz_64bit 0x40000000 Y 0X40000000
maxuprc 3780 Y 3780
maxusers 256 - 256
maxvgs 256 - 256
mesg 1 - 1
minfree - - 0
modstrmax 500 - 500
msgmap 4202 - (2+MSGTQL)
msgmax 8192 Y 8192
msgmnb 65535 Y 65535
msgmni 4200 - 4200
msgseg 32767 - 32767
msgssz 96 - 96
msgtql 4200 - 4200
nbuf 0 - 0
ncallout 10016 - 10016
ncdnode 150 - 150
nclist 8292 - 8292
ncsize 35648 - 35648
ndilbuffers 30 - 30
netisr_priority - - -1
netmemmax - - 0
nfile 189090 - 189090
nflocks 4096 - 4096
nhtbl_scale 0 - 0
ninode 35648 - 35648
nkthread 10000 - 10000
nni - - 2
no_lvm_disks 0 - 0
nproc 4200 - 4200
npty 60 - 60
nstrpty 60 - 60
nstrtel 60 - 60
nswapdev 10 - 10
nswapfs 10 - 10
nsysmap 8400 - ((NPROC)>800?2*(NPROC):800)
nsysmap64 8400 - ((NPROC)>800?2*(NPROC):800)
o_sync_is_o_dsync 1 - 1
page_text_to_local - - 0
pfdat_hash_locks 128 - 128
public_shlibs 1 - 1
region_hash_locks 128 - 128
remote_nfs_swap 0 - 0
rtsched_numpri 32 - 32
scroll_lines 100 - 100
scsi_max_qdepth 8 Y 8
scsi_maxphys 1048576 - 1048576
select_enh 0 Y 0
sema 1 - 1
semaem 16384 - 16384
semmap 4098 - (SEMMNI+2)
semmni 4096 - 4096
semmns 8192 - 8192
semmnu 4196 - 4196
semmsl 2048 Y 2048
semume 100 - 100
semvmx 32767 - 32767
sendfile_max 0 - 0
shmem 1 - 1
shmmax 17179869184 Y 17179869184
shmmni 512 - 512
shmseg 200 Y 200
st_ats_enabled 1 - 1
st_fail_overruns 0 - 0
st_large_recs 0 - 0
st_san_safe 0 - 0
streampipes 0 - 0
swapmem_on 1 - 1
swchunk 4096 - 4096
sysv_hash_locks 128 - 128
tcphashsz 0 - 0
timeslice 10 - (100/10)
timezone 420 - 420
unlockable_mem 0 - 0
vas_hash_locks 128 - 128
vnode_cd_hash_locks 128 - 128
vnode_hash_locks 128 - 128
vol_checkpt_default 10240 - 10240
vol_default_iodelay 50 - 50
vol_fmr_logsz 4 - 4
vol_max_bchain 32 - 32
vol_max_nconfigs 20 - 20
vol_max_nlogs 20 - 20
vol_max_nmpool_sz 4194304 - (4*1024*1024)
vol_max_prm_dgs 1024 - 1024
vol_max_rdback_sz 4194304 - (4*1024*1024)
vol_max_vol 8388608 - (8*1024*1024)
vol_maxio 256 - 256
vol_maxioctl 32768 - 32768
vol_maxkiocount 2048 - 2048
vol_maxparallelio 256 - 256
vol_maxspecialio 256 - 256
vol_maxstablebufsize 256 - 256
vol_min_lowmem_sz 524288 - (512*1024)
vol_mvr_maxround 256 - 256
vol_rootdev_is_vol 0 - 0
vol_subdisk_num 4096 - 4096
vol_swapdev_is_vol 0 - 0
vol_vvr_transport 1 - (1)
volcvm_cluster_size 16 - 16
volcvm_smartsync 1 - 1
voldrl_max_drtregs 2048 - 2048
voldrl_min_regionsz 512 - 512
voliomem_chunk_size 65536 - (64*1024)
voliomem_maxpool_sz 4194304 - (4*1024*1024)
voliot_errbuf_dflt 16384 - 16384
voliot_iobuf_default 8192 - 8192
voliot_iobuf_limit 131072 - 131072
voliot_iobuf_max 65536 - 65536
voliot_max_open 32 - 32
volraid_rsrtransmax 1 - 1
vps_ceiling 64 - 64
vps_chatr_ceiling 1048576 - 1048576
vps_pagesize 4 - 4
vx_fancyra_enable 0 - 0
vx_maxlink 32767 - 32767
vx_ncsize 1024 - 1024
vx_ninode 0 - 0
vxfs_max_ra_kbytes 1024 - 1024
vxfs_ra_per_disk 1024 - 1024
vxtask_max_monitors 32 - 32


Rune kmtune to check yours.

Rgds...Geoff