text outputs slowly to screen

[uk1]

Maybe I should know this, but I dont =( when cat'ing or more'ing a file the text outputs slowly. Possibly a buffer problem?

Thanks in advance

[Pete Randall]

This might be a symptom of memory pressure. Run "vmstat 1 5" and take a look at the "po" column to see if you're paging. Anything other than zero indicates some pressure - numbers above 10 indicate a real problem.


Pete

[Alzhy]

Your system has a lot of wayward processes or processes are getting spawned faster that they complete. This results in your VM (virtual memory) probably severely pressured.

Do the following to uncover the malaise in your system if you still can:

root# UNIX95=1 ps -efxH > processes.txt

And investigate processes.txt after you reboot your box if it is still rebootable or after you've forcefully rebooted it.

[uk1]

There is nothing paging. I will need to look at the output from ps to see if anything abnormal is happening.

[James R. Ferguson]

Hi David:

I'd look at a mismatched network speed/duplex setting as one possible reason.

Are there excessive errors seen with 'lanadmin'? Are you running at the correct LAN speed and duplex modes?

If you have 'glance', what kind of metrics does that show?; network, cpu, I/O, etc. etc.

What has changed in your environment?

Regards!

...JRF...

[Torsten.]

BTW, are you connected through network or terminal?
Just a question ;-)

[uk1]

Lanadmin reports no errors both ways. glance does report vxfsd crunching alot of the cpu time. Funny I never noticed it before.

[Alzhy]

Can you send us a snapshot of swapinfo -atm, glance and output of the posix "UNIX95 =1 ps -efH" ?

[uk1]

The reason swap is running at 35GB is because we were installing ovo. Was a rule complaining about it not being set to 1x memory, to avoid any warnings or errors, I set it temp.

TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 4096 0 4096 0% 0 - 1 /dev/vg00/lvol2
dev 35000 0 35000 0% 0 - 0 /dev/vgswap/lvol1
reserve - 1693 -1693
total 39096 1693 37403 4% - 0 -


Attaching processes, I havent had time to go through them all yet. Kinda of busy responding =)

[James R. Ferguson]

Hi (again) David:

'vxfsd' is the JFS (VxFS) daemon. It is perfectly normal for many to run. You didn't mention I/O rates, wait times, nor the ratio between physical and logical I/O.

Neither did you mention what has changed (if anything you know) and when the poor performance began.

Regards!

...JRF...

[uk1]

Sorry James, in regards to the system preforming poorly. It has been that way since it was installed a few days ago, and continues to be that way after many reboots and patch loads. There is very little activity regarding io. Very low wait times and percent busy.

[Alzhy]

Last few stats - please provide:


vmstat 5 10

sar 5 10

sar -b 5 10


What kind of machine do you have? OVO needs a pretty capable machine to run on. Also how much memory?

[James R. Ferguson]

Hi David:

Make sure that your kernel 'timeslice' is not set less than 10. If it is, it is likely that you are spending more time context switching than doing real work.

Regards!

...JRF...

[uk1]

# vmstat 5 10
procs memory page faults cpu
r b w avm free re at pi po fr de sr in sy cs us sy id
1 0 0 430737 4893412 29 9 0 0 0 0 0 1663 3523 393 1 1 98
1 0 0 431200 4893375 30 10 0 0 0 0 0 1691 2385 366 0 1 99
1 0 0 431200 4893434 12 3 0 0 0 0 0 1661 1967 320 0 1 99
1 0 0 435749 4892350 135 8 0 0 0 0 0 1640 16833 323 3 1 97
1 0 0 435749 4893434 44 1 0 0 0 0 0 1641 5990 317 0 1 99
1 0 0 437982 4893434 21 5 0 0 0 0 0 1651 2882 300 0 0 100
1 0 0 437982 4893434 7 0 0 0 0 0 0 1653 1558 355 0 0 99
1 0 0 426648 4893434 18 8 0 0 0 0 0 1682 6051 401 0 2 97
1 0 0 426648 4893118 103 41 0 0 0 0 0 1685 11124 532 3 2 95
2 0 0 409664 4893118 195 22 0 0 0 0 0 1647 22741 406 4 1 95

# sar 5 10

HP-UX stlap10p B.11.11 U 9000/800 10/24/05

15:32:01 %usr %sys %wio %idle
15:32:06 1 1 0 98
15:32:11 0 1 0 99
15:32:16 1 1 0 98
15:32:21 0 1 0 99
15:32:26 2 1 0 97
15:32:31 2 2 0 96
15:32:36 0 1 0 99
15:32:41 0 1 1 99
15:32:46 1 1 0 98
15:32:51 11 4 0 84

Average 2 1 0 97



# sar -b 5 10

HP-UX stlap10p B.11.11 U 9000/800 10/24/05

15:33:18 bread/s lread/s %rcache bwrit/s lwrit/s %wcache pread/s pwrit/s
15:33:23 0 23 100 7 12 44 0 0
15:33:28 0 1958 100 6 13 56 0 0
15:33:33 0 9 96 6 7 19 0 0
15:33:38 0 36 100 8 13 42 0 0
15:33:43 0 58 100 7 17 60 0 0
15:33:48 0 1052 100 11 44 76 0 0
15:33:53 0 3973 100 10 39 75 0 0
15:33:58 0 81 100 4 18 76 0 0
15:34:03 0 16 100 4 9 55 0 0
15:34:08 0 44 100 4 18 80 0 0

Average 0 725 100 7 19 66 0 0


# dmesg | grep -i physical
physical page size = 4096 bytes, logical page size = 4096 bytes
Physical: 33552384 Kbytes, lockable: 30967084 Kbytes, available: 30883148 Kbytes




# uname -a
HP-UX B.11.11 U 9000/800 unlimited-user license

# ioscan -fnk -Cprocessor
Class I H/W Path Driver S/W State H/W Type Description
===================================================================
processor 0 128 processor CLAIMED PROCESSOR Processor
processor 1 129 processor CLAIMED PROCESSOR Processor
processor 2 136 processor CLAIMED PROCESSOR Processor
processor 3 137 processor CLAIMED PROCESSOR Processor
processor 4 144 processor CLAIMED PROCESSOR Processor
processor 5 145 processor CLAIMED PROCESSOR Processor

Running 1Ghz according to fru dump

[Tim Sanko]

See how close you are to swapping. I might suggest fixing the size of the buffer cache.

max 5% min 5%

Tim

[uk1]

Thanks to everyone here for their suggestions, everything that was provided was helpful. It looks like there is no "magical" answer like we always hope there is. Appears at this point I will have to buckle down at look at preformance issues.

Thanks again,

-David

[Alzhy]

So you have a 32GB rp4440 with 6 1Ghz CPUs. I think you have an untuned Kernel that will take into consideration

(1) amount of memory used for buffer
(2) SHM requirements for youy "small?" oracle OV instance.

Try:

dbc_max_pct 7 dbc_min_pct 3 msgmap 11002 msgmax 8192 msgmnb 440000 msgmni 4096
msgseg 32767
msgssz 2200
msgtql 11000
sema 1
semaem 16384
semmap 4098
semmni 4096 semmns 8192
semmnu 1024
semmsl 2048
semume 512
semvmx 32767
shmem 1
shmmax 0x40000000
shmmni 1024
shmseg 1024
swapmem_on 1