Performance issue on HP-UX

[Yaboto]

I have an HP Unix server running Jbase / t24 banking application.
The End of Day or Close of Business usually takes 15 minutes.
For no apparent reasons it can jump to 5 heures at time.

Can someone suggeste a tool I can use to address this situation ?
What are the possible commands I can be running over a period of time to identify any performance bottlenecks ?

[Dennis Handly]

There is of course glance to check to see whether your bottleneck is CPU, memory, I/O or networking. Are there other new processes running during this time?

[Michael Steele_2]

Here is SEP's script. SEP is another Forum member. I added the last two.

#!/usr/bin/sh
# posix shell
# system.perf.sh gather performance data

RUNSEC=$1

#############################
Server Info
#############################
uname -a > /tmp/HP_info
model >> /tmp/HP_info
uptime >> /tmp/HP_info
cat /etc/fstab >> /tmp/HP_info
lvlnboot -v >> /tmp/HP_info
vgdisplay -v >> /tmp/HP_info
lanscan >> /tmp/HP_info
ioscan -fn >> /tmp/HP_info
cat /etc/inittab >> /tmp/HP_info
crontab -l >> /tmp/HP_info
cat /stand/system >> /tmp/HP_perf_info
dmesg >> /tmp/HP_info


##########################
Metrics that can be put into a 15 - 20 minute cron, run for a week and then check for growth, bottlenecks, etc.
##########################
bdf >> /tmp/HP_info
swapinfo -ta >> /tmp/HP_info
/usr/bin/top -f /tmp/HP_perf_info
nohup sar -qM 1 $RUNSEC > /tmp/HP_perf_info.report &
nohup sar -uM 1 $RUNSEC > /tmp/HP_perf_info.cpu &
nohup sar -b 1 $RUNSEC > /tmp/HP_perf_info.buffer &
nohup sar -d 1 $RUNSEC > /tmp/HP_perf_info.disk &
nohup sar -w 1 $RUNSEC > /tmp/HP_perf_info.swap &
nohup sar -v 1 $RUNSEC > /tmp/HP_perf_info.inode &
nohup /usr/bin/vmstat -d -n 1 $RUNSEC > /tmp/HP_perf_vmstat &
sysdef >> /tmp/HP_perf_info

UNIX95=1 ps -ef -o vsz,pid,ppid,state,wchan,comm | sort -rn | head -10 >> outfile

UNIX95=1 ps -ef -o pcpu,pid,ppid,state,wchan,comm | sort -rn | head -10 >> outfile

[Raj D.]

Yaboto,

In a quick and first shot take a look at the system load, and if you see any thing higher than normal load the time taking 5hr ( than the regular 15 min) is obvious.

May be the server is loaded with some other jobs / batch etc at the same time.

Use # uptime
To see the load factor. if it is greater than 1, that means the server is loaded more than 100 %.

- You can then use # glance to figure out which resource is hitting the limit.
- or # vmstat 4 5 # See virtual memory performance and run queue and idle cpu.
- # sar -d 4 5 # Look for avwait and service time , is it high?. that means high i/o happening, Storage side needs to be check.
- # UNIX95=1 ps -e -o pcpu,pid,ppid,args | sort -rn | head -20 # Check top cpu processes. And use vsz (instead pcpu) for memory processes.

After initial findings trouble shooting can be done for the performance problem.

Cheers,
Raj.




* Remember point assignment (a kind of Appreciation). Someone has to remind you that your point assignment rate is way too low:
"I have assigned points to 159 of 429 responses to my questions."