Slow response on new install. Tuning help.

[Toby Ragsdale]

Hello all,

New to this form, and little HP-UX experience, so looking for some hand holding on a new install.

We purchased a HP-UX11 box for a DB app. I had the VAR do the install, and tuning. I dont have a very good feeling about some of the kernel parms he set, and the performance of the box is very poor.

Currently, ~15 users are accessing one app on this box, and seeing normal response for most of the time. Dramatic slowdowns occur several times per day. I cannot run sam while users are attached, or the users app comes to a complete standstill. Other slowdowns occur when a user performs one of several 'file builds' from the db app.

So thats the background. What I have found thus far;

VAR set kernel parms for memory;
Dbc_max=80
Dbc_min=5
Nbuf=5256
Bufpages=21024
swaprm_on=1

Q1) From everything I can glean from this forum, we are NOT using dbc_max/min as nbuf and bufpages are set. I am trying to set these to 0 now, and change dbc_max to 30 to start. I have seen recommendations in the 10-15 range, but small steps. Is this right?

Q2) vhand is running in the 4-5% range. High? This is symptom of memory starvation?

Q3) And please help on this one, caus it creeps me out. I have changed the kernel parms, rebuilt the kernel, and sam sits FOREVER. What am I doing wrong.

[Rajeev Shukla]

Yes the dbc_max should be set to 10 thats the best value.
Also keep a copy of the current kernal parameters and apply the tuned templates (OLTP Database base setver), i am sure you'll get a better performance that what u are getting currently and then tune other parameters desired, like nfile, max_users, nproc etc..
Use sar to see the disk io, memory bottle neck.

Rajeev

[Toby Ragsdale]

Rajeev,

Thanks. I was looking at the templates.

I was just trying the dbc_max/min, and get this error from sam when building kernel.

UNEXPECTED EXIT: process LANG=C LC_ALL=C /usr/sam/lbin/samx -C -p
13485 -s kc_sa_param /usr/sam/lib//kc.ui exited with a non-zero exit
status.
sh: 13593 Bus error(coredump)
Got any thoughts?

[Michael Steele_2 · Accepted solution]

Sounds like some SAM patch issues so use this for a new kernel instead.

cd /stand/build
/usr/lbin/sysadm/system_prep -s /stand/build/system
vi /stand/build/system

mk_kernel -s /stand/build/system
kmupdate
cp /stand/system /stand/system_old
cp /stand/build/system /stand/system
cd /
shutdown -r now

####################################333

And attach the results of these commands after making changes. Try to the normal state of the server. Idle won't help.

sar -u 5 5
sar -v 5 5
sar -d 5 5
sar -b 5 5
vmstat 5 5
swapinfo -tam

[Michael Steele_2]

Sounds like some SAM patch issues so use this for a new kernel instead.

cd /stand/build
/usr/lbin/sysadm/system_prep -s /stand/build/system
vi /stand/build/system

mk_kernel -s /stand/build/system
kmupdate
cp /stand/system /stand/system_old
cp /stand/build/system /stand/system
cd /
shutdown -r now

####################################333

And attach the results of these commands after making changes. Try to use the normal state of the server. Idle won't help.

sar -u 5 5
sar -v 5 5
sar -d 5 5
sar -b 5 5
vmstat 5 5
swapinfo -tam

[Toby Ragsdale]

Michael,

Did as you suggested. When I vi /stand/build/system, its empty. Should I put the kernel changed in this file? I see a file system.SAM that HAS my changes plus several others.

Getting the sam patch now, but I would like to try the manual method, as sam and I are not getting along.

Thanks
TBR

[Michael Steele_2]

Regarding "...When I vi /stand/build/system, its empty..."

Sorry, this can't be. Please rerun:

cd /stand/build
/usr/lbin/sysadm/system_prep -s /stand/build/system

You should see kernel values in the top half of the system file and device drivers in the bottom.

With great care, try this command for an example of the system file:

# more /stand/system

All you're doing is making a copy of this with 'system_prep'.

[Toby Ragsdale]

Dont know if I was sleeping this AM, but your correct. Did it twice, and had the same result, but after your post, again and it was there. Well, could be typxcing.

OK, I made the changes in the system file, ran mk_kernel -s /stand/build/system and get this;

Generating module: krm...
Compiling conf.c...
Loading the kernel...

Its been sitting for ~15 min. Should it be done? What if it doesn't complete?

Just as a note, from another fourm post hint, deleted system.SAM and changed parms in sam again. It recreated the exact same file with the same results. I cant seem to find the sam patch.

Thanks again...
TBR

[Toby Ragsdale]

30min and sitting at 'loading the kernel'. Got any ideas? I need a trouble free monday.

[Michael Steele_2]

I'd reinstall. This isn't suppose to happen. Use a complete install. A cold, complete. Not an upgrade. And choose the Advanced option.

A cold install takes maybe two hours.

NOTE: Use these file system sizes for a 9 gb or 18 gb disk, but go smaller for a 4 gb boot disk. You can extend later.

To install from a CDROM verify that you have the right cd, 'core', and insert.

You can't do anything at the ISL so recycle the power and interrupt the boot up when the 10 second option is displayed.

This will bring you to the bch or boot console handler.

From here you want to find the core o/s install so run 'sea ipl' or just 'sea'.
'sea' will display every device attached while 'sea ipl' will display only bootable devices, like the cdrom.

You'll get a report with P0, P1... in the left most side of the display, reading across you'll note the HW address and a description of the device.

'bo P1' or whatever corresponds to the cdrom, or 'bo HW PATH'. I.e., 'bo 10/5/2.0'.
Chose the installation option when the menu is dsiplayed and then select 'media only' and 'advanced installation' options.

At this point you'll be filling in data fields. You'll need hostname, ip, gateway ip, ntp ip, nis and dns ip, as well as the file system sizes for the core O/S. This depends on the size of your disk so I'll give you file system sizes for a 4gb disk.

/dev/vg00/lvol1 - /stand - 256 mb
/dev/vg00/lvol2 - swap - 2048 mb
/dev/vg00/lvol3 - / - 256 mb
/dev/vg00/lvol4 - home - 200 mb
/dev/vg00/lvol5 - /opt - 1024 mb
/dev/vg00/lvol6 - /tmp - 512 mb
/dev/vg00/lvol7 - /usr - 1024 mb
/dev/vg00/lvol8 - /var - 1536 mb

256 + 2048 + 256 + 200 + 1024 + 512 + 1024 + 1536 = 6856 mb or 6.9 gb.

This will work on a 9 gb boot disk.

[Michael Steele_2]

Note: "...This depends on the size of your disk so I'll give you file system sizes for a 4gb disk...."

Should read...

This depends on the size of your disk so I'll give you file system sizes for a 9 gb disk..."

[Toby Ragsdale]

Wow. No no wait, I dont think that captured my feelings right nowâ ¦ WOW!
At least as fast as a speeding locomotive, Ill have to wait for my users to see if it makes it to superman speed.


Michael, Rajeev. THANKS!


I will run sar â x 5 5 tomorrow when I have a load on the server, and post.

OK, now some forum questions. How does the point system work on this forum? How about carriage returns? I put them in, but donâ t see them in MY posts. Michaels are readable, but I look like a mess.

WOWWOW

[Toby Ragsdale]

The mk_kernel took 1.5hr. Ill see how it looks tomrrow before reinstalling, but I think it may be a good idea. I think my vendor realy started me off with a poor install.

[Michael Steele_2]

Sir, '..mk_kernel..." take 30 seconds.


If you've got error's post them. Exactly. But one and one half hour's for a new kernel on a newly install O/S is a big, big problem.

What happens when new patches are release every 6 months?

I'd be willing to bet that your server is in an 'UNKNOWN STATE' and unsupportable by HP.

Think about that before blowing off a re-install.

[Bill Hassell]

As mentioned, a cold reinstall is the best choice. You could spend hundreds of hours trying to fix problems a. It sounds like the install has no patches and poor choices for kernel params. NOTE: A cold install takes just under 2 hours. However, don't even think about continuing without patches. Use the latest SupportPlus CDROM or download the HWE, QPK *and* DIAGNOSTICS from http://www.software.hp.com/SUPPORT_PLUS/

As mentioned, there are very big problems with SAM and kernel building that never happen in a normal system install.

[Steven E. Protter]

Here is a great tuning document.

http://www1.itrc.hp.com/service/cki/search.do?category=c0&docType=Security&docType=Patch&docType=EngineerNotes&docType=BugReports&docType=Hardware&docType=ReferenceMaterials&docType=ThirdParty&searchString=UPERFKBAN00000726&mode=id&admit=-682735245+1066010574084+28353475&searchCrit=allwords&printable=true

dbc_max_pct and dbc_min_pct should be low figures and close together. It is very expensive in terms of cpu to change that setting. Mine are 5 and 7.

This system seems messed up enough to require a new cold install as Bill recommends.

One thing I noticed on a recent cold install was SD-UX ran like a one legged dog. To fix it, I downloaded and installed all SD-UX patches I could get my hands on. That was after cold install, June 2003 Service Pack, HWE and the big bundle that comes on the Core OS CD.

SEP

[Toby Ragsdale]

Mr. [sir] Steele,
I should hope that it takes less than 1.5 hrs to build a FILE on my BRAND SPANKING NEW box. I think I got a bad installer. I dont want to bash them (him) too bad till I have a bit more info under my belt. I can tell you I have had this happen to me on NetWare systems I have installed just to be vindicated later. So, Ill just say, I hope there (his) next victim has your help.


Bill,
Thanks for joining the conversation.
Thanks for the link. I will patch. I would wish for a normal install now. Just goes to show, dont ever trust, learn it your self.
Will you and Michael help with the fresh install? Never done it, dont know where to start.


Steven,
Read the doc. That was part of my basis for blame on the bad install. Just dont get how this tech could have come up with some of the settings. He must have had a thought. He definitely has more experience with the OS than I (paid $200/hr). It took no more than a couple searches on this forum, four people to confirm, and my users SCREAMING their heads off to find the correct settings. Im working my way down to your (and everyone else on this forum) 5 â 10 suggestion.


So to all. My next steps are.
1)Watch performance through next week.
2)Research (mostly on the reinstall) (please help)
3)Reinstall over next weekend.



P.S.
Michael. No visible errors. Where should I look?

P.P.S
Not kidding about the . How is everyone doing it? I have tryed 2x return. I have tryed + +013 My posts looks like sh**t. Whats the HP fourm secret squirrel co

[Michael Steele_2]

I, as well as several forum members will be online all day to assist you.

Begin your reinstall and when a question pops up post it.

Reemeber these keys on the keyboard for control:

tab
arrows
enter / return

Note 1: When installing in the TUI if you miss a data field just keep tabbing until you circle around back to it.

Note 2: When entering in the file system sizes use the 'MODIFY' option.

[Michael Steele_2]

Lets begin with you CD media. Please list the part number and title so it can be verified. List all the cd's please, you may need to order new ones.

[Bill Hassell]

As far as a fresh install, all you need is the Core Install CDROM. Start the boot process but interrupt it when you see the message about 10 seconds to interrupt. You'll need the hardware address of the CDROM but you can use the SEA command to find it. You'll get a list of disks, one of which will be the CDROM. (hint: use ioscan -kfCdisk and put the result on paper so you can see all your disks and the CD). Also get a bdf from the current system or search the forums here for suggested mountpoint sizes for a new install.

The SEA command will show you several P codes as in P0 P1 P2 ... Use the appropriate P-number to select the CD and boot from it:

boot p3

and say NO to interact with IPL. After a minute or two, you'll get the main menu. Select the Interactive install option (tab key and then spacebar to select), then OK. Using the arrow keys, tab key and spacebar, got through each of the menus tabs, especially System setup and Disk layout. Once completed, the rest is automatic. (side note: blhconsulting at mindspring.com)

Bill's guide to formatting in the forums:

The secret to providing a blank line is: space return. Just a will be removed automatically (like multiple spaces which are truncated to one space) No fix for this...I use underscores to space this out a bit.

And same for scripts...leading spaces are always deleted.

For complex scripts where spacing is important, always attach a file. NEVER attach .doc .ppt .xls or other specialized files. That's where Notepad is VERY useful.

Note also that text attachments always have a few leading spaces on line 1 so make line 1 a single space.