topic Re: BIG Performance Problems in Operating System - HP-UX

BIG Performance Problems

Nicolas Gagnon — Wed, 16 Oct 2002 13:37:43 GMT

Hi,

I have 2 L2000 4CPU 2GIG RAM (HP-UX 11i) connected to 9TB SAN (VA7400) via Brocade switch. These 2 servers are SAMBA server only (2.2.3). I have about 25 users per server. filesystems have about 1TB. We have special need. We proces about 2-4 TB of stuff per day!

Sice 2 weeks, I've had major performance problems. Proccessors are 100% busy! Each SMBD processes takes large percentages. I've started glance+ and found that each SMBD processes spend all of the time in system calls (mostly read and select). I haven't change anything for last year...

Is there any survey about SAMBA and system calls.

I've attached a screen shot of glance+ system call output.

thanks.

nicolas Gangon

Re: BIG Performance Problems

John Poff — Wed, 16 Oct 2002 13:50:23 GMT

Hi Nicolas,

Has anything changed in your environment in the last two weeks? Patched your systems? Updated your application that uses the Samba shares? Any updates to the Windows that your users are on?

Here is a thread about a performance problem with Samba 2.2.5 and a workaround, but I don't know if it will help you.

http://forums.itrc.hp.com/cm/QuestionAnswer/1,,0x4415237a4bc6d611abdb0090277a778c,00.html

JP

Re: BIG Performance Problems

Nicolas Gagnon — Wed, 16 Oct 2002 14:15:56 GMT

Since the problem have occured we've putted some patches and upgraded SAMBA. but we didn't do anything before.

Re: BIG Performance Problems

James Murtagh — Wed, 16 Oct 2002 14:19:17 GMT

Hi Nicholas,

I don't think anyone can give you a magical answer for this, you get have to dig deep into the problem, using glance, sar etc. There are some good performance cookbooks on the ITRC I believe. Basically you have to find what subsystem(s) is causing the problem. Perhaps use the Performance Advisor window from gpm?

From what you have said, the 100% CPU utilisation could possibly be red herring. If nothing but Samba is running on these servers, these figures could be caused by a number of other performance problems. If nothing has changed on the servers, and you are processing a lot of files per day, it could possibly be down to fragmentation of the filesystems. This would certainly get worse over time.

I assume these are vxfs version 4 filesystems as you are on 11i? Do you have Online JFS? If so, I would be interested in seeing a fragmentation report of one of the filesystems :

fsadm -F vxfs -E -D

Also, check all the usual system logs for any hardware problems.

Regards,

James.

Re: BIG Performance Problems

rakotoarisoa — Thu, 17 Oct 2002 07:02:56 GMT

Hi Nicolas,

Here are somes ways you can use in order to identify where your pb can come from:
* What change(s) did you made since the last time you can remember?

* Try to locate where the bottlenecks are:
i/o
memory
swap space
switch
network components
application(s)

Goog luck!

Re: BIG Performance Problems

harry d brown jr — Thu, 17 Oct 2002 10:19:15 GMT

One, consider MORE memory!

What are your kernel parameters set to?

and what does your swap look like?

How many fibre channel's per L2000 to the switch do you have?

How are the lv's built? Size of lun's? striping?

live free or die
harry

Re: BIG Performance Problems

Paula J Frazer-Campbell — Thu, 17 Oct 2002 11:17:13 GMT

Hi

From just your glance screen I would take a serious look at your disks during the busy periods.

sar -d 1 100

Also glance and examine lvols and then disks.

Paula

Re: BIG Performance Problems

Rory R Hammond — Thu, 17 Oct 2002 17:25:00 GMT

My samba log file is
located at
/var/opt/samba/log.smb

I would look there first.

I would then look to see what data files are being accessed at the time of your problems
(find /director -exec fuser -u {} \;

To see if anything is unusual.

I would also check for grins what my network load is. At one site I luckly solved samba performance by simply putting my server on a 10/100 switch.

good luck