topic Re: CIFS9000 question: broken pipe problem in Operating System - HP-UX

CIFS9000 question: broken pipe problem

Jiang Li_1 — Wed, 14 May 2003 23:43:24 GMT

Hi All,

We just upgraded our SAMBA to CIFS9000 A.01.09.04, and found a lot of below logs (/var/adm/syslog/syslog.log) as below:
------------------------------
May 15 07:26:30 SERVERNAME smbd[17462]: [2003/05/15 07:26:30, 0, pid=17462] lib/util_
sock.c:(499)
May 15 07:26:30 SERVERNAME smbd[17462]: write_socket_data: write failure. Error = B
roken pipe
-----------------------------

We don't have real issue with SAMBA, but I don't know why so many logs pop up.

Could anyone help us on this ? Thanks.

Re: CIFS9000 question: broken pipe problem

Steven E. Protter — Wed, 14 May 2003 23:58:36 GMT

This thread seems to deal with that error.

http://www1.itrc.hp.com/service/cki/docDisplay.do?docLocale=en_US&docId=200000062905687

Samba like everything nowadays needs lots of shared memory resources. Apparently in your setup, yo uneed more.

This is done in the kernel, shmmax,shmseg and such parameters.

ipcs command will also help.

I would be increasing that shared memory.

SEP

Re: CIFS9000 question: broken pipe problem

Jiang Li_1 — Fri, 16 May 2003 00:33:39 GMT

Hi Steven,

Thanks for your information !

I have the below questions when using the solution you pointed me:
1) How to find the shmkey and semkey by running ipcs ?
2) How to tune the kernel parameter such as shmmax, shmseg ... ? We are running HP-UX 11.00 or 11.11.

Thanks for your help !

Re: CIFS9000 question: broken pipe problem

malay boy — Fri, 16 May 2003 00:58:00 GMT

Hi Jiang Lee,
Out of your two question I only know one :

2) How to tune the kernel parameter such as shmmax, shmseg ... ? We are running HP-UX 11.00 or 11.11.
This can be done using SAM.

SAM -> Kernel COnfiguration -> Configurable Parameter.

To change go to ACTION->modify configurable parameter

I look at shmmax and I think it is using HEX.

Good luck.

for ipcs I normally used ipcs -mop to look for oracle shared memory and killed them ..sometimes...

I also interested to know what is answer 1.

regards
mB

Re: CIFS9000 question: broken pipe problem

malay boy — Fri, 16 May 2003 01:15:41 GMT

Ops ! one more thing..You need to reboot the server after changing the parameter.

regards
mB

Re: CIFS9000 question: broken pipe problem

Steven E. Protter — Fri, 16 May 2003 02:14:45 GMT

Answer to question 1.

Thanks to malay for handling #2.

1) How to find the shmkey and semkey by running ipcs ?

Output of the ipcs command.

s 30989 0xd1f1b4c4 --ra-r----- oracle dba
s44448270 0x00000000 --ra------- root sys

m 17 0xed19f7f0 --rw-r----- oracle dba
m 18 0xb54341b4 --rw-r----- oracle dba

To get rid of the first oracle shared memory segment, use the following command either as oracle or root

ipcrm -s 30989

To get rid of the last one:
ipcrm -m 18

I'm attaching my kmtune output in this post so you can see a working configuration on a server that has a samba, apache and a couple of oracle instances on it. This would be considered a generous setup for a machine with 2 G's of Ram, 4 G's swap and a busy samba config.

This is incomplete, but covers the relavent kernel parameters.

msgmax 32768
msgmnb 65535
msgmni (NPROC)
msgseg (MSGMAP+2)
msgssz 128
sema 1
semaem 16384
semmap (SEMMNI+2)
semmni (NPROC*5)
semmns (SEMMNI*2)
semmnu (NPROC-4)
semmsl_override 2048
semume 64
semvmx 32768
sendfile_max 0
shmem 1
shmmax 0X40000000
shmmni 512
shmseg 32

The fun part of doing the kernel modification is that sometimes when you change one thing, you find you need to change a dozen other things to satisfy dependencies.

Its better in those cirumstances to satisfy dependencies than messing with formulas most mortals know nothing about.

Apology for the delay.

regards,

SEP

Re: CIFS9000 question: broken pipe problem

Jiang Li_1 — Fri, 16 May 2003 06:40:23 GMT

Hi Malay & Steven,

Thanks for your help !

The below is my "ipcs" output, I can't find the shmkey and semkey used by SAMBA, Would you please give me a hand ?
-----
#ipcs
IPC status from /dev/kmem as of Fri May 16 15:36:46 2003
T ID KEY MODE OWNER GROUP
Message Queues:
q 0 0x3c0c1632 -Rrw--w--w- root root
q 1 0x3e0c1632 --rw-r--r-- root root
Shared Memory:
m 0 0x410c188f --rw-rw-rw- root root
m 1 0x4e0c0002 --rw-rw-rw- root root
m 2 0x410c0de4 --rw-rw-rw- root root
Semaphores:
s 0 0x410c188f --ra-ra-ra- root root
s 1 0x4e0c0002 --ra-ra-ra- root root
s 2 0x410c0de4 --ra-ra-ra- root root
s 3 0x00446f6e --ra-r--r-- root root
s 4 0x00446f6d --ra-r--r-- root root
s 5 0x01090522 --ra-r--r-- root root
s 6 0x610c07b7 --ra-ra-ra- root root
s 7 0x730c07b7 --ra-ra-ra- root root
s 8 0x700c07b7 --ra-ra-ra- root root
s 9 0x690c07b7 --ra-ra-ra- root root
s 10 0x750c07b7 --ra-ra-ra- root root
s 11 0x630c07b7 --ra-ra-ra- root root
s 12 0x640c07b7 --ra-ra-ra- root root
s 13 0x660c07b7 --ra-ra-ra- root root
s 14 0x6c0c07b7 --ra-ra-ra- root root
s 15 0x6d0c07b7 --ra-ra-ra- root root
s 16 0x6f0c07b7 --ra-ra-ra- root root
s 17 0x410c07d1 --ra-ra-ra- root root
s 18 0x520c07b7 --ra-ra-ra- root root
----------

Thanks

Re: CIFS9000 question: broken pipe problem

Jiang Li_1 — Fri, 16 May 2003 09:36:31 GMT

Hi Steven,

I would like ask you the below questions?
1) I don't hear any complain about the SAMBA usage from our engineers even with the log erros. I don't know if the log errors have any impact to users.

2) Is it a temporary solution by using ipcrm to remove the share memory key ? If we want to permanently eliminate such errors, We need increase the shared memory, is that right ?

Currently, our shmmax & shmseg have the below values:
shmmax=0x4000000
shmseg=120

I found that the above parameters are dynamic in hp-ux11.11. It means that they can be modified without reboot.

I don't know how much I should adjust for the above paramters. Is there any guideline or I just copy your values to our site ? We don't use Oracle in our system.

Thanks very much for your help !

Re: CIFS9000 question: broken pipe problem

Steven E. Protter — Fri, 16 May 2003 18:18:55 GMT

I'm going to paste in your questions to make it easier for me to answer them. Forgive the extra bytes.

>>
1) I don't hear any complain about the SAMBA usage from our engineers even with the log erros. I don't know if the log errors have any impact to users.
<<

A lot of error messages aren't going to effectthe users. During peak samba use times, you could get connections dropping, so these kind of errors are worth dealing with. I'm not getting them. I will post kernel data below for your reference.

>>
2) Is it a temporary solution by using ipcrm to remove the share memory key ? If we want to permanently eliminate such errors, We need increase the shared memory, is that right ?

Currently, our shmmax & shmseg have the below values:
shmmax=0x4000000
shmseg=120
<<

I think shutting down samba, using ipcrm and then starting samba will temporarily stop your error messages. The question is how long.

As far as kernel changes go, I don't know if that will fix it, but this is where I'd start.

Here are my values:
shmmax 4200000000 Y 4200000000
shmmni 4000 - 4000
shmseg 4000 Y 4000

In hex: shmmax 0xfa56ea00 0xfa56ea00
I'm running an oracle server with lots of demand for shared memory. You might not need to go this high.

>>
I found that the above parameters are dynamic in hp-ux11.11. It means that they can be modified without reboot.
<<
My mind is still stuck in 11.00 mode, but yest shmmax and shmseg should be changeable dynamically.

>>
I don't know how much I should adjust for the above paramters. Is there any guideline or I just copy your values to our site ? We don't use Oracle in our system.
<<

Since you don't use oracle you don't need to go as high as me. But you might be using other shared memory eaters. Apache, Softare AG adabas, sybase, informix some to mind.

The good news is that you can change these two guys dynamically, then stop and start samba and watch the error logs, all with minimal user disruption.

If it were my machine, I'd double the figures and keep doing that until either the problem goes away or you find out Steve was wrong and something else needs to be adjusted.

Good Luck.

If you play with this over the weekend, I will be checking the thread. Good Luck.

SEP

Re: CIFS9000 question: broken pipe problem

Jiang Li_1 — Sun, 18 May 2003 13:11:35 GMT

Hi Steven,

I try to double the shmmax on my HP-UX 11.0 box. But the error log still exist. In fact, The system is quite idle after I change the kernel parameter. And the error log occur in 10 minutes after reboot.

Do you have any idea on this ?

Thanks & Regards

Re: CIFS9000 question: broken pipe problem

Steven E. Protter — Sun, 18 May 2003 17:55:50 GMT

Is the error the same as before?

If so, keep bumping up shared memory a few more times. You really can't hurt anything by trying this.

Also belatedly, lets check the install state of this product and all others with the following command:

swverify \*

If there is a problem involving the samba install swremove it and re-install it. If there is a problem with patches, consider reinstalling.

DO NOT reinstall PHKL_18543

DO NOT reinstall PHKL_18543

DO NOT reinstall PHKL_18543

It will make huge problems on your system.

Post any section of the swverify output that catches your eye.

I will check back later today.

SEP

Re: CIFS9000 question: broken pipe problem

Jiang Li_1 — Mon, 19 May 2003 00:29:23 GMT

Hi Steven,

I can't find any clue from swverify output. Anyway, I will try to reinstall SAMBA in a later time and increase the shmmax value appropriately.

Thanks very much for your kindly help ! I want to hold this problem for the time being.

Really appreciate your help!