Alpha 8.3 SAMBA CIFS V1.1 ECO1 PS9 errors

[Peter Barkas]

Running simple standalone server with share security accessed primarily by W2008 server.

Periodically (6 times in the last 61 days) these errors are logged and the W2008 server loses connection. The W2008 server is able to restablish connection later without restarting SAMBA CIFS.

[2010/04/26 01:06:54, 0] SAMBA$SRC:[SOURCE.NSSWITCH]WINBINDD_PASSDB.C;1:(129)
Possible deadlock: Trying to lookup SID S-1-1-0 with passdb backend
[2010/04/26 01:06:54, 0] SAMBA$SRC:[SOURCE.NSSWITCH]WINBINDD_PASSDB.C;1:(129)
Possible deadlock: Trying to lookup SID S-1-5-2 with passdb backend
[2010/04/26 01:06:54, 0] SAMBA$SRC:[SOURCE.NSSWITCH]WINBINDD_PASSDB.C;1:(129)
Possible deadlock: Trying to lookup SID S-1-1-0 with passdb backend
[2010/04/26 01:06:54, 0] SAMBA$SRC:[SOURCE.NSSWITCH]WINBINDD_PASSDB.C;1:(129)
Possible deadlock: Trying to lookup SID S-1-5-2 with passdb backend
[2010/04/26 01:06:54, 0] SAMBA$SRC:[SOURCE.SMBD]SERVICE.C;1:(1009)
'/SAMBA$ROOT/TMP' does not exist or permission denied when connecting to [IPC$] Error was no such file or directory
[2010/04/26 01:06:54, 0] SAMBA$SRC:[SOURCE.SMBD]SERVICE.C;1:(188)
chdir (dsa203:[jetform]) failed
[2010/04/26 01:06:59, 0] SAMBA$SRC:[SOURCE.SMBD]SERVICE.C;1:(188)
chdir (dsa203:[jetform]) failed
[2010/04/26 01:06:59, 0] SAMBA$SRC:[SOURCE.SMBD]SERVICE.C;1:(188)
chdir (dsa203:[jetform]) failed
[2010/04/26 01:07:12, 0] SAMBA$SRC:[SOURCE.SMBD]SERVICE.C;1:(188)
chdir (dsa203:[jetform]) failed

Config:

[global]
workgroup = PHBWORLD
server string = Samba %v running on %h (OpenVMS)
netbios name = HERKRO
security = SHARE
; hosts allow = 10.
guest account = jetformscif
log file = /samba$root/var/log_%h.%m
load printers = No
printcap name = /dev/null
ldap ssl = no
# supposedly not required if winbind is not used but errors are logged if not specified:
idmap uid = 15000-20000
idmap gid = 15000-20000
;
inherit owner = Yes
inherit vms rms protections = Yes
guest only = Yes
guest ok = Yes
vms rms format = streamlf
vfs objects = varvfc
[JF]
path = dsa203:[jetform]
read only = No
vms ods5 volume = No
hosts allow = 10.5.20.45,10.5.20.46,10.5.20.20

[JFIN]
path = dsa607:[jetform]
read only = No
vms ods5 volume = Yes
hosts allow = 10.5.20.46,10.5.20.45,10.5.20.20

[JFR]
path = dsa203:[jetform]
read only = Yes
vms ods5 volume = No
hosts allow = 10.

[JFRIN]
path = dsa607:[jetform]
read only = Yes
vms ods5 volume = Yes
hosts allow = 10.

[P Muralidhar Kini]

Hi Peter,

This looks similar/related to one of your previous posts.
http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=1400607

Is it?

Regards,
Murali

[Peter Barkas]

Murali thank you for the response.

The problem is similar insofar as the Winbind errors are still being logged.

However, that is not the concern here.

When this error is logged, the W2008 server is (temporarily) unable to connect to the share and this causes the remote application to fail:

[2010/04/26 01:06:54, 0] SAMBA$SRC:[SOURCE.SMBD]SERVICE.C;1:(1009)
'/SAMBA$ROOT/TMP' does not exist or permission denied when connecting to [IPC$] Error was no such file or directory

[P Muralidhar Kini]

Hi Peter,

>> '/SAMBA$ROOT/TMP' does not exist or permission denied when connecting
>> to [IPC$] Error was no such file or directory
Just to rule out any obvious errors, I hope the directory exists and has the
required permissions for access.

>> When this error is logged, the W2008 server is (temporarily) unable
>> to connect to the share and this causes the remote application to fail:
This seems to suggest when the server tries to connect to the
share again, it succeeds.
So the problem occurs with the directory but not always. Is that correct ?

Is the problem seen with the same directory always or with different
directories also ?

Regards,
Murali

[Peter Barkas]

Murali, thank you for the response.

Yes, most of the time the share works as expected and the error has happened only 6 times during 61 days.

The share is accessed every day albeit fairly infrequently as it is being used in a test environment until this problem is resolved.

The error has been logged only for the TMP directory.

[P Muralidhar Kini]

Hi Peter,

>> The error has been logged only for the TMP directory.
This makes me believe that the problem has something to do with the way
the TMP directory is accessed.

>> The share is accessed every day albeit fairly infrequently as it is
>> being used in a test environment until this problem is resolved.
Need to know how the TMP directory is accessed.
One possibility is if there is some other accessor of the directory who
may be deleting the directory while others are still accessing it.

How is the directory TMP accessed
- Multiple or single accessors ?
- What kind of operations do each accessor perform on the directory?
- Do any of accessor delete/rename the directory?

When the remote application reties after failure, is the directory TMP
empty to start with or does it have some files?

Regards,
Murali

[Peter Barkas]

The TMP directory was created as part of the SAMBA CIFS installation and has not been deleted since then.

It is referenced in
SAMBA$ROOT:[000000.BIN]SAMBA$SMBD_STARTUP.COM

$! Setup tmpdir
$ tmpdir = f$trnlnm("SAMBA$TMPDIR")
$ define TMP 'tmpdir' ! up to 1.9.16p2
$ define TMPDIR 'tmpdir' ! from 1.9.16p3 on (Andrew told me...)

It is not used directly by any of our applications.

I have never seen a file in the TMP directory.

$ sho log samba$tmpdir/fu
"SAMBA$TMPDIR" [exec] = "SAMBA$ROOT:[TMP]" (LNM$SYSTEM_TABLE)
$

[P Muralidhar Kini]

Hi Peter,

>> The TMP directory was created as part of the SAMBA CIFS installation
>> and has not been deleted since then.
This rules out any problem that may arise due to mutiple access to the
TMP directory.

>> I have never seen a file in the TMP directory.
This is strange. I thought this directory might get used to store some
temporary files at times.

Looks now like its a problem with the permissions associated with the
'/SAMBA$ROOT/TMP' directory.
Similar kind of problem discussed here -
http://www.linuxquestions.org/questions/linux-software-2/samba-share-problems-ipc$-343263/

Regards,
Murali

[Peter Barkas]

Murali

Yes, I saw that posting, however, it seems odd that the error occurs only periodically and that the share works at other times.

Also, the protection on the directory is w:rwed.

Directory SAMBA$ROOT:[000000]

TMP.DIR;1 [COMPUTER,SYSTEM] (RWED,RWED,RWED,RWED)
(DEFAULT_PROTECTION,SYSTEM:RWED,OWNER:RWED,GROUP:RWED,WORLD:RWED)

[Peter Barkas]

The other thing that I have noticed is that all 6 errors have occurred between 01:00 and 07:30.

It is extremely unlikely that file transfers are in progress in the test environment at these times.

There may be various housekeeping activities that run overnight, however, it is not clear why these might trigger this type of error.

[P Muralidhar Kini]

Hi Peter,

>> it seems odd that the error occurs only periodically
Yes, this is the intresting part. why is the problem showing up only some times and not always.

Time to catch up some sleep.
Will get back to you on this later.

Regards,
Murali

[Toine_1]

IHi Peter,

I'm also using SAMBA on I64 (VMS8.3-1H1). I have also end users that reports the same problem. They use a SAMBA Share but sometimes it doens't work. Then I start searching and the problem is solved automaticly.

For me it is also not clear why it works 98% of the time and sometimes (2%) not.

I use winbind.
I have changed some winbind settings and now it is better. (Now 1% of the time the end users can't use the Samba shares.)


/Toine

[P Muralidhar Kini]

Hi Peter,

Whenever the path for any share is omitted in the SMB.CONF file,
CIFS server select SAMBA$ROOT:[TMP] as the default path.
From the configuration that you have provided, the path has been
specified.

Toine,
>> For me it is also not clear why it works 98% of the time and
>> sometimes (2%) not.
Yes, Looks like a intermittent problem.

>> I use winbind.
>> I have changed some winbind settings and now it is better.
>> (Now 1% of the time the end users can't use the Samba shares.)
Can you let us know what settings you have changed in particular.
This may give us a clue as to what the problem may be.

Also this may be something that Peter can also make use of to reduce
the number of of occurrences of the problem.

Regards,
Murali

[Volker Halle]

Peter,

if you're interested in better understanding this problem and the circumstances it is happening in, how about extracting the sources and looking at [SOURCE.SMBD]SERVICE.C for those error messages.

You should find those messages in the source code and also see the code, which does some kind of tests, before deciding to issue those error messages. Maybe just looking at the tests in the source could help better understand this problem.

Nearly all the messages you've posted have been reported at the SAME time (01:06:54). Is it always like this ?

Which operation on the W2008 server is associated with this failing access ?

Volker.

[Paul Nunez]

Hi Peter,

Your best bet may be to increase the log level so the next time it occurs there are plenty of debug messages in the log to review. For this problem, I would set it as:

log level = 4

But notice the error about the /samba$root/tmp directory is associated with the IPC$ share. When a user establishes a session with a server by mapping the first drive to that server, Windows clients always establish an anonymous session and map the IPC$ share to get info from the server and a separate, non-anonymous session for the share the user is mapping.

Then the log shows there was a problem attempting a chdir to the dsa203:[jetform] directory which is the root of the JF share.

Is samba$root on dsa203: as well? Any chance dsa203: is somehow being taken off line during the wee morning hours?

Regards,

Paul Nunez

[Peter Barkas]

Volker

I did consider looking at the source and if I was working with, for example, a module that I had downloaded from CPAN I would have done so.

However, HP have got their name on this and so I prefer to remain detached from the developer layer.

[Peter Barkas]

Paul

I have taken your advice and changed the debug level.

Samba$root is on a different shadow set, dsa101.

As far as I am aware sets are 'never' taken offline, although they are 'split' for backups.

I suspect that there may be a clue in your description of the anonymous/non-anonymous connections and that perhaps access/security is getting mangled internally swapping between the two.

[P Muralidhar Kini]

Hi Peter,

>> However, HP have got their name on this and so I prefer to remain detached
>> from the developer layer.
You can consider forwading this query to HP through official support channel.
i.e. you can forward this question to the local HP customer support.
This way your query would be looked at by the SAMBA/CIFS team.

Regards,
Murali

[Peter Barkas]

Murali

I have had responses here from Shilpa in the past, but perhaps I will log a call.

[P Muralidhar Kini]

Hi Peter,

>> I have had responses here from Shilpa in the past, but perhaps I will log a
>> call.
Yes. Logging a call is a way to go.
I will also forward this query to shilpa or anybody else from the SAMBA/CIFS
team here, so as to get their opinion on this.

Regards,
Murali

[Shilpa K]

Hi Peter,

Following error will be logged after the CRTL reports that a particular directory to which the Windows client was trying to connect, was unavailable:

"'/SAMBA$ROOT/TMP' does not exist or permission denied when connecting to [IPC$] Error was no such file or directory".

So, it seems like /SAMBA$ROOT/TMP was temporarily unavailable.

As Paul indicated, increasing the debug level should help capture more debug information to analyze the problem. Also, could you please post the output of "$ dir/sec dsa203:[000000]jetform.dir" ?

Side note: as for Winbind related errors, CIFS V1.1 ECO1 PS010 patch set correctly recognizes the logical WINBIND_DONT_ENV and based on it, it can disable Winbind functionality. So, suggest you to install PS010 patch set. Also, you can refer the PS010 release notes for information about the logical WINBIND_DONT_ENV. CIFS V1.1 ECO1 PS010 patch set can be downloaded either from the standard CIFS patch set ftp location or from CIFS download page.

Regards,
Shilpa

[Peter Barkas]

Shilpa

Thank you for the response.

Debug level was increased to 4 as suggested but the error in question has not happened since doing so.

Directory DSA203:[000000]

JETFORM.DIR;1 13/387 21-MAY-2001 10:47:35.09 5-MAY-2010 12:28:53.00
[COMPUTER,WADE] (RWED,RWED,RWED,RWED)
(DEFAULT_PROTECTION,SYSTEM:RWED,OWNER:RWED,GROUP:RWED,WORLD:RWED)

Total of 1 file, 13/387 blocks.

I can't see a reference to PS010 at:

http://h71000.www7.hp.com/network/cifs_download.html

Regards

Peter

[Shilpa K]

Hi Peter,

The CIFS patch set for V1.1 ECO1 can be obtained from the link:

http://h71000.www7.hp.com/network/cifs_v1_1.html

You can click on the link under "Download Patches for Alpha and Integrity Servers" and it will take you to the ftp site where CIFS patch sets are available.

Regards,
Shilpa

[Peter Barkas]

Shilpa

Somewhat confusingly http://h71000.www7.hp.com/network/cifs_v1_1.html leads to a page containing information about T1.2.

Anyway I have downloaded the patch set and release notes for PS010.

The notes contain:

"37. WINBIND functionality is not disabled even if the logical WINBINDD_DONT_ENV is
defined system-wide to 1 by executing:

$ DEFINE/SYSTEM WINBINDD_DONT_ENV 1"

which seems to be contrary to what I understood from your posting.

Regards

Peter

[Shilpa K]

Hi Peter,

Sorry for the confusion. The CIFS patch set release notes describes the problems under "Problems fixed:" section and WINBIND_DONT_ENV logical must be defined to 1 as shown in the release notes in order for Winbind to be disabled.

Regards,
Shilpa