SAMBA/CIFS issue

[Timothy P Walsh]

We have SAMBA/CIFS version 1.1 on Itanium 8.3 OpenVMS. We are copying files to the servers from widows environment. Then processing the file by a inhouse utility. We are finding that the file being writen to OpenVMS disk appears not to be write locked while being written through SAMBA. We were using pathworks till this week and our utility picked up that theer was a lock against teh new file being written, thus left the file alone. Now the utility is picking up 0 block files, it appears the file is being written to at the time and there is a typ ahead buffer or cache instead of block written file. Is there anyway to force SAMBA to accept writes without buffering, thus direct writes no buffering, so the file is complete before it is copied ?

I hope somone can help ?

Regards

Tim Walsh

[P Muralidhar Kini]

Hi Tim,

>> We are finding that the file being writen to OpenVMS disk appears not to be
>> write locked while being written through SAMBA.
Is there any particular command that you have used to find out that locks are
not taken on the file during the copy operation?

>> Now the utility is picking up 0 block files
The file copy from the windows to VMS server is in progress and hence the file
should be locked during this time.

The inhouse utility is finding that the file operation is in progress and hence
picking up only zero block files. The file content is still not completely updated
to the file.

>> Is there anyway to force SAMBA to accept writes without buffering, thus direct
>> writes no buffering, so the file is complete before it is copied
It depends on the copy operation as to what lock it takes on the file.
i.e. what mode. The lock mode might prevent any other utility from accessing
the file. Something like a Exclusive lock will prevent others from accessing the
file.

Secondly, it would be good if the inhouse utility gets the complete updated file
rather than a intermediate copy of the file.
Does getting a intermediate copy of the file cause any problems to the inhouse
utility?

Regards,
Murali

[Hoff]

Here's some generic reading on the locking mechanisms:

http://www.samba.org/samba/docs/man/Samba-HOWTO-Collection/locking.html

http://www.snia.org/tech_activities/CIFS/CIFS-TR-1p00_FINAL.pdf

Check your Samba logs.

Check testparm.

Check the CIFS sources, too, as there used to be a do_file_lock call that was pretty mellow in the CIFS source pool. (I haven't looked at the 1.1 code.) The older code allowed everything; there was no locking.

My guess is that CIFS isn't honoring the Windows locking mechanisms here, and isn't detecting the sharable flags correctly for your needs or isn't propagating the lock status over into the VMS locks.

Which means you'll probably want to move to a scheme where you rename the files into visibility (to your VMS pieces) within the share and avoid the issue you're encountering.

Or detect the zero-block file and come back in a few minutes. If your file format has an integrity check, run it. If not, consider adding it and running it.

There are several knobs around deny locks and oplocks that *might* be applicable, but I'd tend to go for the simplest available approach here. Potentially as far as removing CIFS from the sequence and sftp or ftp the files over to VMS, or (depending on the application) serve the data up on-line. Which is the approach I'd likely take here. You'll still have to contend with files in flight and the locks that result (or with a database update), but you'll have fewer cogs spinning.

[lokanath bagh]

Let me think in simple way -
>>We were using pathworks till this week and our utility picked up that theer was a lock against teh new file being written, thus left the file alone.
Does it mean that our utility was getting file currently locked by another user and not touching that file?
>>Now the utility is picking up 0 block files, it appears the file is being written to at the time and there is a typ ahead buffer or cache instead of block written file.
Now your utility is able to open the file but and able to read it with no data. Is this the case?

Lokanath

[Ross Perri]

Hi all,
My name is Ross and I am one of the application developers working with Tim.
Let me apologize in advance for the length of my post.

To answer some of the questions:
1. Does getting a intermediate copy of the file cause any problems to the inhouse
utility?
ANS: Yes. It will cause problems. We need to ensure that the file transfer to the Samba share is complete before processing.

2. At present, we have no control over the application that moves the files to the Samba share. It is a Windows application that uses FTP to transfer the files directly to the input directory of the application that consumes the data files. The files are copied without a rename.
The application that consumes these files is a COBOL application running under VMS on an Integrity server. The application attempts to open the file for sole access (ALLOWING NO OTHERS). If it detects that the file is locked by another user then it will not process the file.

It looks like the COBOL application is not detecting a lock on the file while the file is being copied to the Samba share and therefore processes the file while it is still empty.

From some of the reading I have done, my hunch is that it may have to do with the 'oplocks' parameter in CIFS/Samba that defaults to 'ON'. I believe that turning 'oplocks' OFF may resolve this issue because the locking would then be handled on the server (not at the client). I have also read recommendations that oplocks be turned off in a mixed environment such as ours (Windows applications and VMS/COBOL applications accessign the same share).

Do you believe that this could solve our issue?

Regards,
Ross

[P Muralidhar Kini]

Hi Ross,

From you response, the requirement is clear.
You dont want the in-house utility to process the files which are being modified.

OPLOCKS(opportunistic locking) is a file locking protocol to improve
performance by controlling caching of files on the client. I am not sure whether
it can be used inorder to achive mutual exclusion.

However to know whether the file is in use or not, may be you can try to get a
exclusive lock on it. Only if this succeds then the inhouse utility can proceed
to process the file.

May be you can give this a try.

Regards,
Murli

[Ross Perri]

Hi Murli,

Thanks for the reply. Just to clarify, that is exactly what we are doing. We attempt to open the file for exclusive access (i.e. exclusive lock). However, it seems that the exclusive lock is successful even though the file is still in the process of being copied across.
Which is why I am suggesting turning oplocks off. As you emntioned in your post, oplocks '...improve performance by controlling caching of files on the client'. I believe this is what we need to avoid. The caching on the client may be why we end up with zero-block files.

Regards,
Ross

[P Muralidhar Kini]

Hi Ross,

>> However, it seems that the exclusive lock is successful even though the file
>> is still in the process of being copied across.
Hmmm. This is intresting.

Even though the files may be getting cached in the client, the operation on the
file is not complete. So i would have expected the exclusive mode lock to fail
as the file is currently being operated upon.

>> Which is why I am suggesting turning oplocks off.
Doing this would prevent caching of files on the client, but i am not sure
whether this would solve the problem.

Looks like its worth a try. May be you can give it a try and check if the problem
goes away.

Regards,
Murali

[Volker Halle]

Tim, Ross,

may I suggest, that you log a call with HP to try to get to the real CIFS engineers and thus reduce speculation about what behaviour of CIFS does not match the previous behaviour as seen with Pathworks/Advanced Server. HP is certainly very interested to make CIFS behaviour like Pathworks - whenever possible.

Volker.

[John Gillings]

just testing to make sure ITRC works before (re)typing a response...