when cfs file open with vi other process waiting

[choijg]

HPUX 11.23 v2
SD Itanium 64way

cfs 4.1, cvm 4.1

when the User process open the logfile with

vi editor , live process waiting and live

process time out is occured

how do i fix this problem

I'm really serious.

Please help.

if you want to email me

use choijg @ empal .com

thanks....

[Dave Olker]

Hi,

I don't happen to have a 64-way Itanium Superdome handy, but I do have a 2-node VxCFS cluster setup here in Roseville that I can try duplicating your problem on.

Just to be clear - you're logging onto one of the CFS servers and using vi to open an ASCII log file. Then other processes trying to access the log file are blocked?

Are these other processes on the same server? A different server in the cluster? What command are these other processes using to try to access the log file when they get blocked? Are you using the standard vi tool that ships with HP-UX or some modified version that places a mandatory lock on the file?

Please give me a step-by-step set of directions to duplicate the problem and I'll see if I can get it to reproduce on my systems.

Regards,

Dave

[A. Clay Stephenson]

Check to see if the setgid bit is set on this file. If the setgid bit is set on a data file, it has the effect of promoting an advisory lock to a mandatory lock.

[choijg]

thanks for your reply

I have 3 SD64 server with shared-r/w cfs
mount

and the server process logging the live log

on cfs filesystem file...... pro_act.log

and other server process 5~6 process

logging the file all the same time

logging is no problem, but when the user

open with vi the processes timeout and

waiting.

each time server A open file serverB's log
file and serverC's log file, server process is waiting and timeout, It's middle ware server process/// in SHB(??... T_T)

thank./// It's live situation.

[A. Clay Stephenson]

You need to provide 2 very important pieces of data:

ls -l yourlogfile
type vi

The run-of-the-mill vi shouldn't have a clue about file locking but you might have a wrapper'ed version in place which does.

It would also be useful to know if vi -R yourlogfile does the same thing.

[Dave Olker]

Well, I tried doing something vaguely similar to what you're talking about by doing the following:

ServerA: # cd /cfs/data <--- shared CFS
ServerA: # find / > allfiles
ServerB: # cd /cfs/data
ServerB: # vi allfiles

So while ServerA was doing a find and writing the output to a data file I logged into the second server and was able to vi the file being appended to without a problem.

Can you please check the permissions on the target file to see if Clay's suggestion is true - the setgid bit causing a mandatory lock?

If not, you're best bet would probably be to get a copy of the tusc tool and run the vi command under tusc to capture the system calls made by vi and see which system call vi is hanging on and what error (if any) the system call is returning during the hang. That might give a lot more detail on the underlying cause of the hang.

Regards,

Dave

[choijg]

file permission is -rw-rw-rw

A server process p1, p2, p3, p4 is

logging on live log file "p1234.log"

and the file is logging with cfs shared_read/write

when i open the log file with vi, in online situation the process p1, p2, p3, p4 is timeout and waiting.(0.2~3.0sec db commit time)

The process is waiting to write the log on "p1234.log" file

How do i avoid this situation

Thanks

[Dave Olker]

Ok, it sounds like the problem is not that the process calling vi is hanging - the process calling vi is causing the *logging* processes to hang. If that is the case I would still recommend using tusc to troubleshoot the problem. Instead of running vi under tusc, you would have to use tusc to attach to one of the logging processes and then reproduce the problem by using vi against the log file.

The tusc output from the attached process will hopefully tell you which system call the logging process is making and give some insight into why it's blocking. You might want to attach to the process for a few seconds before calling vi so that you can see some "good" logging entries before calling vi and then you'll have the "bad" logging attempts to compare against.

Regards,

Dave

[choijg]

type vi
vi is a tracked alias for /usr/bin/vi

ls -l TSDD03_cXXX3p.log
-rw-rw-rw- 1 tmax tmax 381395017 Aug 3 04:57 TSDD03_cXXX3p.log

[choijg]

In all process on server has logfile

and fast logging and big logfile has

more problem, but all logfile is same problem.

Do i have to use " tusc -T%T -p P_id"

or other ?

[Dave Olker]

The tusc syntax I'd suggest would be:

# tusc -Eflvtpn -T "" -ccc -r all -w all -o

This will trace all system call entry points and exit points and capture the read or write buffers of the process so that you can see which logfile entries are taking longer than others. That might help you more quickly determine where in the tusc output to start looking for the slower log entries.

If someone else has a better suggested tusc output string by all means suggest away.

Dave

[choijg]

Hi, I trace the process system call

with tusc. But i don't know what is the problem. could you please check the tusc file? if you could please email me // i will replay with the log file// .

// Anyway we will ask this problem
to HP KOREA RC MC TEAM

It's big problem in this SHFG site.

[Dave Olker]

I agree going through the Response Center is appropriate for a problem like this in a mission critical environment.

The tusc trace will hopefully help them identify the root cause, but it will also help if you can tell them the exact point in the tusc output where you issued the "vi" command that caused logging transactions to begin blocking. That will help anyone looking at the logfile know where to focus their attention.

Regards,

Dave

[choijg]

not found contact HP RC