defragment -vn tmp_domain never completes

[Victor Semaska_3]

DS20E w/ SmartArray controller. 2-72GB disks in RAID-1 set (dsk0), 4-146GB disks in RAID-5 set (dsk1).
Tru64 V5.1B w/ PK4.

root_domain, usr_domain, var_domain, & tmp_domain on dsk0.

'# defragment -vn tmp_domain' never finishes, it sat there for 4 days before I killed it. '# defragemtn -v tmp_domain' (no -n option) works fine. Tried using the -n option on other domains and they work. It's only tmp_domain that has the problem.

'# ps -e' shows the process in a sleep state. Ran /sbin/advfs/verify on the domain and didn't find any problems with the domain.

Any ideas?

Thanks,
Vic

[Mark Poeschl_2]

From the man page of 'defragment':

Running the defragment -nv command on a domain with a large amount of meta-
data can cause it to fail. To prevent this from happening, double the size
of per_proc_data_size with the sysconfig command or specify the doubled
value in the /etc/sysconfigtab file.

Have you tried increasing that sysconfigtab parameter?

[Victor Semaska_3]

Mark,

Wow, I forgot I had posted that back in Feb. Anyway, this is tmp_domain and has very little on it so I don't think it's a meta-data problem.

I did track it down to a ksh script that runs all the time. Using lsof I see it has a file open on tmp_domain for some reason.

COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
sxp.k 4394 semaskav 61u VREG 3768,707517 0 822 /cluster/members/member0/tmp (tmp_domain#tmp)

I stopped the script and defragment -vn works. Start the script again and defragment -vn stalls. I checked the script several times and it never references /tmp or /cluster/members/member0/tmp.

I don't know whether it's a bug in defragment or there's something about this script. I have this script running on other servers and defragment -vn doesn't have a problem. The other servers are at the same OS version and patch kit.

My workaround so far has simply been to not defragment tmp_domain on that server.

Thanks,
Vic

[Venkatesh BL]

Can you tell us more about the ksh script that is referred to...

[Johan Brusche · Accepted solution]

Most probably this sxp.k script calls a program that opens a file descriptor on /tmp and uses an flock on it.
defragment also wants to take a lock on the file that it needs to work on and probably waits forever for the first lock to be lifted.

__ Johan.

[Victor Semaska_3]

Johan,

That does seem to be the case. I put a sleep statement before the start of a program it runs and the 'defragment -vn tmp_domain' worked. Once the program started the defragment -vn stalled.

I don't see anything open on tmp_domain with lsof or fuser so I don't know what it's doing.

Anyway, why would the -vn options cause defragment to stall and not just the -v option? The -v option alone works.

Also, I have this set up on other servers and the -vn option works OK. Seems to point to something weird with this particular server.

Thanks,
Vic