system mount problem

[peterchu]

I have two RH servers , host A export a path for host B , host B mount this path , it seems work fine , if I use "df" on host B like below
hostA:/tmp/EDP /home/peter/EDP ,

However , when the user in host B read the file , then it will hang , I have to re-mount it , but if I read the file again , it will also hang again. so I have already mount the path many times, I use "df" , it display

hostA:/tmp/EDP /home/peter/EDP
hostA:/tmp/EDP /home/peter/EDP
hostA:/tmp/EDP /home/peter/EDP
"
"
could suggest what is wrong ? thx.

[Steven E. Protter · Accepted solution]

I'd like to see the contents of /etc/exports for hosta and hostb Make sure they are consistent with the mount commands you are attempting.

On both machines:

exportfs -av

Note and correct any errors you see.

There may be clues in the /var/log/messages file on one or the other server.

The cause of this can be:

inconsistent network settings.
problems with network cabling.
problems with switch settings.

Check the settings of network ip addresses in /etc/sysconfig/network-scripts

It would be nice to see some kind of error message from somewhere. You should be able to find and post something from /var/log

Poke around. NFS is very sensitive to the slightest issues.

SEP

[peterchu]

in host B , i found the error in the /var/log

kernel: nfs: task 1619 can't get a request slot

hostA /etc/exports
/tmp/EDP hostB(rw,no_root_squash)

I successfully export the file ,
the system mount work fine previously ,
could suggest what is wrong ? thx

[Stuart Browne]

Is the 'nfslock' service running ?

[peterchu]

I tried to restart the NFS , but still fail , is it the same as what you said "nfslock" ? thx

[peterchu]

umount -a

umount: /home/peter/EDP
: device is busy
umount: /home/peter/EDP
: device is busy
umount: /home/peter/EDP
: device is busy
umount: /home/peter/EDP
: device is busy
umount: /home/peter/EDP
: device is busy
umount: /home/peter/EDP
: device is busy
umount: /home/peter/EDP
: device is busy
umount: /var: device is busy
umount: /usr: device is busy
umount: /users: device is busy
umount: /tmp: device is busy
umount: /home: device is busy
umount: /dev/pts: device is busy
umount: /: device is busy

except reboot , how can I umount the path.

[Stuart Browne]

Find the process using the files in the mount path (lsof /home/peter/EDP), and kill it off.

If it still won't work, then use 'umount -f /home/peter/EDP'.

This forces a mount of a stale (read: broken) NFS mount.

[peterchu]

I still fail to find the process by lsof

and when use -f , it pop the below mesages
[root@peter tmp]# umount -f /home/peter/EDP
umount2: Device or resource busy
umount: /home/peter/EDP: Illegal seek
umount2: Device or resource busy
umount: /home/peter/EDP: device is busy
umount2: Device or resource busy
umount: /home/peter/EDP: device is busy
umount2: Device or resource busy
umount: /home/peter/EDP: device is busy

[Roberto Polli]

On rh8 sometimes happens such a thing. Still an init 1 is unuseful and the only solution _I_ found is rebooting.

You can play with kernel nfs mount table but you'll lose stability.
(man rpc.mountd)

I think you can change settings in /etc/exportfs using the
sync
directive, which in my experience can reduce a bit those situation. You can set some limits as suggested in
#man nfs

hope somebody has better solution. I'll let you know.
Peace, R.

[peterchu]

I tried to reboot it , when rebooting , i found the system can't umount the path "home/peter/EDP" , then I cool boot it , after the system reboot , I tried to umount the path "umount -f home/peter/EDP" again , it pop the device is busy , how can I umount it ? thx.

[peterchu]

the message like this

umount2: Device or resource busy
umount: /home/peter/EDP : Illegal seek

[Roberto Polli]

are you using the nscd service?
You should:
1)chkconfig nscd off
service nscd stop
2) stop nfs [look @ 1) ]
2a) init 1
3) man rpc.mountd
and look for the file where nfs stores the remote mount point [is something like xtab ] (it takes a client list in a file so he can remember his "clients" when he boots ut) bkup it and clear it

when you play with nfs remember to stop-play start nscd.

Try to track the route of NFS packet [ Reboot without starting NFS and check file changings..
]

Peace, R

[Steven E. Protter]

I'm not sure if Linux even has this command, but here is a useful one from HP-UX

fuser

fuser -cu /mounted_fs

lists processes

fuser -cuk /mounted_fs kills all processes on the mounted filsysetm

fuser -u /folder lists processes in a folder thats not a mounted filesystem

fuser -uk kills processes in a folder

There appears not to be a -u command in Linux but the man for fuser will probably tell you how to accomplish what I'm recommending.

:=)

SEP

[Bharat Katkar]

Yes,
In HPUX it is #fuser -kuc and same can be accomplished in linux as:

# fuser -km /home/peter/EDP

Then try unmount the file system

Hope this works for you.

[peterchu]

I have a founding , host B can create file and vi file at host A but I found that the file can't a big file , in simply , I can't do the process that require high memory , do you have any suggestions ? thx

[Stuart Browne]

If you are going to be dealing with larger file sizes over a 'slow' network link, you may want to look at changing the NFS block sizes from their defaults, to something that suits the link speed available.

Look at 'man 5 nfs', in particular the 'rsize' and 'wsize' values.

[peterchu]

thx stuart,

it seems not the speed problem , it seems the previous "dead" mount still in the memory and the host B can't unmout it , could suggest how to do it ? thx.

[Stuart Browne]

If you can't kill the processes which are holding the mount active.. And you can't forcibly unmount it (using '-f' as mentioned in previous post), then you are left with two things.

1) Stop all NFS on the server.

Depending on how much you use NFS on the server depends whether this is a truely feasable option. You'll need to stop the 'nfs' services, the 'nfslock' and you might also need to stop the 'rstatd' servce. Then stop the 'portmap' service before restarting them, in reverse order.

2) Reboot client workstation, making sure it doesn't use the NFS mount nastily in the boot process.

Then do some testing.

[peterchu]

I tried fuser , no process in the path ,
I explain my case clearly , our network have problem suddenly then all the linkage broken and the system mount between all hosts was disconnected , now I can't mount it back , I also found that if I export a new path , it works fine ( the remote site can mount it ) , but if mount the existing exported path ( I mean the path that exported previously ) , it is not work .
The problem happened in all my hosts , could suggest what can i do ? very thx.

[Stuart Browne]

Assuming that each individual remote system isn't already trying to use that exported path in a previous mount attempt (it should only ever be able to mount any given filesystem once. If you try, you should get an error similar to:

mount: hostA:/tmp/EDP failed, reason given by server: Permission denied

In which case you need to clear the mount table on the client machine (using 'umount' or restarting it).

If this doesn't work, then you need to ensure that the exports are being read correctly, using the 'exportfs -av' command. It will re-export all current exports, but more importatnly, it will *LIST* every mount point and any errors associated with it on the server side.

[peterchu]

very thanks Stuart,

I think you got my point ,

I tried exportfs -av , it list some paths , like it " exporting edp:/home/EDP/temp
reexporting edp:/home/EDP/temp to kernel" , is it a problem ? thx.

[Stuart Browne]

NFS mounts have to be exact.

If you are trying to:

mount hostA:/tmp/EDP /home/peter/EDP

Then you have to EXPORT '/tmp/EDP' on machine 'hostA'.

As you're exporting 'edp:/home/EDP/temp', then on hostB, you have to be using the command 'mount edp:/home/EDP/temp /home/peter/EDP'.

(NOTE: This makes 2 assumptions, that hostname 'edp' resolves to the correct IP, and that the local directory '/home/peter/EDP' exists, and has no contents).