NFS

[Sarwan]

Hi freinds,

This Question may look old but i am unable to find a solution

#uname -a
HP-UX green B.11.11 U 9000/800 544746537 unlimited-user license


I am unable to unmount a File System /APP from client green

This is a nfs mounted File System. This is mounted from Blue:/app /APP nfs rsize=32768,wsize=32768,proto=udp,NFSv3 0 0 1220974403.
nfs server blue is down and we are unable to ping even this server.
this is not listed in /etc/fstab but listed in /etc/mnttab.
I tried lot of option to remove this File System but failed to do soo
I did
#mv /etc/mnttab /etc/mnttab.old
#mount -a

restarted NFS client
#/sbin/init.d/nfs.client stop
#/sbin/init.d/nfs.client start

The system hungs when i try any of these
commands
#fuser -cu /APP
#bdf /APP
#mv /APP /APP.OLD
#lsof /APP

For all this i have to close the telnet session and start a new one. I am done for the day. Please post your suggestions so that i can come and try out tomorrow.

[Laurent Menase]

just an idea
ifconfig lo0:1 addreseofblue

--> this will fake a server ' Blue '
so the next nmount should work.

umount

ifconfig lo0:1 0.0.0.0

I never tried it, but should work

[Rita C Workman]

What you may have is a hung mountpoint, or some process going to that mountpoint is still hung out there, or somebody is cd into it or some other reason.
On a simple system you can find things and clean up the rpc mess. But not knowing your server or what's going on, or what has already been tried here's a short, albeit not so pleasant, solution.

Reboot 'green'.

/Rita

[Gordon Sjodin]

Try:

ps -ef | grep /APP

then kill any associated pids.

[Kapil Jha]

after trying all of the above.
There is one utility called clear_locks which can help you.
Just clear the locks for both the machines and this may help you.

BR,
Kapil+

[Dave Olker]

Some bozo wrote a technical paper about this subject a while back. Suggestions in there might still work...

http://docs.hp.com/en/3929/ForciblyUnmountingNFSFilesystems.pdf

[Sarwan]

I have tried all the above option:


=====================================================================================
Laurent Menase,

I am unable to find out ipaddress of blue as it is down

=====================================================================================

Rita,

Reboot green is not possible as it is a production box, have to go through lot of process and need to give explaination also

=====================================================================================
Kapil,


Clear_locks

#green>clear_locks -s blue
Clearing locks held for NFS client ironwood on server blue
RPC: Unknown host
=====================================================================================
Gordon Sjodin,

Kill process related to this

#green>ps -ef |grep -i /APP
root 22935 18392 0 01:24:10 pts/tH 00:00 mount -o soft blue:/app /APP
root 16021 1 0 09:38:24 ? 00:00 fuser -cu /APP
root 16799 1 0 11:47:38 pts/tB 00:00 lsof /USERS
root 15028 1 0 06:36:31 ? 00:00 grep -i cd /APP
root 16342 1 0 10:47:43 ? 00:00 fuser -CK /APP
root 15293 1 0 07:46:06 ? 00:00 fuser -cu /APP
root 16393 1 0 10:52:08 ? 00:00 fuser -cu /APP
root 16812 16753 0 11:50:23 pts/tB 00:00 mv /APP /APP.OLD
root 22978 22942 1 01:31:09 pts/tJ 00:00 grep -i /APP
root 16801 1 0 11:48:09 pts/tB 00:00 lsof /APP
root 18351 1 0 23:49:42 ? 00:00 fuser -cu /APP

i am unable to kill any of these process
=====================================================================================

now going through Doc provided by Dave

[Dennis Handly]

>Reboot green is not possible as it is a production box

Where are you in getting blue back up?

[Sarwan]

All these problem started because backup is not running on blue because of this NFS stale issue

[Dennis Handly]

>All these problem started because backup is not running on blue because of this NFS stale issue

I thought you said the hangs on green are due to blue crashing?

[Viktor Balogh]

>I am unable to find out ipaddress of blue as it is down

# nslookup blue

[Sarwan]

nslookup doesn't work as blue is down

[Gordon Sjodin]

I had a similar problem and ended up rebooting.

[Sarwan]

I have asked other NFS users for downtime. So i get go i am planning to down all NFS services and try to unmount the FS.
would that be a good idea what you guys feel???

[Sarwan]

Thanks to all for your valuable suggestion.

I had to restart the server blue and now the issue is resolved.

[Viktor Balogh]

>nslookup doesn't work as blue is down

??? It _should_ work on green unless bind was running on server blue. Either I misunderstood something or you didn't describe the situation clear. Try this command on green, (or on a running server):

# nslookup blue

and paste the output here.

[Sarwan]

Sorry i had to restart server green, blue is still down :)

[Sarwan]

Viktor,

#green>nslookup blue
Using /etc/hosts on: green

looking up FILES
Trying NIS
Trying DNS
*** dns1rrc.telcordia.com can't find blue: Non-existent domain

[Sarwan]

Issue resolved

[Viktor Balogh]

Sorry, this isn't meant to be an offense but you wrote it in your first post:

>This is mounted from Blue:/app

How could you mount it if the hostname 'blue' cannot be resolved on the NFS client (green)? Do not mix name resolution with the server itself. These are two separate thing. Even if the server 'blue' is down, there must be an entry in the DNS or in the hosts file (or anywhere else like NIS)

Maybe it is about decomissioning server 'blue' - and that's why you deleted the hostname in DNS/hosts, and the mount was established before when it still had an entry in DNS/hosts.

OK, I'm finished, and your problem is solved. Nice day :)