topic Re: NFS in Operating System - HP-UX

NFS

Sarwan — Mon, 18 Jan 2010 17:42:43 GMT

Hi freinds,

This Question may look old but i am unable to find a solution

#uname -a
HP-UX green B.11.11 U 9000/800 544746537 unlimited-user license

I am unable to unmount a File System /APP from client green

This is a nfs mounted File System. This is mounted from Blue:/app /APP nfs rsize=32768,wsize=32768,proto=udp,NFSv3 0 0 1220974403.
nfs server blue is down and we are unable to ping even this server.
this is not listed in /etc/fstab but listed in /etc/mnttab.
I tried lot of option to remove this File System but failed to do soo
I did
#mv /etc/mnttab /etc/mnttab.old
#mount -a

restarted NFS client
#/sbin/init.d/nfs.client stop
#/sbin/init.d/nfs.client start

The system hungs when i try any of these
commands
#fuser -cu /APP
#bdf /APP
#mv /APP /APP.OLD
#lsof /APP

For all this i have to close the telnet session and start a new one. I am done for the day. Please post your suggestions so that i can come and try out tomorrow.

Re: NFS

Laurent Menase — Mon, 18 Jan 2010 17:53:51 GMT

just an idea
ifconfig lo0:1 addreseofblue

--> this will fake a server ' Blue '
so the next nmount should work.

umount

ifconfig lo0:1 0.0.0.0

I never tried it, but should work

Re: NFS

Rita C Workman — Mon, 18 Jan 2010 17:58:57 GMT

What you may have is a hung mountpoint, or some process going to that mountpoint is still hung out there, or somebody is cd into it or some other reason.
On a simple system you can find things and clean up the rpc mess. But not knowing your server or what's going on, or what has already been tried here's a short, albeit not so pleasant, solution.

Reboot 'green'.

/Rita

Re: NFS

Gordon Sjodin — Tue, 19 Jan 2010 00:33:32 GMT

Try:

ps -ef | grep /APP

then kill any associated pids.

Re: NFS

Kapil Jha — Tue, 19 Jan 2010 04:10:18 GMT

after trying all of the above.
There is one utility called clear_locks which can help you.
Just clear the locks for both the machines and this may help you.

BR,
Kapil+

Re: NFS

Dave Olker — Fri, 18 Jun 2021 09:57:22 GMT

Some bozo wrote a technical paper about this subject a while back. Suggestions in there might still work...

http://docs.hp.com/en/3929/ForciblyUnmountingNFSFilesystems.pdf

Re: NFS

Sarwan — Tue, 19 Jan 2010 06:45:00 GMT

I have tried all the above option:

=====================================================================================
Laurent Menase,

I am unable to find out ipaddress of blue as it is down

=====================================================================================

Rita,

Reboot green is not possible as it is a production box, have to go through lot of process and need to give explaination also

=====================================================================================
Kapil,

Clear_locks

#green>clear_locks -s blue
Clearing locks held for NFS client ironwood on server blue
RPC: Unknown host
=====================================================================================
Gordon Sjodin,

Kill process related to this

#green>ps -ef |grep -i /APP
root 22935 18392 0 01:24:10 pts/tH 00:00 mount -o soft blue:/app /APP
root 16021 1 0 09:38:24 ? 00:00 fuser -cu /APP
root 16799 1 0 11:47:38 pts/tB 00:00 lsof /USERS
root 15028 1 0 06:36:31 ? 00:00 grep -i cd /APP
root 16342 1 0 10:47:43 ? 00:00 fuser -CK /APP
root 15293 1 0 07:46:06 ? 00:00 fuser -cu /APP
root 16393 1 0 10:52:08 ? 00:00 fuser -cu /APP
root 16812 16753 0 11:50:23 pts/tB 00:00 mv /APP /APP.OLD
root 22978 22942 1 01:31:09 pts/tJ 00:00 grep -i /APP
root 16801 1 0 11:48:09 pts/tB 00:00 lsof /APP
root 18351 1 0 23:49:42 ? 00:00 fuser -cu /APP

i am unable to kill any of these process
=====================================================================================

now going through Doc provided by Dave

Re: NFS

Dennis Handly — Tue, 19 Jan 2010 09:04:17 GMT

>Reboot green is not possible as it is a production box

Where are you in getting blue back up?

Re: NFS

Sarwan — Tue, 19 Jan 2010 09:21:11 GMT

All these problem started because backup is not running on blue because of this NFS stale issue

Re: NFS

Dennis Handly — Tue, 19 Jan 2010 10:34:37 GMT

>All these problem started because backup is not running on blue because of this NFS stale issue

I thought you said the hangs on green are due to blue crashing?

Re: NFS

Viktor Balogh — Tue, 19 Jan 2010 14:37:58 GMT

>I am unable to find out ipaddress of blue as it is down

# nslookup blue

Re: NFS

Sarwan — Tue, 19 Jan 2010 15:32:24 GMT

nslookup doesn't work as blue is down

Re: NFS

Gordon Sjodin — Tue, 19 Jan 2010 15:32:57 GMT

I had a similar problem and ended up rebooting.

Re: NFS

Sarwan — Tue, 19 Jan 2010 15:42:22 GMT

I have asked other NFS users for downtime. So i get go i am planning to down all NFS services and try to unmount the FS.
would that be a good idea what you guys feel???

Re: NFS

Sarwan — Wed, 20 Jan 2010 11:01:39 GMT

Thanks to all for your valuable suggestion.

I had to restart the server blue and now the issue is resolved.

Re: NFS

Viktor Balogh — Wed, 20 Jan 2010 11:02:55 GMT

>nslookup doesn't work as blue is down

??? It _should_ work on green unless bind was running on server blue. Either I misunderstood something or you didn't describe the situation clear. Try this command on green, (or on a running server):

# nslookup blue

and paste the output here.

Re: NFS

Sarwan — Wed, 20 Jan 2010 11:07:20 GMT

Sorry i had to restart server green, blue is still down :)

Re: NFS

Sarwan — Wed, 20 Jan 2010 11:09:59 GMT

Viktor,

#green>nslookup blue
Using /etc/hosts on: green

looking up FILES
Trying NIS
Trying DNS
*** dns1rrc.telcordia.com can't find blue: Non-existent domain

Re: NFS

Sarwan — Wed, 20 Jan 2010 11:12:10 GMT

Issue resolved

Re: NFS

Viktor Balogh — Wed, 20 Jan 2010 13:08:52 GMT

Sorry, this isn't meant to be an offense but you wrote it in your first post:

>This is mounted from Blue:/app

How could you mount it if the hostname 'blue' cannot be resolved on the NFS client (green)? Do not mix name resolution with the server itself. These are two separate thing. Even if the server 'blue' is down, there must be an entry in the DNS or in the hosts file (or anywhere else like NIS)

Maybe it is about decomissioning server 'blue' - and that's why you deleted the hostname in DNS/hosts, and the mount was established before when it still had an entry in DNS/hosts.

OK, I'm finished, and your problem is solved. Nice day :)