Cannot umount filesystems

[Tomek Gryszkiewicz]

Hi,
There is an issue on my machine: i have some filesystems on EMC array. Something is broken now: when i want to - for example - show bdf on it, it hangs (cannot even kill -9 it); it hangs also when i want to umount it. I even cannot shutdown this machine - hangs on "Unmountig file system"...
Any ideas? Is there a way to shut it down FORCE without using poweroff button?

-Tomek

[Jeff Schussele]

Hi Tomek,

Run the following - one at a time - pause a second or 2 between syncs:

sync
sync
sync
/etc/reboot -r #to reboot -h to halt w/o reboot

HTH,
Jeff

[Robert Gamble]

Tomek,

Those symptoms could also be caused by a stale NFS mount from another system. Look in /var/adm/syslog/syslog.log for errors.

Otherwise, Jeff's solution should work.

[Tomek Gryszkiewicz]

Thanks, but cannot try it. Even console hangs... Have to power off it.

-Tomek

[Tomek Gryszkiewicz]

Anyway, is the way to umount NFS filesystem when it is blocked like this? I had the same problem like this in the past, and finally did reboot the machine...

-Tomek

[V.Tamilvanan]

Hi,

#reboot -q


Its like a TOC.

[Tomek Gryszkiewicz]

I mean, any way except reboot :)
I love NFS...

-Tomek

[Jakes Louw]

If you are getting queued I/O requests due to hardware errors or stale NFS mounts, then there aren't many things you can do:
- you can try clearing/resolving the hardware problem
- or you can reboot

Shutdown will attempt a tidy closedown of all services, and finally attempt to kill the user processes and unmount the filesystems. In this case, this will never happen, as the I/O subsystem is obviously in a bad state.
Therefore "reboot -r" or "reboot -q" are your only options.

[Bruno Ganino]

Tomek, down the system with classic command "shutdowd -i0 -g0 -y"
after you make the boot and see the result.
If not solved, try check system with "fsck" command.
Regards
Bruno

[Sunil Sharma_1]

Hi Tomek,

This problem is only related to NFS.
so best idea is to restart......
nothing else...
restart server.. it will be OK...
Sunil

[Mark Grant]

Tomek,

IN answer to your last question, there is actually a default NFS timeout and it is expressed as a half life of something or other and lasts hours, possibly days and there isn't anything you can do about them except turn the server back on again. However, you can do soft NFS mounts which timeout much earlier.

If you have this in /etc/fstab

hostname:filesystem mountpoint nfs soft, retry=5 0 0

Then when you mount this, it will be soft mounted and should the server become unavailable it will time out after five retries (about 10 minutes should do it).

[Tomek Gryszkiewicz]

Thanks for it!
Any suggestions about something else in place of NFS? I am a little bit afraid to do NFS on production system.
It sounds silly, but.. maybe Samba?
What do you think about it?

-Tomek

[Mark Grant]

Tomek,

Don't be concerned about NFS, it works fine. We use automount to automatically mount out NFS shares and configure a timeout in /etc/rc.config.d/nfsconf with "AUTOMOUNT_OPTIONS="-t 180".

Therefore we get our NFS mounts mounted when we need them and unmounted automatically when we don't.

[Steven E. Protter]

On the NFS server, you can try shutting down the nfs services.

/sbin/init.d/nfs.client stop
/sbin/init.d/nfs.server stop
/sbin/init.d/nfs.core stop


You may need to do fuser -cuk /filesystem_name on the NFS server.

I'm not sure it will work but its worth a shot.

Last try, which is almost as bad as a reboot.

init 2

SEP

[Todd McDaniel_1]

I occasionally have this problem were BDF hangs... and then upon shutdown it fails to get passed umount portion.

Here is what I did.

I have a unique setup were we have a 2nd hard line to the console... so I login and kill the shutdown command, then issue a reboot command.

Not sure why the BDF fails occasionally, but it seems to hang every once in a while... and a reboot fixes it. However, it seems that it happens if I let it go several months in arow without a reboot.

I firmly believe in not rebooting at all, if it is not necessary. But this issue forces me to consider it a few times a year.

[Alzhy]

Upgrade to VxFS 3.5 on your servers.. it now has a nice vxmount -F option -- forced umount... VxFS 3.5 is a no-cost upgrade and introduced a number of enhancements...

[Michael Steele_2]

Please attach:

# nfsstat -rc
# netstat -m
# nfsstat -s
# vmstat -n
# netstat -s -p udp

Thanks!