df hanging and mounting having issue

[Naj]

Hello expert,

i having problem with mount file and seems like i couldn't umount/mount the path.

need your expert to advise on this

DevFS                    9       9       0  100% /dev/deviceFileSystem
bdf: file system server-b:/var/opt/ignite/recovery/archives/server-a mounted on /var/opt/ignite/recovery/arch_mnt is not responding

it hanging when executed command df on server-a

bdf on server -a

server-a:/dev (root) bdf
Filesystem          kbytes    used   avail %used Mounted on
/dev/vg00/lvol3    1032192  330048  696736   32% /
/dev/vg00/lvol1    2048000  322400 1712208   16% /stand
/dev/vg00/lvol10   9224192 7848592 1365672   85% /var
/dev/vg00/lvol6    4096000 2063896 2016304   51% /var/adm/crash
/dev/vg00/lvol9    6660096 2998584 3632960   45% /usr
/dev/vg00/lvol5    1032192  827920  202704   80% /tmp
/dev/vg00/lvol8    7880704 6285744 1582624   80% /opt
/dev/vg00/lvol7     212992    5832  205544    3% /home
DevFS                    9       9       0  100% /dev/deviceFileSystem
bdf: file system server-b:/var/opt/ignite/recovery/archives/server-a mounted on /var/opt/ignite/recovery/arch_mnt is not responding

bdf on server-b

server-b:/var/opt/ignite/recovery/archives/server-a (root) bdf
Filesystem          kbytes    used   avail %used Mounted on
/dev/vg00/lvol3    1032192  560704  467848   55% /
/dev/vg00/lvol1    2048000  319936 1714648   16% /stand
/dev/vg00/lvol10   33390592 19047240 14254632   57% /var
/dev/vgignite/lvol1
                   142999552 78871011 60121565   57% /var/opt/ignite
/dev/vg00/lvol6    4096000 2895840 1190840   71% /var/adm/crash
/dev/vg00/lvol9    6660096 3609328 3027064   54% /usr
/dev/vg00/lvol5    3096576  659296 2418248   21% /tmp
/dev/vg00/lvol8    14073856 7351688 6669712   52% /opt
/dev/vg00/lvol7     212992    5832  205544    3% /home
DevFS                    9       9       0  100% /dev/deviceFileSystem

Thanks

BR

Naj

[Dennis Handly]

server-b seems to be up.  Have you ever successfully mounted it on server-a?

[Naj]

Hi,

Both servers are running fine.
The issue is from server-a could not mount and giving below error

DevFS 9 9 0 100% /dev/deviceFileSystem
bdf: file system server-b:/var/opt/ignite/recovery/archives/server-a mounted on /var/opt/ignite/recovery/arch_mnt is not responding

Thanks

BR
Naj

[Angesh]

NFS is not responding in server-b side

Could you please share the output for "mount -v" from server -a

[Naj]

Hi,

"mount -v" on server-b

server-b:/root/home/root (root) mount -v
/dev/vg00/lvol3 on / type vxfs ioerror=nodisable,log,dev=40000003 on Sat Aug 21                                                                              19:11:40 2010
/dev/vg00/lvol1 on /stand type vxfs ioerror=mwdisable,log,tranflush,dev=40000001                                                                              on Sat Aug 21 19:11:46 2010
/dev/vg00/lvol10 on /var type vxfs ioerror=mwdisable,delaylog,dev=4000000a on Sa                                                                             t Aug 21 19:12:06 2010
/dev/vgignite/lvol1 on /var/opt/ignite type vxfs ioerror=mwdisable,delaylog,dev=                                                                             40030001 on Sat Aug 21 19:12:07 2010
/dev/vg00/lvol6 on /var/adm/crash type vxfs ioerror=mwdisable,delaylog,dev=40000                                                                             006 on Sat Aug 21 19:12:07 2010
/dev/vg00/lvol9 on /usr type vxfs ioerror=mwdisable,delaylog,dev=40000009 on Sat                                                                              Aug 21 19:12:07 2010
/dev/vg00/lvol5 on /tmp type vxfs ioerror=mwdisable,delaylog,dev=40000005 on Sat                                                                              Aug 21 19:12:07 2010
/dev/vg00/lvol8 on /opt type vxfs ioerror=mwdisable,delaylog,dev=40000008 on Sat                                                                              Aug 21 19:12:07 2010
/dev/vg00/lvol7 on /home type vxfs ioerror=mwdisable,delaylog,dev=40000007 on Sa                                                                             t Aug 21 19:12:07 2010
DevFS on /dev/deviceFileSystem type DevFS defaults,dev=1000000 on Sat Aug 21 19:                                                                             12:16 2010
-hosts on /net type autofs ignore,indirect,nosuid,soft,nobrowse,dev=1000003 on S                                                                             at Aug 21 19:12:53 2010

mount -v on server-a

server-a:/root/home/root (root) mount -v
/dev/vg00/lvol3 on / type vxfs ioerror=nodisable,log,dev=40000003 on Thu Sep 10                                                      14:29:03 2009
/dev/vg00/lvol1 on /stand type vxfs ioerror=mwdisable,log,tranflush,dev=40000001                                                      on Thu Sep 10 14:29:10 2009
/dev/vg00/lvol10 on /var type vxfs ioerror=mwdisable,delaylog,dev=4000000a on Th                                                     u Sep 10 14:29:31 2009
/dev/vg00/lvol6 on /var/adm/crash type vxfs ioerror=mwdisable,delaylog,dev=40000                                                     006 on Thu Sep 10 14:29:31 2009
/dev/vg00/lvol9 on /usr type vxfs ioerror=mwdisable,delaylog,dev=40000009 on Thu                                                      Sep 10 14:29:31 2009
/dev/vg00/lvol5 on /tmp type vxfs ioerror=mwdisable,delaylog,dev=40000005 on Thu                                                      Sep 10 14:29:31 2009
/dev/vg00/lvol8 on /opt type vxfs ioerror=mwdisable,delaylog,dev=40000008 on Thu                                                      Sep 10 14:29:32 2009
/dev/vg00/lvol7 on /home type vxfs ioerror=mwdisable,delaylog,dev=40000007 on Th                                                     u Sep 10 14:29:32 2009
DevFS on /dev/deviceFileSystem type DevFS defaults,dev=1000000 on Thu Sep 10 14:                                                     29:40 2009
server-b:/var/opt/ignite/recovery/archives/server-a on /                                                     var/opt/ignite/recovery/arch_mnt type nfs rsize=32768,wsize=32768,NFSv3,dev=1000                                                     4f3 on Sat Jun  4 15:01:07 2011

[Dennis Handly]

Are both servers 11.31?

server-b:/var/opt/ignite/recovery/archives/server-a on /var/opt/ignite/recovery/arch_mnt type nfs rsize=32768,wsize=32768,NFSv3,dev=1000  4f3 on Sat Jun  4 15:01:07 2011

Did this work on Jun 4 and suddenly stop working?

I assume that instead of server-b, you really have a FQDN there?

[Naj]

Hi,

I can see that folder but when i perform ls command it take long time and hanging (server-a)
i can cd and ls without any error in server-b to location

/var/opt/ignite/recovery/archives/server-a

[Dennis Handly]

>I can see that directory.

That's probably the mount point, which you can always see.

Have you tried unmounting the remounting it?

[Naj]

Hi,

No, it won't. it was hanged during the process

[Dennis Handly]

If 11.31, have you tried umount -f?

[Naj]

Hi,

Remain same..
i can see the background process..trying to kill but it's still there

#kill -9 (pid)

BR
Naj

[Dennis Handly]

What background process?

If kill -9 doesn't work and it's not a zombie, you'll need to reboot.

[Naj]

Hello,

i try to kill 9321 & 24635 but it won't.

The process still there.

I think this is cause why mount/umount could not perform

server-a:/var/opt/ignite (root) ps -ef |grep nfs
    root    33     0  0  Sep 10  ?         0:00 nfs_failover
    root    32     0  0  Sep 10  ?        251:57 nfs_async_io
    root    30     0  0  Sep 10  ?         0:00 nfs_reclaim
    root    34     0  0  Sep 10  ?         0:00 nfs4clntkd
    root    35     0  0  Sep 10  ?         0:00 nfs4_async_io
    root    84     0  0  Sep 10  ?         0:00 nfs4srvkd
    root    83     0  0  Sep 10  ?         0:00 nfslogkd
    root  9321     1  0  Jun  4  ?         0:03 /sbin/fs/nfs/umount server-b:/var/opt/ignite/recovery/archives/server-a /var/opt/ignite/recovery/arch_m
    root 25917     1  0  Jun 23  ?         0:00 /usr/sbin/nfsmapid
    root 24635     1  0  Jun 10  ?         0:00 /sbin/fs/nfs/mount -Fnfs server-b:/var/opt/ignite/clients /var/opt/ignite/recovery/client_mnt
    root  2988 21477  0 07:14:23 pts/11    0:00 grep nfs

BR

Naj

[Dennis Handly]

>I try to kill 9321 & 24635 but it won't.  The process still there.

Time to reboot, they are hung on I/O.

[gnana prakasam]

Hi,

I am also facing the same issue .Is there any way without rebooting the server?

[Dennis Handly]

I am also facing the same issue.  Is there any way without rebooting the server?

If on 11.31, have you tried "umount -f" on the client?

Have you fixed the server?