Difficulties unmounting /home

[Ralph Grothe]

Hello,

don't be misguided, I'm no wizard at all.
Forgot the magic spell to get /home unmounted for filesystem extension under the constraint of a lacking OnlineJFS.

I can't/don't want to go into single user mode now.

I kicked out every user
(btw, how to prevent new logins, can remember from other uinices that a "touch /nologing" sufficed, but obviously unknown to HP-UX)

# fuser -cu /home
/home:


despite device claims to be busy

# umount /home
umount: cannot unmount /home : Device busy


Hm, suspected init to be too helpful with some respawning of getty, thus commented out the cons entry while being logged in as root via lan console

# grep respawn /etc/inittab
#cons:123456:respawn:/usr/sbin/getty console console # system console

#ttp1:234:respawn:/usr/sbin/getty -h tty0p1 9600
#ttp2:234:respawn:/usr/sbin/getty -h tty0p2 9600
#ttp3:234:respawn:/usr/sbin/getty -h tty0p3 9600
#ttp4:234:respawn:/usr/sbin/getty -h tty0p4 9600
#ttp5:234:respawn:/usr/sbin/getty -h tty0p5 9600
krsd:123456:respawn:/sbin/krsd -i
sfd:123456:respawn:/sbin/sfd
#ups::respawn:rtprio 0 /usr/lbin/ups_mond -f /etc/ups_conf
samd:23456:respawn:/usr/sam/lbin/samd # system mgmt daemon
ems4:3456:respawn:/etc/opt/resmon/lbin/p_client


but didn't help :-(

Come on real wizards, this should be an easy trick,
what is the spell?

Ralph

[Dietmar Konermann · Accepted solution]

Hi, Ralph!

Essentially the fuser -c /home should find processes that keep resources of the file system. (Except e.g. open socketss... in this case you could have a look at lsof's output.)

The 1st question I usually ask in this situation: "is or was the file system exported via NFS?". If yes, then you should stop/start rpc.statd and rpc.lockd and try to umount again.

Best regards...
Dietmar.

[Evert Jan van Ramselaar]

You can prevent users from logging in by creating a file /etc/nologin. With that file in place only new logins from root are allowed.

EJ

[Ralph Grothe]

Oh my, I'm such a ****ing idiot!

Of course you are dead right Dietmar.

I forgot that I once on request exported /home/oracle to another box to aid in copying great data chunks from a CD.

A mere exportfs would have shown it to me instantly.

Sorry, for bothering you guys.

Thanks for confirming that it's the same /etc/nologin under HP-UX, didn't find such a hint in login manpage

[Ralph Grothe]

Dietmar,

I think I still need a little hlep.

I release (i.e. umounted) all nfs mounts to this server.
Then I unexported everything, and should down the whole nfs stuff by the init scripts
/sbin/init.d/nfs.{server,client,core}

Now none of the daemons is running anymore

# ps -ef|grep -e lockd -e statd -e nfsd -e biod -e rpc|grep -v grep
root 3 0 0 Jul 23 ? 11:21 statdaemon
root 1147 1 0 Jul 23 ? 0:55 /opt/dce/sbin/rpcd
root 11968 807 0 Jul 24 ? 0:00 /usr/dt/bin/rpc.ttdbserver

except for some RPC presumably done by the transaction tracking daemon from the PerfView suite.


Still can't get /home unmounted though!

# umount /home
umount: cannot unmount /home : Device busy



# ll /etc/xtab
-rw-r--r-- 1 root root 0 Aug 4 10:04 /etc/xtab

# grep nfs /etc/mnttab; echo $?
1


What else can I do?

[Robert-Jan Goossens]

Hi Ralph,

What does the command

cd /
# fuser -c /home

give you ?

You can also use the tool lsof

http://hpux.connect.org.uk/hppd/hpux/Sysadmin/lsof-4.67/


Robert-Jan.

[Dietmar Konermann]

Hi, Ralph!

Well, that's getting interesting now.

1) NFS seems to be ruled out
2) no cascading mounts (I assume so)
3) no swap devices actiive on /home (I hope so)
4) also losf does not find anything (true?)

Hmm, those are all known cases that come into my mind.

Is CA Unicenter installed?

Best regards...
Dietmar.

[HP-UX Enthusiast]

Hi Ralph
I know you've been executing fuser on the filesystem mount point. Try running fuser on the logical volume block device file for /home. This might show you the offending process(es).

[Ralph Grothe]

Sorry guys for the delay,

have been distracted by another construction site.
Now back to the unwieldy mount.

HP-UX Enthusiast,

feeding fuser with the LV doesn't seem to be digestible

# fuser -cu $(bdf /home|awk '$1~/lvol/{print$1}')
/dev/vg00/lvol4: fuser: could not find file system mounted at /dev/vg00/lvol4.

but we've got an FS on it


# fstyp -v $(bdf /home|awk '$1~/lvol/{print$1}')
vxfs
version: 4
f_bsize: 8192
f_frsize: 8192
f_blocks: 25600
f_bfree: 14422
f_bavail: 14422
f_files: 1312
f_ffree: 8192
f_favail: 8192
f_fsid: 1073741828
f_basetype: vxfs
f_namemax: 254
f_magic: a501fcf5
f_featurebits: 0
f_flag: 16
f_fsindex: 7
f_size: 25600

[Massimo Bianchi]

Hi,
maybe you have the mnttab dirty ?

Can you post a "mount -p" ?
Once i could not umount a FS and spent 20 minutes looking for process when i had another FS mounted on a subdir of it :)

fusr didn't show anything in that case.

Massimo

[Ralph Grothe]

Dietmar,

please, forgive me.
Today is not my day.
Our office temperature has already reached the 30 deg/C, with no air movement (draft sensitive colleague wouldn't allow me to turn on the fan), windows shut with no possibilty to open them as they removed the handles (facade renovation under work), and me and my brain already is melting away.

Of course you were right again, and I'm blinded.
We have a blooming submount here:

# bdf -l|grep home
/dev/vg00/lvol4 204800 89424 114480 44% /home
/dev/vg01/lvORA 10240000 3940174 6116854 39% /home/oracle

Ok, the usual mantra did work after the staggered umount.

The OS is always right.
It's the idiot who is interacting with the OS who is to blame.

Or, how was the saying?
Unix is a user friendly OS.
It's only very picky about what kind of users it is friendly to.

I'm so sorry for keeping you up.
I hope the points will compensate.

[HP-UX Enthusiast]

Hi Ralph
Glad you've found a solution, but just for future reference drop the -c switch on fuser when running it against the LVM block device file, since the -c is for displaying the use of a mount point.

[Ralph Grothe]

You are right,
I just was too lazy to consult the manpage again ;-)