Phantom mount point, umount gives block device required

[Brad Whitlock]

I was trying to interrupt a script, and probably interrupted a mount command in the process. I'm left with a phantom mount point that I can't get rid of.

I should have /dev/vg58/lvol1 mounted at /bcv/u01/oracle/xproddata. If I run bdf | grep vg58, I see that lvol and mount, but the lvol size numbers are those of the / filesystem. Which makes sense, if the file system really isn't mounted.

If I try to umount by either the special file or directory, I get " umount: cannot unmount /bcv/u01/oracle/xproddata : Block device required "

If I cd to the directory, there is no lost+found, and running a " bdf . " from it gives the proper info for the / filesystem (as it should IF the /dev/vg58/lvol1 filesystem were unmounted).

I found the thread from last year describing a similar problem, where the solution was to mv /etc/mnttab and run mount to rebuild it. I tried that, and the rebuilt mnttab still has this phantom line in it, and I still can't umount it. Where does the mount command get the information to rebuild the mnttab?

Any other suggestions short of rebooting the machine? It's a large database development server, so rebooting is possible but a bit of a pain to schedule.

[generic_1]

You may need to reboot.
NFS Sucks.

[Brad Whitlock]

Yes, NFS has its problems, but this doesn't (directly at least) involve NFS. The file system is local. I've also tried stopping and starting the automounter and nfs.client processes, with no good effect.

[Patrick Wallek]

What happens if you try to mount that LV again?

[generic_1]

lsof might help locate any problem processes, I have better luck with it than fuser. Sometimes LVM has issues with submounts that have multiple other directories and filesystems in it and gets confused. You could refresh/replace your mnttab. stop nfs, clear the nfs version as well, sorry forgot the name, unmount, remount, start nfs perhaps.

I have the feeling you'e tried it all.
I think your probably going to have to do the dreaded reboot.

[Brad Whitlock]

> What happens if you try to mount that LV again?

It gives the already mounted, or mount point busy, or etc. error.

[Brad Whitlock]

"lsof /bcv/u01/oracle/xproddata" acts as if I ran "lsof /", or with no parameters. All open files show up, and there I'd expect the file to be "/", it shows "/bcv/u01/oracle/xproddata" instead.

"lsof +d /bcv/u01/oracle/xproddata" displays nothing. "lsof +d /" displays swapper, init, lvmkd, etc etc.

There shouldn't be any complicated stuff here. The filesystem was just formatted, mounted at one directory, then I umounted it here and mounted at another -- but apparently did my ctrl-c on the script doing the mount (of this and other filesystems) while it was processing this first one. So there aren't any submounts, or subdirectories, or really anything else on the filesystem.

I'm feeling more and more like reboot is the only hope.

P.S. to my earlier answer: mounting to the same directory gives the "is already mounted on" message; mounting to a different directory /mnt gives the "is already mounted, /mnt is busy, or allowable number of mount points exceeded" message.

[Bernhard Mueller]

Brad,

you might try
mount -o remount /mount-point

Regards,
Bernhard

[Brad Whitlock]

Bernhard,

Thanks for the suggestion. When I use the -o remount, it gives the message:

vxfs mount: /dev/vg58/lvol1 or /bcv/u01/oracle/xproddata, no such file or directory

But both the special file and the directory do exist.

[uk1]

I had the same problem. I guess first things first before I list alot of info, does the entry for the dir and lvol exist in mnttab?

[Brad Whitlock]

David: Yes, mnttab has an entry with the /dev/vg58/lvol1 special and the /bcv/u01/oracle/xproddata directory. When I rm the mnttab and let it get re-created, the entry comes back. However, a bdf shows the special and directory, but the file system size information is from the / root filesystem.

[Brad Whitlock]

FYI: My coworker arranged to get the server rebooted, and after reboot the problem went away (as expected). So I can't try any more suggestions, and can't do any more detective work. But if anyone says "the EXACT same thing happened to me, and this was the fix", I'll be sure to file it away!

Thanks for all the responses.

-- Brad