It's not possibly to log in when the root file system is full (Trusted Mode)

[Augusto Vinhaes]

Hi all !

I've got some machines in Trusted Mode and I discovered that nobody can log in when the root file system is full (including the root user). I tried the console but it does not work. I'm thinking about to create a new file system for /tcb but I don't know if I will have problems when booting in single user mode.

Regards,
Augusto Vinhaes
(Intelig Telecomunica????es)

[Christopher McCray_1]

Hello,

I think that you first and foremost endeavor is to determine what is causing your / to fill up and then fix that, whether it being to create a separate filesystem for a given application, redirect output files to another location, etc.

When you boot into single-user mode, only / and /stand is available right away.


Hope this helps

Chris

[Peter Kloetgen]

Hi Augusto,

first thing you should do, is to find out what has filled up your root filesystem! Have a look for big files :

find / -type f -size +1000 -exec ls -l {} \;

look for a bad backup try with a wrong device file.... Is there anything in /dev, which is not normal? Look for something like:

/dev/rmt/o (wrong name for tape device...)

To get into single user mode, use:

(escape boot sequence, interact with IPL, and then:)

IPL> hpux -is

Allways stay on the bright side of life!

Peter

[harry d brown jr]

That's why it's important to keep user crap out of the root filesystems. post a bdf and maybe we can help you create a better fs structure.

live free or die
harry

[Dave Chamberlin]

I have to agree with Christopher - the problem is that the root filesystem is filling up. This should be a VERY rare occurrence. Make sure you are not writing data to one of your mount points in /.

[Augusto Vinhaes]

I know that it's not common to have the root fs full and it should not contain files that belongs to applications, but what can I do when it is full ? The console doesn't work.

[Fragon]

By your "I tried the console but it does not work", did you mean you can not use your console now?
I think you can use web console or lan console and login as single user mode. Then remove some large useless files first. Of course now the God will smile!

Live as you wish!
ux

[Christopher Caldwell]

When this happens, generally a reboot is all that works. Boot into single user mode and remove the offending file.

Folks shouldn't put things in /, that file system isn't designed to change significantly. However, a common problem I see is daemons with / as a working directory that dump core into / - thus filling up /. Very aggravating.

[Fragon]

Hi Augusto, can you dial-up to your server by using HP-supported modem? This can also lead you to single user mode.

ux

[Augusto Vinhaes]

I agree with Christopher, I've got some daemons that are running with the root user and I can't change them. So the core files are always on /.

[Christopher McCray_1]

Hello, again.

You say you have daemons running as root and you can't change them, but it goes to what I'm saying before with respect to applications and their locations. Lots of things are installed in /opt, but if it isn't a separate filesystem, it will fill up / eventually if not watched.

I think Harry made an excellent suggestion in that you post your bdf output (when you get things back under control), so that we may possibly assist in making a more efficient fs structure for you.

Chris

[Martin Johnson]

Do "mkdir /core" this will prevent core file from being dumped in /.

HTH
Marty

[Augusto Vinhaes]

I think the file system lay-out is not so bad and /opt is already a separate fs. The df -n output follows:

/oraads/oradata /dev/vg_ads/lv_adsdat
/orabdom/oradata /dev/vg_bdom/lv_bdomdata
/orabdom/REDO_B /dev/vg_bdom/lv_bdomredob
/orabdom/REDO_A /dev/vg_bdom/lv_bdomredoa
/orabdom/bdomarch /dev/vg_bdom/lv_bdomarch
/orabdom /dev/vg_bdom/lv_orabdom1
/adspd01/mon_arch/bdom /dev/vg_bdom/lv_monarch
/adspd01/mon_arch/rmdy /dev/vg_rmdy/lv_monarch
/itsmbck /dev/vg_itsm/lv_itsmbck
/wom clapkg.dtc.intelig:/wom
/aspf_ixdm /dev/vg_ads/lv_ixdm
/oraads/ads_log2b /dev/vg_ads/lv_log2b
/oraads/ads_log2a /dev/vg_ads/lv_log2a
/oraads/ads_log1b /dev/vg_ads/lv_log1b
/oraads/ads_log1a /dev/vg_ads/lv_log1a
/oraads /dev/vg_ads/lv_oraads
/AlmaDelivery /dev/vg_ads/lv_ads
/app/rmdy /dev/vg_rmdy/lv_rmdy1
/orarmdy/rmdyarch /dev/vg_rmdy/lv_rmdyarc
/rmdydata02 /dev/vg_rmdy/lv_rmdyd02
/rmdydata01 /dev/vg_rmdy/lv_rmdyd01
/orarmdy/rmdy_log2b /dev/vg_rmdy/lv_rmdyl2b
/orarmdy/rmdy_log2a /dev/vg_rmdy/lv_rmdyl2a
/orarmdy/rmdy_log1b /dev/vg_rmdy/lv_rmdyl1b
/orarmdy/rmdy_log1a /dev/vg_rmdy/lv_rmdyl1a
/orarmdy /dev/vg_rmdy/lv_orarmdy
/app/remedy /dev/vg_rmdy/lv_rmdy
/itsm /dev/vg_itsm/lv_itsm
/ora_log1/oradata /dev/vg_itsm/lv_itsmlog1
/ora_log2/oradata /dev/vg_itsm/lv_itsmlog2
/oraarch/itsm /dev/vg_itsm/lv_architsm
/oraitsm /dev/vg_itsm/lv_oraitsm
/oraweb /dev/vg_itsm/lv_oraweb
/archives /dev/vg_arc/lv_arc
/home /dev/vg00/lvol5
/opt /dev/vg00/lvol6
/tmp /dev/vg00/lvol4
/usr /dev/vg00/lvol7
/var /dev/vg00/lvol8
/stand /dev/vg00/lvol1
/ /dev/vg00/lvol3

Rgds,
Augusto

[Pierce Byrne_1]

is it possible that one of your filesystems isn't mount?

[Augusto Vinhaes]

Hi !

Someone from HP told me to create a separate fs for /tcb. I've asked him if there's some problem to boot in single user mode and He said that there's no problem. Is there some problem ?

[Darren Prior]

Hi Augusto,

I wouldn't recommend having /tcb as a separate file system. Here's some reasons why:

* it would not be mounted in single user mode
* it could be unmounted or replaced - causing problems if someone tried to login, change a password or anything else that would read/write /tcb.
* /tcb is generally a very small directory in terms of bytes used.
As stated near the top of this thread, you need to investigate why / is filling up rather than avoiding the issue and using potentially dangerous workarounds.

regards,

Darren.

[Bill Hassell]

If this is your production machine, don't do it. While it might work, you are simply putting a bandage over a very serious problem.

First, there should be *NO* core files in / This is likely because root's $HOME is still in the worst possible location: /

To prevent major disasters (such as the / filesystem filling up or accidently removing every file and directory in the entire computer), move root's $HOME immediately. Start by creating a separate root $HOME, perhaps on /home/root. Then move *all* files from / to root's new $HOME and change /etc/passwd from / to /home/root. I call these files: root droppings (sort of like bird droppings). Verify that all is well by logging through another window.

Now when you go to single user mode, you'll get a warning about /home/root not existing. Not to worry, just type: mkdir /home/root and you won't see the error again. To get back root's $HOME with all the files, type: mount /home

Now if /home/root has a significant number of files (and more important, large files), you and your other root users must stop creating large numbers of files in root's $HOME. root is much too powerful to be used for anything except system maintenance. (see reference to root droppings above)

You also need to identify the core files by using the file command: file /core to see what the name of the process or daemon might be and contact the programmer to get it fixed.

Finally, summarize the / directory by filesystem using:

du -kx / | sort -rn > /var/tmp/du.slash

Then look at the top directories to see where all the space is being used. Any application directories must be moved out of the / directory. If necessary, use symlinks to keep the old directory name and move the app to /opt/ where it belongs.

[Christopher McCray_1]

Well put, Bill, as well as the others who have responded previously.