HP ux root filesystem full.

[sanjay_20]

Hi

In my server Hp9000-K series Hp ux 11 is installed .
The root filesystem is in a logical volume. Now the problem is that the root file system is showing 97 % .Its causing the warning message as "root full" & the application like "exceed" is exiting frequently.
Can someone tell me how to extend the root file system size in this condition without disturbing the existing OS & data on it.

Thanks in advance
Sanjay

[Robert-Jan Goossens]

Hi Sanjay,

The first thing you could check is if someone made an error using a tar command.

# ll /dev/rmt

check if there is a large file omn

check for other large files in /

# find / -xdev -size +1000 -exec ll {} \;

Best regards,
Robert-Jan

[Joseph Loo]

hi,

u may like to look for any core files or unwanted files in the "/" filesystem, cause there is no way of increasing the / filesystem without rebooting (that includes /stand and swap filesystem).

usually, we tend not to dump any unnecessary files in the root directory.

regards.

[melvyn burnard]

well it might be quicker and easier to look for what may be filling up / file system.

For example, check in /dev and make sure no-one has accidentally created or put a large normal file there. A simple example is where someone does a tar or other such command and specifies an incorrect device, e.g. /dev/rmt/om instead of 0m.
The next place to look is if you do not have /var/adm/crash as a separate file system, do you have crash dumps in there? If yes, then unless they are relevant, remove them.

Another favourite is /var/adm/sw/save, where older versions of patches are saved to be used in the event of you wishing to revert to a previous patch.

Just a few thoughts.

[baiju_3]

Also ,

if /var is not a seperate file system check,

/var/adm/btmp
/var/adm/crash


Thanks,
BL.

[Betty Fessler]

Another option is to commit patches if the system is stable.

[Hoang Chi Cong_1]

And after you do everything to reduce the root filesystem but the trouble persit,
You have to extend root volume in single user mode. (Recommend make a full system backup before extend)
Regard,
HoangChiCong

[generic_1]

You might want to keep a session open to that system as root until you get the problem resolved. That is a pretty full. If it fills up you are gonna have extra reboots :). Make tape recoverying with the interactive option is a easy way to rebuild the system and choose the new correct filesystem sizes/layouts too assuming your disks have enough space to have that flexability. Good Luck.

[Bill Hassell · Accepted solution]

In your case, root filesystem does NOT have just OS files and directories. The reason is that an 11.00 version of HP-UX needs about 50 to 75 megs--no more. If your filesystem has a lot more in it, there have been one or more mistakes.

The first is the size of /dev. Do this:

du -ks /dev

It should report less than 200k, more like 80k. If it is MUCH larger then someone has misspelled /dev/null (ie, /dev/rmt/om, /dev/nul or /dev/null2 or soemthing similar) and it would have to be root or a root process (since /dev is 755 permissions). There are no ordinary files in /dev:

find /dev -type f -exec ll {} \;

If there are, move or remove them from /dev. Then there may be application of developer junk left in the / directory 'because it's easy to remember /' These directories do not belong in /...move them. And of course there may be some logfile (NEVER allow logfiles in /) that is growing. Find the culprit(s) by looking for big directories (NOT big files):

du -kx / | sort -rn | head -20

It should look something like this:

69232 /
41080 /etc
38448 /sbin
26680 /etc/vx
21656 /etc/vx/type
10160 /etc/opt
8616 /etc/vx/type/static
6344 /etc/vx/type/gen
5904 /sbin/fs
5400 /etc/opt/resmon

The numbers are in kbytes. These are all OS directories. I suspect that there are other directories at the top of your list that need to be moved. Or there are very large files in / which you can see with:

ll / | sort -rnk5 | head -20

NOTE: You root HOME directory is probably in / (very bad idea). Create a new location for root's HOME (perhaps /home/root) and move all the ordinary files to that location. Find ordinary files with:

ll / | grep ^-

NOTE: Some of those files may be part of some developer's package or a vendor's package. If they are and they are quite large, use symbolic links to move them to /opt.

Enlarging the / directory requires a re-installation (or Ignite/UX which is the same thing) and is normally not needed by managing the / space carefully.

[sanjay_20]

Hi to all,

I will let you know the findings of the suggestions.

Thanks a lot
Sanjay

[Nguyen Anh Tien]

root (/) is unable to extend even you are in single mode.
The best way you can do is finding big file or core file inside this file system
1. find core file:
find / -name core -exec ll {} \;
2. find all big file or folder inside one folder
#du -sk *|sort -nr
Regard.
P/S: pls do not forgot to assign point to each solution.
tienna

[sanjay_20]

Dear all,

1) There is no any unwanted file in the /dev directory.
2)checked /var no any big file , actually /var is on the separate vol.
3) The vol details are as ;
*Volgrop : vg00
*Physical volumes for this group : /dev/dsk/c0t6/d0
/dev/dsk/c0t5/d0
/dev/dsk/c0t4/d0

*Logical volumes on the group : /dev/vg00/lvol1 to /dev/vg00/lvol10

* lvol on /dev/dsk/c0t6d0
/ on /dev/vg00/lvol3
/stand on /dev/vg00/lvol1

* lvol on /dev/dsk/c0t4d0
/var on /dev/vg00/lvol6
/var/tmp on /dev/vg00/lvol9
.
.
.
.
and other data partitions on the remaining lvol.

Still the / size is showing 97% .

Is it possible using any Hp tool/software to online extension of the root partition without disturbing the exsting data on the volumes ?

Thanks
Sanjay

[Bill Hassell]

> Is it possible using any Hp tool/software to online extension of the root partition without disturbing the exsting data on the volumes ?

Not possible. The first 3 volumes on your boot disk must be contiguous (cannot be extended). If you don't find the files and directories that do not belong in /, you have no choice but to create an Ignite/UX backup for VG00, then restore it interactively and make / bigger.

However, before you go to all that trouble, can you post the results of this command:

du -kx / | sort -rn | head -20

If /sbin and /etc are not at the top of your list, use this command in the largest directories:

ll /largedir | sort -rnk5 | head -20

This will show the largest files in those rogue directories.

[sanjay_20]

Hi
1)Please see the output of the
#du -kx / |sort -rn |head -20

it looks the /sbin / etc are on the top .

2) I have found two log files as /etc/rc.log
/etc/rc.log.old

I have moved these files to other location. with these it increases the no of files available in / but still the percenrage size remains 97% .
Is it Ok? or revet back these files ?

Thanking you
Sanjay

[Kent Ostby]

/sbin looks a bit bigger than one would expect.

Can you post the output from:

find /sbin -size +1000000c

This will be files greater than 1 megabyte.

[Bill Hassell]

Yes, /sbin is way too largte, by about 20megs. The problem is located in /sbin/init.d (it's the largest diectory in /sbin and should only be about 1000Kb (1 meg) NOT 28Megs!! DO this to find the bad file(s):

ll /sbin/init.d | sort -rnk5 | head -20

The very largest files in this directory should be less than 40Kb. Somone has dropped a giant file (or several) in this directory.

NOTE: rc.log and rc.log.old are always in the /etc directory. When you shutdown and reboot, these files will bve recreated. If these are very large (larger than 20-40Kb) then read them and look for problems. rc.log is the bootup log and rc.log.old is the shutdown log.

[sanjay_20]

Hi ,
Please find the output of the commands,

Kindly suggest

Thanks
sanjay

[Stephen Keane]

Lose the nohup.out to start with!

[RikTytgat]

It's quite obvious: remove the file /sbin/init.d/nohup.out

It hasn't been changed recently, so the nohupped process has probably exited.

Rgds,
Rik

[Anthony Lennan]

You should remove nohup.out. That is just a log file that has been created when some one has used the nohup command.

Moving it or removing it should resolve help resolve your disk space issues.

Rgds,
Anthony

[Bill Hassell]

As suggested, the problem is with the nohup.out command. Someone ran a program using the nohup command, probably in the background:

nohup some_program &

and this program produced an immense amount of data. Since this file is in the /sbin/init.d directory, it was very likely cause by a locally developed startup script that was improperly designed.

The /sbin/init.d directory is for startup scripts that start processes and the process runs by itself, in the background. While the above commands (nohup and &) will do this, processes that are to run forever at startup must produce *NO* output to stdout and stderr. The reason is that there is no terminal associated with a process that is to run by itself. Instead, any output should be directed to a logfile, ideally loacted in /var/adm with the rest of the logfiles.

Take a look at the contents of nohup.out and you'll likely recognize the program or script that produced this giant file. Then move it or remove it and you'll have avoided reloading you operating system because of a full / disk.

[sanjay_20]

Hi ,

The File " nohup.out" is moved to other filesystem & now the root size is around 70%.
This avoided to resize the root filesystem.

Thanks to all of you for your valuable suggestions.Specially my very thanks to Bill Hassel for his valuable suggestions & detailed informatiion provided for my queries.

with thanks again !!
Sanjay

[Michael Steele_2]

this is nonesense - /var is not apart of / where the problem is