vxfs: NOTICE: msgcnt 25 mesg 001: V-2-1: vx_nospace - /dev/root file system full (1 block extent)

[Sharma Sanjeev]

Hi

As suggested before u need to check for large files under root,than mv or delete unwanted files.

please post find results

Regards
Sanjeev

[V. Nyga]

Hi again,

to speed up the 'du' command you can try to umount as many directories as possible (oracle directories ...).
Just try a 'umount -a'.

You also can do a 'init 1' to go to the first stage of boot up.
Then you can umount more disks.

>But when I calculate the mounted directory of the root lvm ( /, sbin and /etc), I did not find the 1 GB space.

What's about /dev, /home, /bin ...?

V.

[RUET]

Volkar
I 'm quite sure you are wrong or didn't understand what I wanted to show.

I know that / is full
but may be it's masked data or directories by mount point.

If you create a 2GB files under /opt before mounting a /opt Filesystem.
If then you mount /opt, it will mask the 2GB files. You are no more able to find it, but it's still there and consuming data space in the / file system ...
and you are no more able to find it .. except unmoounting /opt

[Ganesan R]

Hi Aungshuman,

There are few possiblities as people are explained earlier. Common one is open files. If someone have killed one process which has open files on root filesystem, then that space will not be released until you kill the process or reboot the server. You already rebooted the server so it is out.

Now,
>>>But when I calculate the mounted directory of the root lvm ( /, sbin and /etc), I did not find the 1 GB space. It is identical to other system.<<<

Did you calculated each and every subdirectory under root. You may missed out some directory or subdirectory. The best way is use this command to find which directory consumes how much space. So that you can dig into the big directory and delete the unwanted files.

#du -xk / |sort -nr

You can also list the first 20 big directories under root by this command

#du -xk / |sort -nr |head -20

[V. Nyga]

@PR - yes, you're right
Files under mounted directories are possible.
But it makes more sense to search for strange files first.

Btw. - there's no /dev/vg00/lvol4 mounted, also it's defined:
LV Name /dev/vg00/lvol4
LV Status available/syncd
LV Size (Mbytes) 512
Current LE 16
Allocated PE 16
Used PV 1

Normally this is /tmp - there's a new logical volume defined:
/dev/vg00/lvtmp 7340032 26988 6855989 0% /tmp

This could be a good candidate for hidden files ....

Please tell us your results Aunsghu ...

V.

[Aungshuman Paul]

Hi,

One of my colleague solve the problem. As he informed , there was a crash dump file in the /var/adm/crash. Even though I did not check yet. But I have some confussion, carsh folder under var lvm.I will inform you after know the problem and its resolution.

Aungshu

[Sharma Sanjeev]

Hi Aunsghu

Yes, if some crash file is their that can take lot of space.

As per ITRC rules if you are happy with replies please assign points

Regards
Sanjeev

[Cleiton L. Siqueira]

Hi,

First you can verify /etc/lvmconf/*.old files.
Sometimes, this files are large.

Then you can use the lsof app, to verify if any process in memory is using a file that was deleted before.

[Bill Hassell · Accepted solution]

> /dev/vg00/lvol3 1048576 1044544 4024 100% /

Hint: use bdf / to show a single location such as a mountpoint.

/dev/vg00/lvol8 8912896 641304 8209784 7% /var

There appears to be a lot of room in /var. What do the commands:

bdf /var/adm/crash
ll -d /var/adm/crash

show? If it shows "/" at the end, then this (potentially) very large directory is really on /, a very bad location. Does the ll -d command show a symbolic link to /?

> /dev/vg00/lvol3 1048576
This is very large for "/". Apparently, someone has stored non-OS related files in /, another very bad idea. Note also that looking for big files is the wrong approach -- look for big directories. Use this command to find the biggest directories:

du -kx / | sort -rn | head

The two largest directories in / must be /sbin and /etc, each about 30-75 MB. Anything else at the top of the list is in the wrong directory. You can move mistakes like this without affecting your programs by creating another lvol or finding space in a more appropriate directory such as /opt, then move all the files to this new location. Then replace the original directory name with a symbolic link to the new location.

OS files will not fill up 1GB of / space so your problem is related to a program that is storing data in the wrong location.

[Aungshuman Paul]

Thanks,

Now problem is solved. And it was due to /var/adm/crash file which was belong to root lvm.

Thanks again for everyone's help