Disable dmesg vx_nospace alert

[Shahril M]

Hi folks,

I have some file systems where it is normal for them to reach 100%. I have already taken them into consideration for my bdf monitoring script.

However, how do I disable vx_nospace alerts in dmesg for such FSs?

msgcnt 15 vxfs: mesg 001: vx_nospace - ...

Rgds,
Shah

[James R. Ferguson]

Hi Shah:

You don't; you can't.

The 'dmesg' buffer holds critical alerts. A filesystem at 100% utilization is a critical event. You need to expand your filesytem capacity so that you do not reach this level. That, and only that, will "disable" the 'vx_nospace' message generation.

Regards!

...JRF...

[Shahril M]

Hi James,

I appreciate the reason behind "You don't". But the nature of this filesystem is that the application will populate it up till max.

I have already asked the application vendor how this can be avoided, but their response is as above - the nature of this filesystem.


Rgds,
Shah

[Shahril M]

Hi folks,

I forgot to add one more thing. Three of such application servers I inherited also have such file systems, but they do not generate the dmesg alerts for them - I cannot remember whether alerts for other FSs (eg. /var) gets generated.

This particular server is one installed during my time.

They are all running 11.11.


Rgds,
Shah

[Bill Hassell]

Your vendor has created a very poor piece of code if it is designed to request more space than is available -- all the time. *ALL* operating systems will have problems with full filesystems and code that does not manage this problem needs rework. I manage several systems that take in massive amounts of data every day from hundreds of sites and the amount varies every day. The code to store the data watches disk space and automatically resizes the filesystems larger or creates an additional lvol on the fly. The code also reduces the filesystems in the same way and all this happens every day.

If you are concerned that the vx_nospace messages are rolling off the rest of your dmesg buffer, be sure to log dmesg using cron like this:

05,15,25,35,45,55 * * * * /usr/sbin/dmesg - >>/var/adm/dmesg.log

Now you won't lose any messages and the messages get a timestamp. Like any logfile, be sure to trim them regularly.

[Shahril M]

Hi Bill,

I only agree that the vendor should have provided a way to configure how much of the FS I want to let it grow up to.

You see, the FS is used like a cache, so it is my users' best interests that there be as much data in there as possible.

The app does not "request more space than is available". Once it is full, it will adopt some replacement algorithm.


Rgds,
Shah

[Duncan Edmonstone]

Shah,

>>The app does not "request more space than is available". Once it is full, it will adopt some replacement algorithm.


If that were true you would get one and only one vx_nospace message per filesystem as it filled up for the first time. The message is only written when a process requests more extents on the filesystem than are available - it isn't repeated until a process requests more extents again.

As this is a kernel message, there's no way to turn it off, so you will have to live with it. If you ISV can't 'fix' his code (and I would agree with Bill that this is a 'broken' implementation), the you're only option is to do as suggested and write out dmesg to a file - maybe you could filter out what you want to ignore - e.g. if your filesystem that is always full is on LV /dev/vg01/lvol1 you could do the following:

05,15,25,35,45,55 * * * * /usr/sbin/dmesg - | grep -v "vx_nospace - /dev/vg01/lvol1 " >>/var/adm/dmesg.log


HTH

Duncan

[Ganesh Balraman]

Hi,

# dmesg -n4 sometimes stops the level 4 erorr meaaseges.

NOt sure.

regds
Ganesh

[Shahril M]

Hi Duncan,

I had thought that the dmesg alert appears at some interval as long as the FS is full, not whenever "a process requests more extents". If that were the case, then the application should be smarter by checking what is avail and not attempt to "request more".

So, allow me some time to monitor whether the alert gets repeated, as I do not expect space to be released by the app.

If it does not repeat, then I do not have to be concerned about this after all.

Till then, thanx.


Rgds,
Shah

[Bill Hassell]

You can easily test this by creating a very small lvol and filesystem(ie, 10-20 megs), then mount the filesystem and cp a much larger file into that filesystem. You should get a single entry in dmesg. Do it again and you'll see another. Don't do anything to the full filesystem and there should be no further messages. There is nothing in the OS that monitors filesystem sizes on a regular basis.