topic Re: BDF timing out in Operating System - HP-UX

BDF timing out

Josh Dalziel — Tue, 03 Aug 2004 11:55:52 GMT

From time to time I have a machine that is maxed out on disk I/O. While it is maxed out, My file system monitoring scrpit is failing. Basicly BDF times out. I was wondering if anyone has a quick fix for this?

Re: BDF timing out

Dave Olker — Tue, 03 Aug 2004 12:02:46 GMT

I don't believe there is a fix for this.

It's my understanding that when bdf is attempting to get the statistics on the filesystem it issues a "sync" on the filesystems to get a stable picture of the available disk space and inode usage. If the disks are completely I/O bound, then the sync issued by bdf will get queued and bdf will block waiting for the sync to complete.

Also, I know that if you have any NFS mounted filesystems on this system and the NFS server is not responding, that will also cause bdf to hang while it waits for a response from the NFS server, assuming the filesystem is mounted with the "hard" option (default).

Regards,

Dave

Re: BDF timing out

RAC_1 — Tue, 03 Aug 2004 12:04:14 GMT

Restrict your bdf to file systems you want to check.

like bdf /stand etc.

Any nfs mount points??

bdf -l (local FSs)

Re: BDF timing out

Steven E. Protter — Tue, 03 Aug 2004 12:12:24 GMT

Lower I/O use. That will fix it.

bdf usually, eventually returns something if you are patient.

This issue can be triggered also by a stale NFS link. If you have an nfs filesystem mounted and the other server has been booted making the connection stale, bdf will take a long time answering or fail.

bdf will also sometimes fail when a disk goes and a filesystem is mounted.

SEP

Re: BDF timing out

Josh Dalziel — Tue, 03 Aug 2004 12:51:43 GMT

I can not say about checking for every mount point itself, but that would be a long process itself seeing how I have around 150 mount file systems. I have been thinking about using sar and my fstab in a joint venture to tell me what I want to know

Re: BDF timing out

harry d brown jr — Tue, 03 Aug 2004 13:23:24 GMT

What does

showmount -ade

show?

live free or die
harry

Re: BDF timing out

Dave Olker — Tue, 03 Aug 2004 17:05:41 GMT

Are there certain types of filesystems you're interested in, where you could limit the output of bdf to just those types of filesystems? For example, to limit the output to only vxfs filesystems use "bdf -t vxfs". That would eliminate any hfs, cdfs, lofs, nfs, etc. filesystems from the report and possibly return the report faster.

Dave

Re: BDF timing out

Bill Hassell — Tue, 03 Aug 2004 17:27:37 GMT

bdf doesn't timeout. It issues requests to the filesystem routines to sync and then it summarizes the results. If bdf hangs, it is usually due to NFS (don't scan NFS filesystems) or a dead/dying disk. First, change your bdf to bdf -l or bdf -t vxfs (and perhaps separately run bdf -t hfs). You don't want to analyze NFS or CDFS in your space monitoring. Note also that running bdf on a really busy system (busy=creating and/or removing lots of inodes) will impact everything that uses the disks. That means that your script should never run every minute, perhaps once every 10 minutes.

Also add a high water marker in your script so that once it reports a problem, it will not report the same problem again unless it gets worse.

For dead/dying disks, you'll need to check in syslog for error messages as well as the word "stale" from vgdisplay -v (for all volume groups that have mirrors). For non-mirrored volumes, look for anything coming out of stderr in vgdisplay and lvlnboot -v. Put those tests in your diskspace checker since they are as much of a problem as full filesystems.