1834558 Members
3259 Online
110069 Solutions
New Discussion

/ at 91%

 
Claude Fortin
Occasional Advisor

/ at 91%

Hello,

When I make a bdf, my / was at 91%
But yesterday, it was at 66%

Nothing change on the server, no new file?
Bad statistics?
13 REPLIES 13
Ron Irving
Trusted Contributor

Re: / at 91%

this happened to me once. Look in the /dev directory for large 'rmt' devices. Did you attempt a backup recently? It may have written it out to the file, instead of the tape.

Hope this helps.

rri
Should have been an astronaut.
Santosh Nair_1
Honored Contributor

Re: / at 91%

Typically this happens to me when some process is trying to write to where it thinks that a filesystem exists...but for some reason or other, there is no filesystem. I would start by doing a bdf on /* and then du -sk on all local directories to find which is taking up the space.

Hope this helps.

-Santosh
Life is what's happening while you're busy making other plans
Volker Borowski
Honored Contributor

Re: / at 91%

Hi,

most likely a coredump has been written due to a died application.

Go with "fine / -name core -print" to locate it.

Check if your /var/adm/syslog/syslog.log contains information.

In addition, it would be helpfull, if you could qualify the amount of MB you lost.

Best regards
Volker
Sachin Patel
Honored Contributor

Re: / at 91%

Hi Claude,
Either big cron file or you might have perform backup and it has created file in /dev/rmt/

Sachin
Is photography a hobby or another way to spend $
James R. Ferguson
Acclaimed Contributor

Re: / at 91%

Hi Claude:

Look for 'core' files and look for files in /dev that don't belong there (i.e. regular files as opposed to non-character-special files beginning with "c" in the mode bits field (the first) as output by 'ls'.

It is likely that a tape backup device file specification was mis-keyed and instead of "0m" (zero) the letter "o" was substituted.

Regards!

...JRF...
Jim Turner
HPE Pro

Re: / at 91%

Hi Claude,

To find the location of your offender:

# for a dir-by-dir summary
du -kxs /* | sort -rn | more

# for the whole list
du -kx / | sort -rn | more

All the best,
Jim
Bill McNAMARA_1
Honored Contributor

Re: / at 91%

Start with this:

# find / -local -size +30000 -print

Bill
It works for me (tm)
Shannon Petry
Honored Contributor

Re: / at 91%

Another thing to remember is that the % of disk used means squat! 91% of 100MB and 91% of 1GB are quite different.

While the system reports and enforces writes based on the percentage, the default size for / in HP-UX 11.0 is about 86MB. A simple core file can make up the 30% difference between days.

Regards,
Shannon
Microsoft. When do you want a virus today?
Claude Fortin
Occasional Advisor

Re: / at 91%

Thanks for suggestions, did not find cause, decided to reboot when used increased to 92% and " / " went back to 33% ??
Jim Turner
HPE Pro

Re: / at 91%

Hi (again) Claude,

What your observation tells me is this:

1. Someone or something created an enormous file (one or more) under /.

2. One or more processes had the file open.

3. The file was deleted.

4. bdf would still show the space as used until every last proc using that file was killed.

5. Rebooting killed said proc(s).

6. bdf now reflects reality.

All the best,
Jim
Roger Baptiste
Honored Contributor

Re: / at 91%

Claude,

The suggestions so far have been comprehensive covering all possibilities.
Since you say, only a reboot helped clearing
the space, a runaway process writing into
the / filesystem could have been the culprit.
To figure out such processes in the future:
* Make use of top or glance and see which
process is consuming a lot of CPU time
rapidly.
* look at ps listing and see the CPU times
of processes which are changing.

Reboot may have solved the problem , but
it could be a temporary lull, before it
occurs again. (rest of the things being
the same).

From a File check point of view:
find / -xdev -print >/tmp/filelist
and then a
du -sk on the files listed in /tmp/filelist
would give you a clue on the largest files
under root. (-xdev stops scanning below
root filesystem).

HTH
raj
Take it easy.
Darrell Allen
Honored Contributor

Re: / at 91%

Hello Claude,

Just a few additions to what others have already said. It could be a log file that was growing and was "rolled" when you booted. If it was /var growing I'd think something in /var/adm and perhaps something was writing much to /var/adm/syslog/syslog.log. But since this is /...

To help identify the culprit, you can narrow it down to suspect directories like this:

find / -type d -xdev | xargs du -sk >/file1

Wait awhile (30 minutes?)

find / -type d -xdev | xargs du -sk >/file2
join -j 2 file1 file2 >file3

file3 will be formatted as:
directory firstsize secondsize

Simply look for large discreprencies between the first and second sizes to narrow down to a directory. Then you can hopefully identify the file.

Darrell
"What, Me Worry?" - Alfred E. Neuman (Mad Magazine)
Darrell Allen
Honored Contributor

Re: / at 91%

Major omission! file1 and file2 must be sorted on their 2nd field before join will perform correctly. Use this:

find / -type d -xdev | xargs du -sk >/file1
sort -k2,2 file1 >sorted1

Wait awhile (30 minutes?)

find / -type d -xdev | xargs du -sk >/file2
sort -k2,2 file2 >sorted2
join -j 2 sorted1 sorted2 >file3

file3 will be formatted as:
directory firstsize secondsize

Sorry for the boo-boo!

Darrell
"What, Me Worry?" - Alfred E. Neuman (Mad Magazine)