df and du do not agree for AdvFS fileset

[jamesD]

I removed some files from a 100% full AdvFS file system and it still reports 100% capacity.

# du -sk /infra
1233555 /infra

# df -k /infra
Filesystem 1024-blocks Used Available Capacity Mounted on
truechange#data 8192000 8192000 0 100% /infra


Thanks in advance,

James

[Michael Schulte zur Sur]

Hi James,
I hope, the files, you deleted, are not still opened by other programs. You can delete the directory entry but the space in used will not be freed until the last process is stopped.
greetings,
Michael

[Steven Schweda]

Removing files which some process has open
will not free the disk space (until that
process closes the file (intentionally, or by
dying)). "du" may not see them, however.

There are probably other possible
explanations, but that one is popular.

Naturally, rebooting is a reliable way to
kill any offending process, but if that's
undesirable, you may need to whack things
more selectively. (Or throw more (non-open)
files overboard.)

[jamesD]

I should have mentioned that this persists across reboots and unmounts. I have also just run the verify advfs tool and did not see anything suspicious.

[Steven Schweda]

> I should have mentioned that this persists
> across reboots and unmounts.

Too bad.

> I have also just run the verify advfs tool
> and did not see anything suspicious.

That was my next (last?) suggestion.

[Rob Leadbeater]

Hi James,

What does:

# showfdmn -k truechange

give you ?

Is it possible there are some clone filesets, that might be confusing things ?

# showfsets -k truechange

Might give some clues...

Hope this helps,

Regards,

Rob

[Vladimir Fabecic]

Did you try to run "quotacheck" on that file system?

[Mikko Niskanen_1]

Just in case, if this is a Trucluster node, the other node might still hang on the deleted files.

Does any other fileset (mount point) than /infra use file domain "truechange"?

[jamesD]

Quotacheck does not report any discrepancies.

This is not a TruCluster node.

There are no other file sets in this domain.


# showfdmn -k truechange

Id Date Created LogPgs Version Domain Name
49076145.0002fe92 Tue Oct 28 15:00:21 2008 512 4 truechange

Vol 1K-Blks Free % Used Cmode Rblks Wblks Vol Name
1L 8192000 6862840 16% on 128 128 /dev/re1d
# showfsets -k truechange
data
Id : 49076145.0002fe92.1.8002
Files : 245708, SLim= 0, HLim= 0
Blocks (1k) : 63030791036600, SLim= 0, HLim= 0
Quota Status : user=off group=off
Object Safety: off
Fragging : on
DMAPI : off

[Rob Leadbeater]

Hi James,

I'll take a guess that something in the fileset has become corrupt, especially given the rather large blocks figure in the showfsets output.

What sort of files are in the file system ?
Lots of small files by any chance ?

I suspect that you're going to be into the realms of deleting and recreating the domain, and restoring from backups...

Cheers,

Rob