Operating System - HP-UX
1831356 Members
3012 Online
110024 Solutions
New Discussion

differences using ls on like systems

 
SOLVED
Go to solution
Todd Bowden
Valued Contributor

differences using ls on like systems

I have a weird listing whenever I do a "ls -lart" on one system (HP-UX 11) than another (HP-UX 11). Im doing an rsync of data from one system to another making them look exactly alike. After completion of the rsync command I do a du -sk in the directory and the two directories are different slightly. The difference isnt with files but with "." Its 1024 off from system to system. Could there be an explanation for this?

For example:
system 1
==================
drwxr-xr-x 2 hrprd psoft 2048 Sep 10 19:32 .

drwxr-xr-x 2 hrprd psoft 3072 Sep 10 19:32 .

Its not in every directory that its like this but a few.

Anyone know why?

Todd
8 REPLIES 8
Mark Grant
Honored Contributor

Re: differences using ls on like systems

I think I'm right in saying that directories don't shrink when files are removed. If this is the case then I would expect the larger ones to be the originals that in the past have had files removed and the smaller ones would be newly created.

I know this certainly used to be the case as I remember in the old days you had to make your own lost+found directories which had to be big enough to hold the entries they might need in the future. In order to do this you'd create a load of files in the directory and then remove them all again.
Never preceed any demonstration with anything more predictive than "watch this"
Vitek Pepas
Valued Contributor

Re: differences using ls on like systems

It looks like one of the directories contains more files than the other.
Patrick Wallek
Honored Contributor
Solution

Re: differences using ls on like systems

The most probably explanation is the the larger directory at one point had more files in it at one point than the other. While the directory size will grow as more files get added, it will not shrink when files are removed.

You could potentially have an empty directory but the directory itself (.) could be 128000 if it had LOTS of files sometime in the past.

I would worry too much about the directory size itself not being the same as long as all the files in the directory are what you expect them to be.
A. Clay Stephenson
Acclaimed Contributor

Re: differences using ls on like systems

It's because directories are not truncated when files are removed but rather the entries are simply marked as available for reuse. Even if you completely removed the directories from the destination box before doing an rsync there could still be a size difference because the source directory might have "holes" that would not be copied to the target.
If it ain't broke, I can fix that.
Todd Bowden
Valued Contributor

Re: differences using ls on like systems

To all,

I think Mark, Clay, and Patrick have it right. There were other files in these directories that might have made them grow and now Im doing a rsync and deleting the files off the destination server to sync up with the source server. I just thought an easy way of verifying directory size was with "du -sk". Obviously I cant trust that any more. :) Would there be something else I could use to make sure everything was syncing up as normal?

Todd
A. Clay Stephenson
Acclaimed Contributor

Re: differences using ls on like systems

Well, if you are truly paranoid, you could do a find . -type f -exec cksum {} \ | sort > srclist; (or better rather than -exec use xargs but I leave it as -exec for clarity)
and then do the same of the destination directory and create a destlist. You could then diff those files. Cksum does a CRC checksum for all the data in a file so it can take a while. After a few times with this, I think you will come to conclusion the rdist works quite well.
If it ain't broke, I can fix that.
Mark Grant
Honored Contributor

Re: differences using ls on like systems

Todd,

If you were able to start off by making an copy of the directpry system using "tar" or something then they would be the same. I don't know, however, if rsync would be able to handle synchronizing the two afterwards.
Never preceed any demonstration with anything more predictive than "watch this"
Bill Hassell
Honored Contributor

Re: differences using ls on like systems

The sizes of files and directories that are copied from one directory to another will almost always be different! But that's OK since the reason relates to how data is stored.

When you create a new directory, it has very few slots to hold names so it must be expanded to hold new names. If a directory is filled with hundreds or thousands of names, then most of the names are removed, the directory will be shown as having a large size (with ls -ld) but only a few files--the directory does not return the empty name slots. Thus, a copy of a directory will be significantly smaller because the unneeded slots are not created. But the copy is just fine even though the occupied amount of space is smaller.

On the other hand, copying files may result in a larger copy. This is due to sparse files. All Unix system can be directed to create a sparse file by writing an inital record, then, using lseek(2), writing record number 1 million. The files contains 2 records with a million imaginary spaces in between so the file is small on the disk.

But copy this file serially (it was never created serially) and all the missing (undefined) records are supplied as a stream of zeros, thus the copy is significantly larger. The most common sparse file is a core dump which can have empty spaces where gaps in the memory allocation exist. Now the file is identical to any program that reads the data whether it is the original or larger copy--they produce the same exact records when read by an application.

Note also that cksum is a good way to verify files and will produce the same result for sparse files and their copies.


Bill Hassell, sysadmin