copy a filesystem best solution (local host or remote host)

[support_billa]

hello,

>Who made that statement and where is it documented?

>I see nothing in cksum(1), nor in the source that indicates any limits.

> Other than the command line limit that find(1) uses.

we got these informations hp, we made an software call and they send it to the experts.

they told us, that is a problem of cksum(1). i can ask abot more informations, if you want ?

> On 11.23, I have no problems passing in 33,800+ files to cksum.

i needed a little bit time (extend vg,...) to create the same filesystem at hpux 11.23.

i tested it with 106948 files and it is the same problem :-((

regards

[James R. Ferguson]

---

@support_billa wrote:

we got these informations hp, we made an software call and they send it to the experts.

they told us, that is a problem of cksum(1). i can ask abot more informations, if you want ?

> On 11.23, I have no problems passing in 33,800+ files to cksum.

i needed a little bit time (extend vg,...) to create the same filesystem at hpux 11.23.

i tested it with 106948 files and it is the same problem :-((

---

Hi:

My quess would be that it's not the number of files but rather the length of the argument list that 'cksum' can handle.  Since you have an open case (?) to Support, why not ask for specifics?  Isn't opensource wonderful? :-)

Regards!

...JRF...

[support_billa]

hello,

@ the length of the argument list that 'cksum' can handle.  Since you have an open case (?) to Support,

@ why not ask for specifics?  Isn't opensource wonderful? :-)

the length of the argument list that 'cksum' can handle is the problem.

the support close with this argument the case, i have an idea, i will tested it at a Linux Server . you will get the informations.

regards

[support_billa]

about my question of  command "sync" :

test with filesystem size: 13 GB:  166  copies and 78 were wrong ( after "fsadm" tasks it isn't possible to umount )

test with filesystem size: 256 MB: 521 copies and only 2 were wrong ( after "fsadm" tasks it isn't possible to umount )

regards

[James R. Ferguson]

---

@support_billa wrote:

about my question of  command "sync" :

---

Hi:

The 'syncer' daemon (which you invoke with 'sync') flushes modifed file buffers from memory to disk.  There is no harm in forcing this action and it is common to do 'sync;sync;sync' before starting a backup.  If you run 'bdf', by default, and automatic 'sync' occurs to give a better reflection of disk utilization.

Regards!

...JRF...

[Dennis Handly]

>My guess would be that it's not the number of files but rather the length of the argument list that 'cksum' can handle.

Yes but I filled up my argument list.  I may not have had the same number and individual lengths, so there could be some bugs in how find(1) collects the list but this doesn't seem like a limitation.  It quacks like a bug.

>why not ask for specifics?  Isn't opensource wonderful? :-)

Right.  (I assume you mean it isn't open?)

>the length of the argument list that 'cksum' can handle is the problem.

But there is nothing in the HP's cksum that cares about the length.  It just indexes into argv until the end.

[James R. Ferguson]

---

@Dennis Handly wrote:

>My guess would be that it's not the number of files but rather the length of the argument list that 'cksum' can handle.

Yes but I filled up my argument list.  I may not have had the same number and individual lengths, so there could be some bugs in how find(1) collects the list but this doesn't seem like a limitation.  It quacks like a bug.

---

So it seems; data-dependent perhaps.

---

@Dennis Handly wrote:

>why not ask for specifics?  Isn't opensource wonderful? :-)

Right.  (I assume you mean it isn't open?)

---

Yes, I had my tongue in my cheek when I said that, wishing we could see the HP source :-)

---

@Dennis Handly wrote:

But there is nothing in the HP's cksum that cares about the length.  It just indexes into argv until the end.

---

This is what I would expect but was wondering if for some reason the argument list was copied and or otherwise constrained.   Thanks for the info.

Regards!

...JRF...

[support_billa]

hello,

i have a new issue :

i spoke also about compare directories with "diff".

does command "diff" a size limit ? maybe 2 GB ?

case:

#/old_fs/file : 8932617921 Bytes

diff /old_fs /new_fs

Error :
diff: input file /old_fs/file: Value too large to be stored in data type

how can i find in a fs , if a file is greater maybe 2 GB :

my proposal:
find /fs  -size +2097152000c | wc -l

regards

[James R. Ferguson]

---

@support_billa wrote:

how can i find in a fs , if a file is greater maybe 2 GB :

my proposal:
find /fs  -size +2097152000c | wc -l

---

find /fs -type f -size +$((2*1024*1024*1024))c -exec ls -ld {} +

Regards!

...JRF...

[Dennis Handly]

>find /fs  -size +2097152000c | wc -l

(Any reason you want to use wc and lose info the detailed info?  Also this isn't the exact value of 2 Gb.)

>find /fs -type f -size +$((2*1024*1024*1024))c -exec ls -ld {} +

If not using ksh, this may overflow the 32 bit int for sh.

[support_billa]

    about my question of  command "sync"  :

The 'syncer' daemon (which you invoke with 'sync') flushes modifed file buffers from memory to disk.  There is no harm in forcing this action and it is common to do 'sync;sync;sync' before starting a backup.  If you run 'bdf', by default, and automatic 'sync' occurs to give a better reflection of disk utilization.

 Today i got info about FS Command /usr/sbin/fsadm -F vxfs  -e -d, the duration was very long sometimes , a colleague  tried to stop with "kill -9" , after that , file "lost+found/.fsadm"  were created !

today i copied a filesystem named /PATCHDEPOT  (wich includes with "swreg" registered HPUX SW-Depot's ) .
FS has 1511631 Files and  430832 Directories and after copy and fsadm steps, "umount" was after 2. attempt successfully , my meaing : "fsadm" or  "vxupgrade" isn't finished til 100 %.

maybe in the file cache because i didn't find a process with "lsof"

info:

filesystem_fsadm_action: /usr/sbin/fsadm -F vxfs  -b 314769408 /new/PATCHDEPOT
filesystem_fsadm_action: /usr/sbin/fsadm -F vxfs  -e -d /new/PATCHDEPOT

filesystem_version_upgrade: /sbin/vxupgrade -n 6 /new/PATCHDEPOT
filesystem_version_upgrade: /sbin/vxupgrade -n 7 /new/PATCHDEPOT

umount_local_fs: FS : "/new/PATCHDEPOT"
umount_local_fs: /usr/sbin/umount "/new/PATCHDEPOT"
attempt  /usr/sbin/umount "/new/PATCHDEPOT" 1 of 10  !

regards

[James R. Ferguson]

---

@support_billa wrote:

Today i got info about FS Command /usr/sbin/fsadm -F vxfs  -e -d, the duration was very long sometimes , a colleague  tried to stop with "kill -9" , after that , file "lost+found/.fsadm"  were created !

today i copied a filesystem named /PATCHDEPOT  (wich includes with "swreg" registered HPUX SW-Depot's ) .
FS has 1511631 Files and  430832 Directories and after copy and fsadm steps, "umount" was after 2. attempt successfully , my meaing : "fsadm" or  "vxupgrade" isn't finished til 100 %.

maybe in the file cache because i didn't find a process with "lsof"

---

As I recall, I told you in another thread that the 'lost+found/.fsadm' file is created the first time any 'fsadm' is run.  The file is used as a lock for the process.  Once created, it isn't removed.  Your 'kill -9' [while stupid] had nothing to do with that.

With regard to you second comment about unmounting, if you have any open files in the filesystem, or you are 'cd'ed into it then you won't be able to unmount.  It's not clear to me what you did or observed or why.

I would never kill an LVM or 'fsadm' process; certainly not with a 'kill -9'.  For any process, using a 'kill -9' means that the process can't catch and handle the signal.  Specifically this means that there is no way to cleanup temporary files and shared memory, or gracefully do anything!

Plan, slow down, and be patient.

Regards!

...JRF...

[support_billa]

@ It's not clear to me what you did or observed or why.

i make following steps :

filesystem_fsadm_action: /usr/sbin/fsadm -F vxfs  -b 314769408 /new/PATCHDEPOT
filesystem_fsadm_action: /usr/sbin/fsadm -F vxfs  -e -d /new/PATCHDEPOT

filesystem_version_upgrade: /sbin/vxupgrade -n 6 /new/PATCHDEPOT
filesystem_version_upgrade: /sbin/vxupgrade -n 7 /new/PATCHDEPOT

then i want to umount the filesystem !!!!
it is a temporary Mountpoint , which only exists for copy a filesystem. so nobody will use this filesystem !
i didn't find a process with "lsof".

so i make a workaround , and try 10 times to umount the filesystem :

umount_local_fs: FS : "/new/PATCHDEPOT"
umount_local_fs: /usr/sbin/umount "/new/PATCHDEPOT"
attempt  /usr/sbin/umount "/new/PATCHDEPOT" 1 of 10  !

this week i copied 40 filesystems (with millions of files) and this case occurs only at large filesystems.


@As I recall, I told you in another thread that the 'lost+found/.fsadm' file is created the first time any 'fsadm' is run.  The file is used @as a lock for the process.  Once created, it isn't removed.  Your 'kill -9' [while stupid] had nothing to do with that.

yes i know, for one filesystem ( 40 GB, 500.000 files ) it ran about 1 hour.

[support_billa]

hello,

a new topic occurs: after copy process the quotas of a filesystem were not ok.

i made : quotacheck_vxfs (VxFS file system quota consistency checker ) , ok ?

- how can i check, if a filesystem have quotas ?

- check if a file root_directory/quotas exists ?

- check if quotas are ok : quotacheck -F vxfs -V fs;echo $?

regards