topic Re: copy command in Operating System - HP-UX

copy command

Eric Locsin — Wed, 19 Sep 2007 08:08:21 GMT

Good morning.

A developer created a script that will copy a file (greater than 2 GB) from one location to another. The script is scheduled in CRON. Part of what the script does is that it checks for a certain job to end before it performs the copy. If the job is still running, it will reschedule itself using "at".
The script will run successfully in CRON if the job is completed. The sript will run successfully if we manually start it. It will also run successfully if we manually run the "at" command to schedule it. But if the script runs and it reschedules itself using the "at" command, it will fail with the following message:
"cp: bad copy to /download/ods/prod/item/data/item_MW.txt.tmp: write: File too large. load_item_MW.sh : ...ERRORED with status 1. "

Just to make it clear, we have been able to successfully copy the file manually so I doubt it's a file size issue.

Any ideas?

Re: copy command

Dave Hutton — Wed, 19 Sep 2007 08:11:55 GMT

Do you have large files turned on the filesystem you are copying to?
servera:root /root/ # fsadm -F vxfs /filesystem
largefiles

Should return largefiles. Otherwise you are limited to 2 gigs or smaller.

I notice your file is called item_MW.txt.tmp, does it write it to a temp location that you by-pass when you manually copy it?

Re: copy command

Eric Locsin — Wed, 19 Sep 2007 08:28:30 GMT

Largefiles is set on all the filesystems. The copied file is actually copied to the same location but with just the ".tmp" added to the end. There's 49 GB available on the filesystem so I don't believe it's running out of room during the copy.

Re: copy command

A. Clay Stephenson — Wed, 19 Sep 2007 09:22:45 GMT

If I can take you at your word that the target filesystem is largefiles enabled (and the keyword "largefiles" in /etc/fstab does NOT insure that the filesystem is largefiles enabled) then you are probably hitting ulimit. Have a look at how ulimit is set in /var/adm/cron/.proto.

You could also be hitting quotas for this user but I suspect that ulimit is your boy.
One way to test this is to put a ulimit command with output directed to a file inside the at'ed script.

Re: copy command

Eric Locsin — Wed, 19 Sep 2007 10:05:23 GMT

I don't have a lot of experience with ulimit. Here's the output of /var/adm/cron/.proto:

# @(#)B.11.11_LR
cd $d
ulimit $l
umask $m
$<

and "ulimit -a" for the user the script runs under:

time(seconds) unlimited
file(blocks) unlimited
data(kbytes) 256000
stack(kbytes) 8192
memory(kbytes) unlimited
coredump(blocks) 4194303
nofiles(descriptors) 3072

Re: copy command

Tim Nelson — Wed, 19 Sep 2007 12:17:12 GMT

I have never approached this by using a cron job to schedule an "at" job so bear with me.

How about a test.
Create a test script that does something simple. Schedule it in cron to run as an "at" job. This will make sure that croning an at works. Then start chipping away at all the other possibilities.

Or
Cut the losses and define another way to re-run the process. Have it go to sleep or just wait until the next cron iteration.

Re: copy command

A. Clay Stephenson — Wed, 19 Sep 2007 12:22:01 GMT

Put the ulimit command in the cron'ed script and at'ed script as I asked and direct the output to a file. That should be most definitive. By the way, are you running this in anything other than the POSIX shell?

Re: copy command

panchpan — Wed, 19 Sep 2007 22:27:20 GMT

Hi Eric,
Could you please paste the contents of crontab, which has all the commands to allow run of jobs or at command or validation dependent of prior job finishing before executing or copying.

Thank you!

Re: copy command

Eric Locsin — Thu, 20 Sep 2007 09:31:11 GMT

After a couple of tests, it does look like a ulimit issue. Thanks for the tips!

Re: copy command

panchpan — Thu, 20 Sep 2007 21:47:04 GMT

For my reference and knowledge Please --

Hi Eric,
Could you please paste the contents of crontab, which has all the commands to allow run of jobs or at command or validation dependent of prior job finishing before executing or copying.

Thank you!