copy command

[Eric Locsin]

Good morning.

A developer created a script that will copy a file (greater than 2 GB) from one location to another. The script is scheduled in CRON. Part of what the script does is that it checks for a certain job to end before it performs the copy. If the job is still running, it will reschedule itself using "at".
The script will run successfully in CRON if the job is completed. The sript will run successfully if we manually start it. It will also run successfully if we manually run the "at" command to schedule it. But if the script runs and it reschedules itself using the "at" command, it will fail with the following message:
"cp: bad copy to /download/ods/prod/item/data/item_MW.txt.tmp: write: File too large. load_item_MW.sh : ...ERRORED with status 1. "

Just to make it clear, we have been able to successfully copy the file manually so I doubt it's a file size issue.

Any ideas?

[Dave Hutton]

Do you have large files turned on the filesystem you are copying to?
servera:root /root/ # fsadm -F vxfs /filesystem
largefiles

Should return largefiles. Otherwise you are limited to 2 gigs or smaller.

I notice your file is called item_MW.txt.tmp, does it write it to a temp location that you by-pass when you manually copy it?

[Eric Locsin]

Largefiles is set on all the filesystems. The copied file is actually copied to the same location but with just the ".tmp" added to the end. There's 49 GB available on the filesystem so I don't believe it's running out of room during the copy.

[A. Clay Stephenson]

If I can take you at your word that the target filesystem is largefiles enabled (and the keyword "largefiles" in /etc/fstab does NOT insure that the filesystem is largefiles enabled) then you are probably hitting ulimit. Have a look at how ulimit is set in /var/adm/cron/.proto.

You could also be hitting quotas for this user but I suspect that ulimit is your boy.
One way to test this is to put a ulimit command with output directed to a file inside the at'ed script.

[Eric Locsin]

I don't have a lot of experience with ulimit. Here's the output of /var/adm/cron/.proto:

# @(#)B.11.11_LR
cd $d
ulimit $l
umask $m
$<

and "ulimit -a" for the user the script runs under:

time(seconds) unlimited
file(blocks) unlimited
data(kbytes) 256000
stack(kbytes) 8192
memory(kbytes) unlimited
coredump(blocks) 4194303
nofiles(descriptors) 3072

[Tim Nelson]

I have never approached this by using a cron job to schedule an "at" job so bear with me.

How about a test.
Create a test script that does something simple. Schedule it in cron to run as an "at" job. This will make sure that croning an at works. Then start chipping away at all the other possibilities.

Or
Cut the losses and define another way to re-run the process. Have it go to sleep or just wait until the next cron iteration.

[A. Clay Stephenson · Accepted solution]

Put the ulimit command in the cron'ed script and at'ed script as I asked and direct the output to a file. That should be most definitive. By the way, are you running this in anything other than the POSIX shell?

[panchpan]

Hi Eric,
Could you please paste the contents of crontab, which has all the commands to allow run of jobs or at command or validation dependent of prior job finishing before executing or copying.

Thank you!

[Eric Locsin]

After a couple of tests, it does look like a ulimit issue. Thanks for the tips!

[panchpan]

For my reference and knowledge Please --

Hi Eric,
Could you please paste the contents of crontab, which has all the commands to allow run of jobs or at command or validation dependent of prior job finishing before executing or copying.

Thank you!