scripting tar and gzip, and resource questions????

[rmueller58]

All,

I have a process on weekends that archives off database exports on an NFS mount from my HP/UX box to a NAS NFS mount. it is a tarball, simple enough.. No problem. I wanted to compress the files to save some space, (I am a bit anal about disk use even on a terabyte system.) I come in on monday to gzip the files and my UX box goes to a crawl. Users can't get in or get locked up, I can't run any commands and and if I can I am lucky to kill the gzip process before I get inundated with screaming users saying that they can't access the system.

I want to run the process on the weekend I've tried kicking off the gzip however I think something is screwed up..

here is my main script:
for dbn in `cat /pei/peitools/bin/explist.txt`
do
export DBNAME=$dbn
export FS=/exportfs/dbexport/onconfig.pei/
echo $FS$DBNAME
echo $FS$DBNAME.tar
echo "======================= $FS$DBNAME =====================================" >>/nas/snapback.log
date >> /nas/snapback.log
ls -la /nas/$FS$DBNAME >> /pei/snapdrive/snapback.log
/sbin/tar -cvf /nas/`date +%m%d%Y`$DBNAME.tar $FS$DBNAME
date >> /nas/snapback.log
echo "====================== $FS$DBNAME =====================================" >>/nas/snapback.log
done
echo "======================================================================" >> /nas/snapback.log
date >> /nas/snapback.log
ls -la *.tar >> /nas/snapback.log
echo "===========================

Below is the script I've tried cronning for later to gzip the file:

# more rotatesnap
cd /nas
for fname in `ls *.tar`
do
gzip $fname
done


I've installed the GNU tar as well and then tar doesn't work. I've tried the tar -xzf / and i get a zero or small file with nothing in it.

I've been scratching my head with this one and looking for ideas..

Thanks..

Rex M.
Educational Service Unit #3, Omaha NE

[Jeff_Traigle · Accepted solution]

gzip is in /usr/contrib/bin, which is not in the minimal PATH that is set when cron runs. Add it to the PATH in the gzip script or just give the full path for gzip when you invoke the command.

[Jose Mosquera]

Hi,

Just to ensure use gzip full path info, and to obtain full compression ratio use -9 option.

cd /nas
for fname in `ls *.tar`
do
/usr/contrib/bin/gzip -9 $fname
done

Also, gzip -tv options show you the status of a .gzip file, where "OK" is succesfully.

Rgds.

[Steven E. Protter]

Sounds like a simple performance problem.

check and possibily modify the following kernel parameters.

maxuprc default 75 bump it up to 150 or 200
nproc
nfile

These are the common ones that can cause this issue.

See the performance data collection script which you should run when you have the problem. This will let you know what other kernel issues exist.

SEP

[rmueller58]

Steven,

Does the kernel change require reboot? or are they realtime?

[rmueller58]

Steven,
looked at params see below:

maxuprc is already defined as 512
maxuprc 512 512 Static N/A M
nproc 3072
nfile 51548