topic Problems with DIFF in Operating System - HP-UX

Problems with DIFF

Kent A. Dekker — Tue, 14 Nov 2006 11:02:05 GMT

I have a job that runs nightly and diffs two large files (about 1.4 GB each). Job has been running fine and takes about an hour to complete.

We upgraded from 11.0 to 11.11 and now job taking 100% of 1 CPU for hours and not completing. We moved the files to an 11.0 pre-poduction system bu still having problems there so not sure it has anything to do with the 11.11 upgrade.

BDIFF is not an option as it does not return he data the same was we need it. Anyone had any experience with diff and large files? have we reached some threshold with file sizes? Somewhat stumped for now but forging on.

Regards,
Kent Dekker

Re: Problems with DIFF

Peter Godron — Tue, 14 Nov 2006 11:25:03 GMT

Kent,
how much memory is on the machine ?

From "man diff"
"The default algorithm requires memory allocation of roughly six times the size of the file. If sufficient memory is not available for handling large files, the -h option or bdiff can be used (see bdiff(1))."

Re: Problems with DIFF

Kent A. Dekker — Tue, 14 Nov 2006 11:32:42 GMT

We have 16 GB of physical memory. As of this note GLANCE is showing and RSS of 517.3 MB memory used by the process. It has been running for about an hour and 10 minutes.

Re: Problems with DIFF

Jaime Bolanos Rojas. — Tue, 14 Nov 2006 12:15:00 GMT

Kent,

I do not know much about diffs, but other troubleshooting way would be to download tusc and see where and why is it that the program is hanging to be able to find the culprit.

http://hpux.connect.org.uk/hppd/hpux/Sysadmin/tusc-7.8/

Regards,

Jaime.

Re: Problems with DIFF

Sheriff Andy — Tue, 14 Nov 2006 14:15:17 GMT

This is really far reaching, but thought that I might try.

Do you have OnLine JFS installed? If so, what is the output when you run;
# vxlicense -p

We just recently upgraded our server from 11.0 to 11.11. Our backups on our clone file systems went down really bad. We found out that Online JFS didn't carry over the license after the upgrade. Once we got the license issue fixed, all was well.

Once again, this is far reaching but might relate.

Re: Problems with DIFF

Kent A. Dekker — Tue, 14 Nov 2006 14:43:01 GMT

vrts:vxlicense: INFO: Feature name: HP_OnlineJFS [50]
vrts:vxlicense: INFO: Number of licenses: 1 (non-floating)
vrts:vxlicense: INFO: Expiration date: No expiration date
vrts:vxlicense: INFO: Release Level: 22
vrts:vxlicense: INFO: Machine Class: All
vrts:vxlicense: INFO: Site ID: 0

Re: Problems with DIFF

Kent A. Dekker — Fri, 17 Nov 2006 15:19:12 GMT

Working with HP Response center. Thanks for the feedback and help.