File up to 2GB size, even with "large files" enabled on the file system.

[Pablo Matías Rodriguez_1]

I'm experiencing some problems with a log file. This application log file reaches 2GB of disk space and application falls. On the fisrt place I thought, large files was not enabled on file system; but i checked and it was enabled, I also checked ulimit sets
$ ulimit -a
time(seconds) unlimited
file(blocks) unlimited
data(kbytes) unlimited
stack(kbytes) 131072
memory(kbytes) unlimited
coredump(blocks) 4194303
nofiles(descriptors) 2048

Infomation abou os is:

$ uname -a
HP-UX milodon B.11.00 U 9000/800 1683349343 unlimited-user license

what can be failing? The problem is that there is no way to make application to release log file without shuting it down (because application keeps file locked in memory)... what can be failing?

[A. Clay Stephenson · Accepted solution]

First of all, I assume there is spare room in the filesystem itself. With largefiles enabled and ulimit unlimited there are two more things that can come into play. 1) quotas -- check to see if you are running quotas 2) (and more likely) the application itself does not handle large files. This is especially suspect if this is a legacy application.

[Jeff Schussele]

Hi Pablo,

I think the big clue is in the parens in that last sentence.

Have a look at the following kernel params
maxdsiz
maxssiz
maxtsiz
Are any of these "close" to 2GB in size.

I suspect you overflow one of these & that causes the app to fail.
Do you get a core dump?
Does a file /path/to/core give a SIGSEGV indication?
That would be a big clue.

HTH,
Jeff

[Marvin Strong]

When you say you checked, did you check /etc/fstab and saw largefiles as an option?

Are you certain the filesystem was created with largefiles option? If not you can enable it with fsadm.

fsadm -F vxfs -o largefiles /dev/vg??/lvol??

Note: your if your filesystem type is hfs, then, change above to -F hfs.

[Nicolas Dumeige]

Have you consider the possibility that the application is 32 bits and suffers 2 GB limitation ?

[Thierry Poels_1]

hi,

if it's a simple text log file, then how about appending some data manually? In this way you can rule out if it is an application issue or system issue.

good luck,
Thierry.

[KapilRaj]

Trust you are not writing on to an NFS mounted filesystem ! NFS has some limitations !

Kaps

[Pablo Matías Rodriguez_1]

Clay:
First of all, I assume there is spare room in the filesystem itself.
Yes, of course.
1) quotas
There are no quotas enabled
2) (and more likely) the application itself does not handle large files. This is especially suspect if this is a legacy application.
Yes, this seems a pretty logical an plain explanation for this is a 32 bit application, and, 32 bits application has memory limitacion running on 64 bits operating systems. This links the problem with Nicolas Dumeige's answer. "Have you consider the possibility that the application is 32 bits and suffers 2 GB limitation ? " I knew about this limitation but I was not considering this, I think this is pretty simply and clean answer for what can be happening.
Jeff said: I think the big clue is in the parens in that last sentence. Yes, I was not thinking about a memory limitation, but I guess this must be the answer, for 32 bit applications. Kernel parameters can be well set an tunned, but this limitation still exists.
Other considerations were yet considered:
When you say you checked, did you check /etc/fstab and saw largefiles as an option? yes
Trust you are not writing on to an NFS mounted filesystem ! NFS has some limitations !
No, this is not a NFS mounted file system.
I think this is clearly an limitation that comes by application side. Thanks all of you for your quick answer...

[Nicolas Dumeige]

Maybe you can create a pipe name after the name of the log and put a background process to append all pipe output to a log file larger than 2 GB.

Or you can purge (archive) the log wich seems simplier.

But if the application is writting the log when you issue a { cp log log.save ; cat /dev/null > log ; }, you'll get a sparse file.

Cheers

[Todd McDaniel_1]

Pablo,

to check if largefiles is working...

run this command...


#prealloc toddfile 2200000000

[Pablo Matías Rodriguez_1]

Tod:

I run this command, and it worked perfectly... problem is clearly not on file system's largefile mode; but in application bittnes limitation.

[Todd McDaniel_1]

Glad I can help!!

I have this philosophy that works for me... regarding troubleshooting UNIX.


Eliminate the improbable, unplausible, untenable, impossible causes, and you will be left with the probable, plausible, tenable, possible solutions...


I may have assimilated that from someone, but I cant give credit b/c I dont know where I heard it.

I think it is a loose paraphrase of Sherlock Holmes.

[Michael Schulte zur Sur]

Pablo,

have you tried
cp -p log log.old
cp /dev/null log
to cut the log to zero?

Is the log binary?

greetings,

Michael

[Greg OBarr]

I like Nicholas' idea about using a named pipe. I was thinking the same thing and was disappointed someone had already brought it up... ;)

Anyway, it would be interesting to see if that would work. Use mkfifo to create a named pipe, then have a background script that cats from the named pipe into another log file. You could even have the script check the size of the output file each time it cycles through the loop and have it create a new output file when the old one gets to be a certain size...