Large File systems in Bash Shell

[Kiran_27]

Hi ,
My application is residing in a HP UX 9000/800 Superdome Machine. I am using a bash shell ( my application requires it ) and some time I will have to manupilate files more than 2 GB of size . In bash shell I am not able to do it . It is showing the error " File too large".
The disk supports large file system and in SH shell I am able to do . Is there any way to manipulate large files in Bash shell .

Regards
Kiran

[Muthukumar_5]

Large file support are based on file system based not based on SHELL type. You can use fsadm command to tune a file system with Large Files supported.

hth.

[Robert-Jan Goossens]

Hi Kiran,

Could this be a ulimit problem?

http://forums1.itrc.hp.com/service/forums/questionanswer.do?admit=716493758+1117618489969+28353475&threadId=114562

Regards,
Robert-Jan

[Kiran_27]

I am able to do the manipulations in SH shell but not in Bash Shell for large file system.

So I feel it has something to do with shells.

Regards
Kiran

[Suraj Singh_1]

Hi Kiran,

1. Run "fsadm [-F vxfs] path_of_mount_pt" on the filesystem where this command is giving error. If the o/p is nolargefiles, then you would need to enable largefile support to this filesystem:
# fsadm -F vxfs -o largefiles path_of_mount_pt

2. Issue "ulimit" command, and check the o/p, if o/p normally is unlimited, if not, then set it to unlimited:
# ulimit -Hf unlimited
# ulimit -f unlimited

Regards

[Suraj Singh_1]

Meaning of options available in ulimit:

OPTIONS
-a All current limits are reported.
-c The maximum size of core files created.
-d The maximum size of a process's data segment.
-f The maximum size of files created by the shell. (This is the default option)
-H Change and report the hard limit associated with a resource.
-l The maximum size that may be locked into memory.
-m The maximum resident set size.
-n The maximum number of open file descriptors.
-p The pipe buffer size.
-s The maximum stack size.
-S Change and report the soft limit associated with a resource.
-t The maximum amount of cpu time in seconds.
-u The maximum number of processes available to a single user.
-v The maximum amount of virtual memory available to the process.

[Kiran_27]

The output of the ulimit shows the following
core file size (blocks) 2097151
data seg size (kbytes) 1048576
file size (blocks) unlimited
max memory size (kbytes) unlimited
open files 1200
pipe size (512 bytes) 16
stack size (kbytes) 8192
cpu time (seconds) unlimited
max user processes 1001
virtual memory (kbytes) unlimited

My concern is if we were able to do file manipulation in SH shell , then why we are not able to do it in BASH Shell. Does bash have any limit for the max data size

[Suraj Singh_1]

Hi Kiran,

Did u check the ulimit -a output by changing your current shell to bash shell?

Within the script, try increasing ulimit...
#!/bin/bash
ulimit -Hf unlimited
ulimit -f unlimited
..
..

Regards

[Kiran_27]

Below are the output with different shell
BASH
bash-2.05$ ulimit -a
core file size (blocks) 2097151
data seg size (kbytes) 1048576
file size (blocks) unlimited
max memory size (kbytes) unlimited
open files 1200
pipe size (512 bytes) 16
stack size (kbytes) 8192
cpu time (seconds) unlimited
max user processes 1001
virtual memory (kbytes) unlimited

SH Shell
ulimit -a
time(seconds) unlimited
file(blocks) unlimited
data(kbytes) 1048576
stack(kbytes) 8192
memory(kbytes) unlimited
coredump(blocks) 4194303
nofiles(descriptors) 1200

As you can see it is some setting issue with Bash shell , but could not figure out which one....
Hope you have some idea of what is happening ?
Regards
Kiran

[Suraj Singh_1]

Hi Kiran,

I've been working on it for long, and at last i am able to get something. Here 'getrlimit' & 'setrlimit' might come into picture.

man getrlimit

Do you have gcc installed? If no, download it from http://hpux.connect.org.uk/hppd/hpux/Gnu/gcc-3.4.3/

A small c program will be able to get RLIMIT_FSIZE parameter...

#include
#include
#include
#include

int main(void)
{
struct rlimit rlimit_cur;

getrlimit(RLIMIT_FSIZE, &rlimit_cur);
printf("cur: %lu max: %lu\n", rlimit_cur.rlim_cur, rlimit_cur.rlim_max);
return(0);
}

In my system the o/p was:
cur: 2147483647 max: 2147483647

Now this is equal to 2 GB.

You might have to increase this limit! (I am not very sure)