topic Large File systems in Bash Shell in Operating System - HP-UX

Large File systems in Bash Shell

Kiran_27 — Wed, 01 Jun 2005 04:23:42 GMT

Hi ,
My application is residing in a HP UX 9000/800 Superdome Machine. I am using a bash shell ( my application requires it ) and some time I will have to manupilate files more than 2 GB of size . In bash shell I am not able to do it . It is showing the error " File too large".
The disk supports large file system and in SH shell I am able to do . Is there any way to manipulate large files in Bash shell .

Regards
Kiran

Re: Large File systems in Bash Shell

Muthukumar_5 — Wed, 01 Jun 2005 04:33:00 GMT

Large file support are based on file system based not based on SHELL type. You can use fsadm command to tune a file system with Large Files supported.

hth.

Re: Large File systems in Bash Shell

Robert-Jan Goossens — Wed, 01 Jun 2005 04:39:10 GMT

Hi Kiran,

Could this be a ulimit problem?

http://forums1.itrc.hp.com/service/forums/questionanswer.do?admit=716493758+1117618489969+28353475&threadId=114562

Regards,
Robert-Jan

Re: Large File systems in Bash Shell

Kiran_27 — Wed, 01 Jun 2005 04:58:11 GMT

I am able to do the manipulations in SH shell but not in Bash Shell for large file system.

So I feel it has something to do with shells.

Regards
Kiran

Re: Large File systems in Bash Shell

Suraj Singh_1 — Wed, 01 Jun 2005 05:03:01 GMT

Hi Kiran,

1. Run "fsadm [-F vxfs] path_of_mount_pt" on the filesystem where this command is giving error. If the o/p is nolargefiles, then you would need to enable largefile support to this filesystem:
# fsadm -F vxfs -o largefiles path_of_mount_pt

2. Issue "ulimit" command, and check the o/p, if o/p normally is unlimited, if not, then set it to unlimited:
# ulimit -Hf unlimited
# ulimit -f unlimited

Regards

Re: Large File systems in Bash Shell

Suraj Singh_1 — Wed, 01 Jun 2005 05:07:51 GMT

Meaning of options available in ulimit:

OPTIONS
-a All current limits are reported.
-c The maximum size of core files created.
-d The maximum size of a process's data segment.
-f The maximum size of files created by the shell. (This is the default option)
-H Change and report the hard limit associated with a resource.
-l The maximum size that may be locked into memory.
-m The maximum resident set size.
-n The maximum number of open file descriptors.
-p The pipe buffer size.
-s The maximum stack size.
-S Change and report the soft limit associated with a resource.
-t The maximum amount of cpu time in seconds.
-u The maximum number of processes available to a single user.
-v The maximum amount of virtual memory available to the process.

Re: Large File systems in Bash Shell

Kiran_27 — Wed, 01 Jun 2005 06:41:05 GMT

The output of the ulimit shows the following
core file size (blocks) 2097151
data seg size (kbytes) 1048576
file size (blocks) unlimited
max memory size (kbytes) unlimited
open files 1200
pipe size (512 bytes) 16
stack size (kbytes) 8192
cpu time (seconds) unlimited
max user processes 1001
virtual memory (kbytes) unlimited

My concern is if we were able to do file manipulation in SH shell , then why we are not able to do it in BASH Shell. Does bash have any limit for the max data size

Re: Large File systems in Bash Shell

Suraj Singh_1 — Wed, 01 Jun 2005 23:30:18 GMT

Hi Kiran,

Did u check the ulimit -a output by changing your current shell to bash shell?

Within the script, try increasing ulimit...
#!/bin/bash
ulimit -Hf unlimited
ulimit -f unlimited
..
..

Regards

Re: Large File systems in Bash Shell

Kiran_27 — Thu, 02 Jun 2005 00:42:08 GMT

Below are the output with different shell
BASH
bash-2.05$ ulimit -a
core file size (blocks) 2097151
data seg size (kbytes) 1048576
file size (blocks) unlimited
max memory size (kbytes) unlimited
open files 1200
pipe size (512 bytes) 16
stack size (kbytes) 8192
cpu time (seconds) unlimited
max user processes 1001
virtual memory (kbytes) unlimited

SH Shell
ulimit -a
time(seconds) unlimited
file(blocks) unlimited
data(kbytes) 1048576
stack(kbytes) 8192
memory(kbytes) unlimited
coredump(blocks) 4194303
nofiles(descriptors) 1200

As you can see it is some setting issue with Bash shell , but could not figure out which one....
Hope you have some idea of what is happening ?
Regards
Kiran

Re: Large File systems in Bash Shell

Suraj Singh_1 — Thu, 02 Jun 2005 02:57:07 GMT

Hi Kiran,

I've been working on it for long, and at last i am able to get something. Here 'getrlimit' & 'setrlimit' might come into picture.

man getrlimit

Do you have gcc installed? If no, download it from http://hpux.connect.org.uk/hppd/hpux/Gnu/gcc-3.4.3/

A small c program will be able to get RLIMIT_FSIZE parameter...

#include
#include
#include
#include

int main(void)
{
struct rlimit rlimit_cur;

getrlimit(RLIMIT_FSIZE, &rlimit_cur);
printf("cur: %lu max: %lu\n", rlimit_cur.rlim_cur, rlimit_cur.rlim_max);
return(0);
}

In my system the o/p was:
cur: 2147483647 max: 2147483647

Now this is equal to 2 GB.

You might have to increase this limit! (I am not very sure)