topic Re: split command not splitting a file in Operating System - HP-UX

split command not splitting a file

Deepu Chakravarty — Wed, 04 Mar 2009 06:35:36 GMT

I have on the server OS HPUX 11.11. I need to split a file of the size >2GB. When I am issuing the command 'split -l 20000 voda', it is splitting the file into one file only (xaa). It is not splitting the file into many files of the line size 20000 each.
Any suggstion?

Re: split command not splitting a file

Matti_Kurkela — Wed, 04 Mar 2009 06:55:03 GMT

Please show us the output of commands:

file voda

wc -l voda

and:

wc -l xaa

MK

Re: split command not splitting a file

Deepu Chakravarty — Wed, 04 Mar 2009 07:41:14 GMT

$ file voda.txt
voda.txt: ascii text
$ wc -l voda.txt
196 voda.txt
(it seemms to be wrong, this file has many many lines. See the file size below)
-rw-rw-rw- 1 root sys 2517645178 Mar 3 16:52 voda.txt

# wc -l xaa
156 xaa
( I have terminated the file as it is continuing)

Re: split command not splitting a file

Matti_Kurkela — Wed, 04 Mar 2009 08:02:49 GMT

The system seems to think the file has only 196 *very very long* lines. The wc command calculates this by counting the line terminator characters, which is so simple operation it's unlikely to go wrong.

Please remember that tools like "vi" or "more" will wrap long lines into multiple shorter lines *on screen only*.

Alternatively, the file might have non-standard line terminator characters. If so, no Unix tool will recognize the lines correctly until the file is run through an appropriate converter.

I recommend you use "split -b" and specify the size in bytes, not in lines. This will avoid any problems with line termination characters.

MK

Re: split command not splitting a file

Deepu Chakravarty — Wed, 04 Mar 2009 08:54:37 GMT

If I use the following command as you suggested: split -b 50000000 voda.txt
then only one file with data is creating 'xaa', other files xab,etc though have created with size 50000000 bytes but those files are not showing ( with vi) any data inside of the file. These files are blank. If I onen any file with 'vi', it is showing "xat" Line too long, but no data.

Re: split command not splitting a file

Suraj K Sankari — Wed, 04 Mar 2009 09:05:54 GMT

HI,
Try this example

In this simple example, assume myfile is 3,000 lines long:
split myfile
This will output three 1000-line files: xaa, xab, and xac.

Working on the same file, this next example is more complex:
split -l 500 myfile segment
This will output six 500-line files: segmentaa, segmentab, segmentac, segmentad, segmentae, and segmentaf.

Finally, assume myfile is a 160KB file:
split -b 40k myfile segment
This will output four 40KB files: segmentaa, segmentab, segmentac, and segmentad.

of visit this site
http://www.unix.com/shell-programming-scripting/14576-split-file-using-awk.html

Suraj

Re: split command not splitting a file

Deepu Chakravarty — Wed, 04 Mar 2009 11:20:25 GMT

Sorry No suggestions are working. Only first file is showing data with ""xaa" Line too long and other splitted files have no data. Next files "xad" [Not edited] line 1 of 1 --100%--

Re: split command not splitting a file

OldSchool — Wed, 04 Mar 2009 17:31:02 GMT

the actual "split" command used, its results, and a long listing of the result files, as well as perhaps a "wc" on all of the resulting files.

using vi isn't reliable in the context as 1) it doesn't appear that it really is an ascii file ("file" guesses based on some magic values)
2) it seems to be composed of a few, very long lines, which vi is going to complain about.

you might try "cat"ing the results back together and then "diff" them w/ the original to if they are indeed correct.

Re: split command not splitting a file

Dennis Handly — Wed, 04 Mar 2009 18:28:58 GMT

split(1) is documented as supporting large files in 10.20:
http://docs.hp.com/en/5965-4406/ch04s09.html#interfaces-commands

>OldSchool: you might try "cat"ing the results back together

Yes, that will prove split works.

Re: split command not splitting a file

Dennis Handly — Thu, 05 Mar 2009 00:48:54 GMT

I had no problems with a smaller binary file:
split -b 1m /stand/vmunix vm