System Administration
cancel
Showing results for 
Search instead for 
Did you mean: 

split command not splitting a file

Deepu Chakravarty
Regular Advisor

split command not splitting a file

I have on the server OS HPUX 11.11. I need to split a file of the size >2GB. When I am issuing the command 'split -l 20000 voda', it is splitting the file into one file only (xaa). It is not splitting the file into many files of the line size 20000 each.
Any suggstion?
9 REPLIES
Matti_Kurkela
Honored Contributor

Re: split command not splitting a file

Please show us the output of commands:

file voda

wc -l voda

and:

wc -l xaa

MK
MK
Deepu Chakravarty
Regular Advisor

Re: split command not splitting a file

$ file voda.txt
voda.txt: ascii text
$ wc -l voda.txt
196 voda.txt
(it seemms to be wrong, this file has many many lines. See the file size below)
-rw-rw-rw- 1 root sys 2517645178 Mar 3 16:52 voda.txt

# wc -l xaa
156 xaa
( I have terminated the file as it is continuing)
Matti_Kurkela
Honored Contributor

Re: split command not splitting a file

The system seems to think the file has only 196 *very very long* lines. The wc command calculates this by counting the line terminator characters, which is so simple operation it's unlikely to go wrong.

Please remember that tools like "vi" or "more" will wrap long lines into multiple shorter lines *on screen only*.

Alternatively, the file might have non-standard line terminator characters. If so, no Unix tool will recognize the lines correctly until the file is run through an appropriate converter.

I recommend you use "split -b" and specify the size in bytes, not in lines. This will avoid any problems with line termination characters.

MK
MK
Deepu Chakravarty
Regular Advisor

Re: split command not splitting a file

If I use the following command as you suggested: split -b 50000000 voda.txt
then only one file with data is creating 'xaa', other files xab,etc though have created with size 50000000 bytes but those files are not showing ( with vi) any data inside of the file. These files are blank. If I onen any file with 'vi', it is showing "xat" Line too long, but no data.
Suraj K Sankari
Honored Contributor

Re: split command not splitting a file

HI,
Try this example

In this simple example, assume myfile is 3,000 lines long:
split myfile
This will output three 1000-line files: xaa, xab, and xac.

Working on the same file, this next example is more complex:
split -l 500 myfile segment
This will output six 500-line files: segmentaa, segmentab, segmentac, segmentad, segmentae, and segmentaf.

Finally, assume myfile is a 160KB file:
split -b 40k myfile segment
This will output four 40KB files: segmentaa, segmentab, segmentac, and segmentad.

of visit this site
http://www.unix.com/shell-programming-scripting/14576-split-file-using-awk.html

Suraj
Deepu Chakravarty
Regular Advisor

Re: split command not splitting a file

Sorry No suggestions are working. Only first file is showing data with ""xaa" Line too long and other splitted files have no data. Next files "xad" [Not edited] line 1 of 1 --100%--
OldSchool
Honored Contributor

Re: split command not splitting a file

the actual "split" command used, its results, and a long listing of the result files, as well as perhaps a "wc" on all of the resulting files.

using vi isn't reliable in the context as 1) it doesn't appear that it really is an ascii file ("file" guesses based on some magic values)
2) it seems to be composed of a few, very long lines, which vi is going to complain about.

you might try "cat"ing the results back together and then "diff" them w/ the original to if they are indeed correct.
Dennis Handly
Acclaimed Contributor

Re: split command not splitting a file

split(1) is documented as supporting large files in 10.20:
http://docs.hp.com/en/5965-4406/ch04s09.html#interfaces-commands

>OldSchool: you might try "cat"ing the results back together

Yes, that will prove split works.
Dennis Handly
Acclaimed Contributor

Re: split command not splitting a file

I had no problems with a smaller binary file:
split -b 1m /stand/vmunix vm