perl, sed, awk.. date format translation.

[Tim D Fulford]

Hi

This question just comes from soemthing I was working on...

I have date in a file in the following format

yyyymmddHHMM|.. other stuff here ...

I want to tranlate it into

yyyy/mm/dd|HH:MM|... other stuff here ...

I came up with a solution in perl, which after various people looked at became about 5 different (equally valid) solutions... so lets see how many we can generate.. I really want the shortest

Tim

[Muthukumar_5]

You can use date formation simply as,

date +'%Y/%m/%d|%H:%M|'

hth.

[Muthukumar_5 · Accepted solution]

With sed as,

# echo "yyyymmddHHMM|.. other stuff here ..." | sed 's/^\(....\)\(..\)\(..\)\(..\)\(.*\)/\1\/\2\/\3|\4:\5/g'
yyyy/mm/dd|HH:MM|.. other stuff here ...

hth.

[RAC_1]

echo "$(echo ${var}|cut -c1-4)/$(echo ${var}|cut -c5-6)/$(echo ${var}|cut -c7-8)|$(echo ${var}|cut -c9-10):$(echo ${var}|cut -c11-12)"

[Muthukumar_5]

With perl as,

# echo "yyyymmddHHMM|.. other stuff here ..." | perl -ne 's%(....)(..)(..)(..)(.*)%$1/$2/$3|$4:$5%g;print;'
yyyy/mm/dd|HH:MM|.. other stuff here ...

To update into the same file then,

perl -pi -e 's%(....)(..)(..)(..)(.*)%$1/$2/$3|$4:$5%g;print;'


hth.

[Steve Lewis]

$ X="200510141022|"
$ echo $X | awk 'function s(F,L){return substr($0,F,L)}{printf("%s/%s/%s|%s:%s|",s(1,4),s(5,2),s(7,2),s(9,2),s(11,2))}'

2005/10/14|10:22|

[Tim D Fulford]

RAC.. could not get your to work... what is ${var}... I'm guessing it is the line, if so how do I get it into the sed statement?

Tim

[Muthukumar_5]

Using awk :)

echo "yyyymmddHHMM|.. other stuff here ..." | awk -F"\n" '{ y=substr($1,0,4);m=substr($1,5,2);d=substr($1,7,2);h=substr($1,9,2);r=substr($1,11);print y"/"m"/"d"|"h":"r; }'

hth.

[Arunvijai_4]

var=200510162222
echo "$(echo ${var}|cut -c1-4)/$(echo ${var}|cut -c5-6)/$(echo ${var}|cut -c7-8)|$(echo ${var}|cut -c9-10):$(echo ${var}|cut -c11-12)"

2005/10/16|22:22
(RAC's script)

-Arun

[RAC_1]

for var in $(do
var=200510162222
echo "$(echo ${var}|cut -c1-4)/$(echo ${var}|cut -c5-6)/$(echo ${var}|cut -c7-8)|$(echo ${var}|cut -c9-10):$(echo ${var}|cut -c11-12)|$(echo ${var}|cut -c13-)"
done

[Muthukumar_5]

To make RAC solution to work then,

echo "yyyymmddHHMM|.. other stuff here ..." | while read line;
do
echo "$(echo $line| cut -c1-4)/$(echo $line| cut -c5-6)/$(echo $line| cut -c7-8)|$(echo $line| cut -c9-10):$(echo $line| cut -c11-)"
done

hth.

[RAC_1]

Correction.

for var in $(do
echo "$(echo ${var}|cut -c1-4)/$(echo ${var}|cut -c5-6)/$(echo ${var}|cut -c7-8)|$(echo ${var}|cut -c9-10):$(echo ${var}|cut -c11-12)|$(echo ${var}|cut -c13-)"
done

[Tim D Fulford]

OK I've got the idea with RAC's scripts....

I'm very impressed, but I am also looking dfor the shortest, neatest solution...

Keep them comming

Tim

[Muthukumar_5]

sorry RAC.

We can not make use of for when you are having input line as .

It will take one string and next like that. You must use with while loop only.

Difference:
# cat file
yyyymmddHHMM|.. other stuff here ...

# while read line;
> do
> ho $line| cut -c7-8)|$(echo $line| cut -c9-10):$(echo $line| cut -c11-)"
> done < file
yyyy/mm/dd|HH:MM|.. other stuff here ...
#
#
# for var in $(> odo
> ho ${var}|cut -c7-8)|$(echo ${var}|cut -c9-10):$(echo ${var}|cut -c11-12)|$(echo ${var}|cut -c13-)"
> done
yyyy/mm/dd|HH:MM||..
othe/r/|:|
stuf/f/|:|
here//|:|
...//|:|

Take care. I heard that legends too make mistakes some time (Like you :))

hth.

[Tim D Fulford]

I can forgive.... OK now we have a few variants... my solutions were

perl -i.org -pe 's/(^[0-9][0-9][0-9][0-9])([0-9][0-9])([0-9][0-9])([0-9][0-9])([
0-9][0-9])/$1\/$2\/$3\/\|$4\:$5/'

Which eventually got refinmed to

perl -i.org -pe 's/^(\d{4})(\d{2})(\d{2})(\d{2})(\d{2})/$1\/$2\/$3\/\|$4\:$5/'

can anyone get shorter??

Tim

[Tim D Fulford]

note

... I'm not too stingy on the old points... ... I'm not fussed if you made another subission...

I only want it to work, in principle (well I did fiddle with RAC's to get it to work, then everyone told me another way to refine it!!)

Tim

[Muthukumar_5]

Another way as,

while read pattern rest;
do
echo "$(echo $pattern | perl -ne '@lt=split//;@var=(@lt[0..3],"/",@lt[4..5],"/",@lt[6..7],"|",@lt[8..9],":",@lt[10..@lt]);print @var;') $rest"
done < file

hth.

[Muthukumar_5]

Tim,

You are using with {pattern count} and I used directly as \(....\) like that. However, both these are memory operations required to remember on $1.. $9 variables. May take more system resource for bigger files.

RAC's solution with cut is very fast. (Hoping..).

Thread is given scripting hour to produce solutions.

hth.

[Arunvijai_4]

Perl is always faster than sed,awk,cut for large amount of data..

-Arun

[Muthukumar_5]

Tim,

Which application is generating this information into a file? Is it your own application / some other thing.

If it is user manageable script then try to use date +'%Y/%m/%d|%H:%M|' instead of available one so that it will log details based on your requirement.

hth.

[Arunvijai_4]

May server log files generate yyyymmddHHMM|other stuffs here .... . So, it may not be a good idea to put in a user manageable script.. Just a thought to share

-Arun

[Muthukumar_5]

Arun,

cut is core functionality with os so that it must be faster than perl on operation surely.

I am tired...bcas of more replies to this thread itself. I am just retiring from this thread. :))

Take care.

[Arunvijai_4]

Muthu, i said "large amount of data".. Remember our perl classes last month (A performance comparison)

-Arun

[Tim D Fulford]

OK

1 - raw nerver has been touched!!! I wrote the original script to put teh data into yyyy/mm/dd|HH:MM|... other stuff ... formatt. I handed over all scripts and it was decided to remove the various delimiter cahracters as it "saved space" or something equally trivial.. So now I have to post process the file back to its original format.... grrrr

2 - I'm not worried about if perl is "supposed" to be better than sed... I can test that easily on a sample file. see attached..

3 - I'll do some tests and give my results (you can too!!)

Tim

[James R. Ferguson]

Hi Tim:

You can increase both brevity and readability by changing your forward slash delimiter to another character (e.g. "%"). This eliminates the escaping back slashes near forward slashes:

# perl -i.org -pe 's%^(\d{4})(\d{2})(\d{2})(\d{2})(\d{2})%$1/$2/$3|$4:$5%' file

I also dropped one "/" since your original substitution yielded yyyy/mm/dd/|HH:MM|...

Regards!

...JRF...