topic Re: Very large numbers in a script in Operating System - HP-UX

Very large numbers in a script

Stephen Taylor_1 — Tue, 26 Aug 2003 19:18:24 GMT

Hi,

I have a script that needs to process a list of very large numbers with at least 8 places to the right of the decimal. The numbers are so large that the only tool that I have found that will add or subtract them is bc. I read each line from a file and then call bc to get the result. My problem is that bc is very slow and I need to calculate at least hundreds of these for on-line reports. It takes maybe 2 minutes to get the results. I tried awk but it doesn't work on these large numbers. Does anyone know of a fater tool?

TIA,
Steve

Re: Very large numbers in a script

harry d brown jr — Tue, 26 Aug 2003 19:19:01 GMT

perl

live free or die
harry

Re: Very large numbers in a script

H.Merijn Brand (procura — Tue, 26 Aug 2003 19:36:27 GMT

Harry! Please be a *little* bit more specific :)

a5:/u/usr/merijn 109 > perl -Mbignum=a,50 -le 'print sqrt(20)'
4.4721359549995793928183473374625524708812367192231
a5:/u/usr/merijn 110 > perl -Mbigint -le'$x = 24354**24; print $x->as_hex'
0x34e3b074445143af79a6d97d93787b0297b5697e1950d0b73e710c91a0f81c0eeb047b64edfcb5c981000000
a5:/u/usr/merijn 111 >

Enjoy, have FUN! H.Merijn

Re: Very large numbers in a script

curt larson_1 — Tue, 26 Aug 2003 19:40:27 GMT

have you tried the dtksh shell
/usr/dt/bin/dtksh

use typeset -F fixed precision
or typeset -E scientific

for floating point numbers

Re: Very large numbers in a script

Jeff Schussele — Tue, 26 Aug 2003 19:43:50 GMT

Ahh Merijn - Now you're just showing off AND having fun, I suspect. ;~))

Cheers,
Jeff

Re: Very large numbers in a script

A. Clay Stephenson — Tue, 26 Aug 2003 19:49:19 GMT

I'm assuming that the number are very large and you want perfect precision -- thus floating-point is out. You could use Perl's BigInt module but I'm going to suggest a surprising tool -- the one you are using now, bc. You just need to be a bit smarter in how you invoke it.

I'll guess that you are doing something like this:

#!/usr/bin/sh
INFILE=myfile
TOT=0
cat ${INFILE} | while read X
do
TOT=$(echo scale=8; ${TOT} + ${X} | bc)
done
echo "Total = ${TOT}"

The problem is that each invocation of bc costs you a fork and an exec BUT if we could do this as a co-process then that overhead is gone and I will predict at least a 10X (and probably much greater) improvement.

#!/usr/bin/sh
INFILE=myfile
TOT=0

bc |&
print -p "scale=8"
cat ${INFILE} | while read X
do
print -p "${TOT} + ${X}"
read -p TOT
done
print -p "quit"
echo "Total = ${TOT}"

You could improve this still further by using a register and simply adding to it:
e.g. print -p "a+=${X}"

I think you will be amazed at how well this will work.

Re: Very large numbers in a script

A. Clay Stephenson — Tue, 26 Aug 2003 20:13:42 GMT

Hi Steve:

Here's my still better version taking advantage of a bc register; in this case 'a'.

#!/usr/bin/sh
INFILE=myfile

bc |&
print -p "scale=8;a=0"
cat ${INFILE} | while read X
do
print -p "a+= ${X}"
done
print -p "a"
read -p TOT
print -p "quit"
echo "Total = ${TOT}"

That should be about as fast as it will get \; ig you are do multi-columned calculations then you would need a register for each column but that exercise is left for the student.

Regards (and it ain't always Perl), Clay

Re: Very large numbers in a script

Stephen Taylor_1 — Tue, 26 Aug 2003 20:31:51 GMT

Thanks everyone. I know I need to learn perl but Clay's solution was amazingly fast!!! It went from about 2 minutes to about 1 second! I didn't know that the shell could read from and print to another process like that. That was a neat trick!

Thanks,
Steve