Operating System - HP-UX
1833906 Members
2028 Online
110063 Solutions
New Discussion

Very large numbers in a script

 
SOLVED
Go to solution

Very large numbers in a script

Hi,

I have a script that needs to process a list of very large numbers with at least 8 places to the right of the decimal. The numbers are so large that the only tool that I have found that will add or subtract them is bc. I read each line from a file and then call bc to get the result. My problem is that bc is very slow and I need to calculate at least hundreds of these for on-line reports. It takes maybe 2 minutes to get the results. I tried awk but it doesn't work on these large numbers. Does anyone know of a fater tool?

TIA,
Steve
7 REPLIES 7
harry d brown jr
Honored Contributor

Re: Very large numbers in a script

perl


live free or die
harry
Live Free or Die
H.Merijn Brand (procura
Honored Contributor

Re: Very large numbers in a script

Harry! Please be a *little* bit more specific :)

a5:/u/usr/merijn 109 > perl -Mbignum=a,50 -le 'print sqrt(20)'
4.4721359549995793928183473374625524708812367192231
a5:/u/usr/merijn 110 > perl -Mbigint -le'$x = 24354**24; print $x->as_hex'
0x34e3b074445143af79a6d97d93787b0297b5697e1950d0b73e710c91a0f81c0eeb047b64edfcb5c981000000
a5:/u/usr/merijn 111 >

Enjoy, have FUN! H.Merijn
Enjoy, Have FUN! H.Merijn
curt larson_1
Honored Contributor

Re: Very large numbers in a script

have you tried the dtksh shell
/usr/dt/bin/dtksh

use typeset -F fixed precision
or typeset -E scientific

for floating point numbers
Jeff Schussele
Honored Contributor

Re: Very large numbers in a script

Ahh Merijn - Now you're just showing off AND having fun, I suspect. ;~))

Cheers,
Jeff
PERSEVERANCE -- Remember, whatever does not kill you only makes you stronger!
A. Clay Stephenson
Acclaimed Contributor
Solution

Re: Very large numbers in a script

I'm assuming that the number are very large and you want perfect precision -- thus floating-point is out. You could use Perl's BigInt module but I'm going to suggest a surprising tool -- the one you are using now, bc. You just need to be a bit smarter in how you invoke it.

I'll guess that you are doing something like this:

#!/usr/bin/sh
INFILE=myfile
TOT=0
cat ${INFILE} | while read X
do
TOT=$(echo scale=8; ${TOT} + ${X} | bc)
done
echo "Total = ${TOT}"

The problem is that each invocation of bc costs you a fork and an exec BUT if we could do this as a co-process then that overhead is gone and I will predict at least a 10X (and probably much greater) improvement.

#!/usr/bin/sh
INFILE=myfile
TOT=0

bc |&
print -p "scale=8"
cat ${INFILE} | while read X
do
print -p "${TOT} + ${X}"
read -p TOT
done
print -p "quit"
echo "Total = ${TOT}"

You could improve this still further by using a register and simply adding to it:
e.g. print -p "a+=${X}"

I think you will be amazed at how well this will work.



If it ain't broke, I can fix that.
A. Clay Stephenson
Acclaimed Contributor

Re: Very large numbers in a script

Hi Steve:

Here's my still better version taking advantage of a bc register; in this case 'a'.

#!/usr/bin/sh
INFILE=myfile

bc |&
print -p "scale=8;a=0"
cat ${INFILE} | while read X
do
print -p "a+= ${X}"
done
print -p "a"
read -p TOT
print -p "quit"
echo "Total = ${TOT}"

That should be about as fast as it will get \; ig you are do multi-columned calculations then you would need a register for each column but that exercise is left for the student.

Regards (and it ain't always Perl), Clay
If it ain't broke, I can fix that.

Re: Very large numbers in a script

Thanks everyone. I know I need to learn perl but Clay's solution was amazingly fast!!! It went from about 2 minutes to about 1 second! I didn't know that the shell could read from and print to another process like that. That was a neat trick!

Thanks,
Steve