HP floating point format (HP 1000)

[Tim Kendon]

Dear forum,

I need to read some old binary files with (32 bit) floats written in HP floating point format. I need to write my own routine for conversion to IEEE format (so please don't advocate using HP provided 'black-box' routines e.g. FCHI (fortran conversion routine)

I found a "thread" which provided the general structure of the HP floating point format

http://www.hp.com/products1/rte/tech_support/files/guru98column.txt

[Admin : the above link is no longer valid]

This is great, but

1. what is the normalization bit?
2. how do I treat the exponent (for IEEE format it is dependent on the coming fraction!)
3. what about the 'Inf' and 'Nan' scenarios

Appreciate any help.

[Raj D.]

Tim,

what about : HP-UX floating-point guide :


https://support.hpe.com/hpesc/public/docDisplay?docLocale=en_US&docId=c02676252

Hope this helps,
Raj.

[Tim Kendon]

Raj,

thanks for the quick response!

To be honest, I haven't read the web-link in detail, although I had scanned it before.

It seems to me that the web-link describes the IEEE format (which is the format that HP-UX systems use today, I think. At least our HP-UX system at work use IEEE format.)

Sometime between 1995-2010 HP-UX seems to have made a switch between an old "HP floating format" and the current IEEE format.

As I said, I haven't read the web-link in detail, so I ought to do that.

[Steven Schweda · Accepted solution]

> 1. what is the normalization bit?

Ask Google? I was led to:

http://docs.hp.com/en/B3906-90006/go01.html

(The rest of the manual looks as if it might
be interesting, too, but IEEE-centric.)

> 2. how do I treat the exponent (for IEEE
> format it is dependent on the coming
> fraction!)

Huh? For a given value and mantissa, I'd say
that the exponent is pretty well determined.

> 3. what about the 'Inf' and 'Nan' scenarios

For that, you may need actual info on the old
HP implementation.

Tiny clues (search for '"floating point" hp
1000'):

http://bitsavers.org/pdf/hp/1000/5953-8761_HP_1000_A-Series_Hardware_Technical_Data_Dec84.pdf

A diligent search might find more.

Get an emulator and poke around?

http://simh.trailing-edge.com/hp2100.html

[Dennis Handly]

>1. what is the normalization bit?

It seems it is a normal bit.

>3. what about the Inf and NaN scenarios

It seems they weren't invented yet.

>It seems to me that the web-link describes the IEEE format

Yes and more specifically only Integrity. Steven has the URL for the PA version.

>Sometime between 1995-2010 HP-UX seems to have made a switch between an old "HP floating format" and the current IEEE format.

Perhaps you mean 1985, when PA-RISC came out.

>Raj: what about: HP-UX floating-point guide

Not hardly. That's only for Integrity. And a shorter URL is:
http://www.hp.com/go/fp

>Steven: > 1. what is the normalization bit?
http://docs.hp.com/en/B3906-90006/go01.html

This seems bogus. They seem to be the exact opposites. I.e. there seems to be an explicit bit for the HP 1000 but IEEE has a phantom bit, for all but denormalized numbers.

[Tim Kendon]

Dennis, Steven thanks for the input.

Steven, on the small matter of the exponent, for IEEE format

0 00000001 00000000000000000000000 = +1 * 2**(1-127) * 1.0 = 2**(-126)
0 00000000 10000000000000000000000 = +1 * 2**(-126) * 0.1 = 2**(-127)

where the 0.1 and 1.0 are binary numbers. I interpret the above as showing that the IEEE exponent is dependent on the coming fraction. I found the above example at

http://www.psc.edu/general/software/packages/ieee/ieee.php

[Admin:The above link is no longer valid]

I was concerned that a similar exception might arise for the old HP format!

Anyway thanks again for the constructive inputs.

[Dennis Handly]

>I interpret the above as showing that the IEEE exponent is dependent on the coming fraction.

The latter is a denorm. I.e. the mantissa is interpreted differently depending on the exponent field, not the other way around.

[Stan Sieler]

With the manuals cited, you should be
able to write a small amount of code to
convert HP format to IEEE. We have such
code in one of our products, taking about
200 lines of C (including comments).

Example header:

real32 mpe_real32_to_ieee32 (int32 *status, uint32 input);

// possible status values:
// C2I_OK (which is 0)
// C2I_EXPONENT_OVERFLOW
// C2I_EXPONENT_UNDERFLOW
// C2I_DENORMALIZED_CONVERSION

I.e., there are some special cases the
caller needs to handle.

If you're interested in discussing this
offline, email me at sieler@allegro.com

(Our product source code refers to the HP
format reals as "MPE reals", because the
focus of the product was on the HP 3000
... but they're the same reals as used
on the HP 1000.)

Stan

[Stan Sieler]

Oh, why "uint32" instead of some type
like "mpe_real_32_type" for the input
parameter (the MPE 32-bit real-by-value)?

For various reasons, our code is picking
up 32-bit (or 64-bit) packets), determining
their type (IEEE real, MPE real, ascii text,
32-bit int, other), and then calling
an appropriate conversion routine. I.e.,
the data isn't really a 32-bit integer,
but is just a packet of 32 bits.

[Dennis Handly]

>Stan: real32 mpe_real32_to_ieee32(int32 *status, uint32 input);

Tim wants HP 1000 floating point not HP 3000.

>I.e., there are some special cases the caller needs to handle.

Yes, since they don't fit.

>but they're the same reals as used on the HP 1000.)

That's not what my ACD says about HP 3000:
S | 9 bit exponent bias +256 | binary point | 22 bit fraction|

This doesn't match Tim's first URL. I would assume left hand, right hand issues would make them incompatible.

[Tim Kendon]

Dennis/everyone, again thanks for the inputs.

Using Steven's link

http://bitsavers.org/pdf/hp/1000/

I was able to find the "HP 1000 Computer Programming Reference Manual". This manual confirms the (32 bit) floating point data format that was given in my earlier link

http://www.hp.com/products1/rte/tech_support/files/guru98column.txt
[Admin : the above link is no longer valid]
Whilst the reference manual doesn't explicitly refer to a normalization bit it does state that the "Floating point numbers are considered normalized when the mantissa sign bit and adjacent bit are opposite in polarity". So the (32 bit) data structure given in these two links seems consistent.

However the manual does not explicitly state (for dummies like me) how to "treat" the bits. I had assumed something like,

decimal number
= 2**(exp*esign) * [b + (1-n)]*sign

where
exp = the exponent bits converted to decimal
esign = the sign of the exponent
n = normalisation bit (0 if normalized?)
b = binary number relating to fraction/mantissa
sign = sign of fraction

But so far this has not produced the desired conversion. To assist me, I fortunately (or unfortunately ... depending on whether it is a red-herring) have a 32 bit HP float binary on file for which I believe the decimal answer = 0.05

I have converted the 4 byte binary data to bits (using python and bitarray). I have used this approach (python/bitarray) before to check if I can read/decipher a standard IEEE 4 byte real, and succeeded, so I think python/bitarray is doing an adequate conversion from bytes to bits. The resulting bit pattern for the HP 1000 float which I believe should be a decimal of 0.05 is

bits = 01101000 11010110 01001110 11110101
bits = 00010110 01101011 01110010 10101111

One is big endian (the other little!). HP-UX is big endian but I am reading on windows so I cannot be entirely confident which of the above to use. So I am in the process of trial and error, hoping that the above bit pattern is indeed equal to a decimal of 0.05.

Of course the above bit pattern may be a red-herring (for which I apologies whole-heartedly in advance), but if someone fancies a challenge over their lunch break and manages to crack the pattern ...

[Dennis Handly]

>I fortunately have a 32 bit HP float binary on file for which I believe the decimal answer = 0.05

Please use "HP 1000 float" and not HP. There are probably N chauvinistic HP computers that just used just "HP" so don't fall into that trap.

>The resulting bit pattern for the HP 1000 float which I believe should be a decimal of 0.05 is
bits = 01101000 11010110 01001110 11110101
bits = 00010110 01101011 01110010 10101111
>One is big endian (the other little!).

No, that's not how scummy little endian works. Only the bytes are swapped, not the bits.

>hoping that the above bit pattern is indeed equal to a decimal of 0.05.

The pattern for .05 for IEEE is: 3d4ccccd
So you need to find a pattern of 11001100...

[Dennis Handly]

Taking the IEEE formats for .05 I get these bits:
float:
3d4ccccd 0.050000
S:0 E:-5 M:0x4ccccd
S:0 M:1.10011001100110011001101 E:0000101 1
double:
3fa999999999999a 0.050000
S:0 E:-5 M:0x999999999999a
S:0 M:1.1001100110011001100110011001100110011001100110011010 E:0000000101 1

Note: The last bit of the M field needs to be removed as not fitting in the HP1000 format.
So .05 appears to be:
0 1.1001100110011001100110 0000101 1

The M field may be twos complement to match the comment about "opposite in polarity".

[Tim Kendon]

Dennis thanks again!

I agree, I think I should be certainly looking for this 1100... pattern.

It seems I first need to check the bit pattern that I am producing with this bitarray tool.

Then I need to try and confirm 0.05 is indeed the decimal number represented by the 4 bytes. There is a tool on our HP-UX system called FCHI which actually does the conversion from HP 1000 float to IEEE for me. I believe FCHI is a black box routine provided by HP, but it should at least provide confirmation of the input data and final result.

If (and it's a big IF) I manage to write the conversion routine from HP 1000 float to IEEE, I will place the conversion routine (albeit in python) on the email thread in recognition of everyones' input and to save future bother for other unfortunates like me! But as I am writing these routines "after hours" it might still take a few weeks to resolve.