Operating System - Linux
1748073 Members
4506 Online
108758 Solutions
New Discussion

"shift count is too large" & little-endian issue in file asn1.cpp

 
SOLVED
Go to solution
AlexVi2
Occasional Advisor

"shift count is too large" & little-endian issue in file asn1.cpp

Hi,

I have some questions related to program from file asn1.cpp (Copyright
(c) 1999 Hewlett-Packard Company)


---------------------------
unsigned long mask;
    ...
mask = 0x1FF << ((8 * (sizeof(long) - 1)) - 1);
    /* mask is 0xFF800000 on a big-endian machine */
---------------------------

First of all, we have got the following warning on 64-bits model
warning: shift count is too large
        unsigned long mask = 0x1FF << ((8 * (sizeof(long) - 1)) - 1);
So, result of the shift may be undefined

Second. What about little-endian machine?
        Should mask be equal 0xFF800000?
        Should mask be equal 0?

Thanks,

Alex



#######################################
Fragment of file asn1.cpp
################ BEGIN ################

  Copyright (c) 1999
  Hewlett-Packard Company

[--snip--]

  A S N 1. C P P

  ASN encoder / decoder implementation

  VERSION 2.8

  DESIGN:
  Peter E. Mellquist

  AUTHOR:
  Peter E Mellquist


  LANGUAGE:
  ANSI C++

  OPERATING SYSTEM(S):
  MS-Windows Win32
  BSD UNIX

[--snip--]



/*
* asn_build_int - builds an ASN object containing an integer.
*  On entry, datalength is input as the number of valid bytes
following
*   "data".  On exit, it is returned as the number of valid bytes
*   following the end of this object.
*
*  Returns a pointer to the first byte past the end
*   of this object (i.e. the start of the next object).
*  Returns NULL on any error.
*/
unsigned char * asn_build_int( unsigned char *data,
                               int *datalength,
                               unsigned char type,
                               long *intp,
                               int intsize)
{
/*
* ASN.1 integer ::= 0x02 asnlength byte {byte}*
*/

    long integer;
    unsigned long mask;

    if (intsize != sizeof (long))
return NULL;
    integer = *intp;
    /*
     * Truncate "unnecessary" bytes off of the most significant end of
this
     * 2's complement integer.  There should be no sequence of 9
     * consecutive 1's or 0's at the most significant end of the
     * integer.
     */
    mask = 0x1FF << ((8 * (sizeof(long) - 1)) - 1);
    /* mask is 0xFF800000 on a big-endian machine */
    while((((integer & mask) == 0) || ((integer & mask) == mask))
  && intsize > 1){
intsize--;
integer <<= 8;
    }
    data = asn_build_header(data, datalength, type, intsize);
    if (data == NULL)
return NULL;
    if (*datalength < intsize)
return NULL;
    *datalength -= intsize;
    mask = 0xFF << (8 * (sizeof(long) - 1));
    /* mask is 0xFF000000 on a big-endian machine */
    while(intsize--){
*data++ = (unsigned char)((integer & mask) >> (8 * (sizeof(long) -
1)));
integer <<= 8;
    }
    return data;
};

################ END ################

 

 

 

P.S.This thread has been moved from  HP-UX>Languages and Scripting to Linux > sysadmin- HP Forums Moderator

9 REPLIES 9
Steven Schweda
Honored Contributor

Re: "shift count is too large" & little-endian issue in file asn1.cpp

 
AlexVi2
Occasional Advisor

Re: "shift count is too large" & little-endian issue in file asn1.cpp

Hi,

> uname -srvmpio

Linux 2.6.32-279.5.2.el6.x86_64 #1 SMP Tue Aug 14 11:36:39 EDT 2012 x86_64 x86_64 x86_64 GNU/Linux

 

> icpc -v

icpc version 12.1.3 (gcc version 4.4.6 compatibility)

 

> icpc -V

Intel(R) C++ Intel(R) 64 Compiler XE for applications running on Intel(R) 64, Version 12.1 Build 20120212 Copyright (C) 1985-2012 Intel Corporation.  All rights reserved.

 

0x1FF is a 32-bit integer on any compiler

 

Do you know where can newer version of asn1.cpp be downloaded?

 

Thanks

 

 

Steven Schweda
Honored Contributor

Re: "shift count is too large" & little-endian issue in file asn1.cpp

 
Dennis Handly
Acclaimed Contributor
Solution

Re: "shift count is too large" & little-endian issue in file asn1.cpp

>mask = 0x1FF << ((8 * (sizeof(long) - 1)) - 1);

 

As Steven mentioned, you'll need to adjust the type of the constant to UL:

mask = 0x1FFUL << ((8 * (sizeof(unsigned long) - 1)) - 1);

 

>What about little-endian machine?  Should mask be equal 0xFF800000?

 

If you are on a wrong endian machine, you need to look at what is being done.

You may want to byte swap the data so that it looks at the high order 9 bits.  And if the data is already in native format, there is nothing to do.

But I think the comment is bogus, the mask is the same on any machine, as Steven said.

 

>while ((((integer & mask)

 

This code is broken.  The mask and integer variables should be the same type.  That way it works in 64 bit mode.

 

>0x1FF is a 32-bit integer on any compiler

 

No, the Standard defines 0x1FF as an int, if it fits.  There are some "broken" compilers where it is smaller and others where it is larger.

 

>My Google search for "asn1.cpp" easily found one with:

 

Did it have those bogus comments about "big-endian" removed?  Is it 32 and 64 bit safe?

AlexVi2
Occasional Advisor

Re: "shift count is too large" & little-endian issue in file asn1.cpp

 

>  One might ask why you would post a GNU/Linux question (with no
identification) in an HP-UX forum, when there is a similar GNU/Linux
form not far away.

 

Because this program is Copyright (c) Hewlett-Packard Company

AlexVi2
Occasional Advisor

Re: "shift count is too large" & little-endian issue in file asn1.cpp

> Did it have those bogus comments about "big-endian" removed?

 

All newer asn1.cpp versions I found have the same "big-endian" related comment.

AlexVi2
Occasional Advisor

Re: "shift count is too large" & little-endian issue in file asn1.cpp

> But I think the comment is bogus, the mask is the same on any machine, as Steven said.

 

> >while ((((integer & mask)

 

Thanks.

 

It should be: while ((((static_cast<unsigned long>(integer) & mask)

Dennis Handly
Acclaimed Contributor

Re: "shift count is too large" & little-endian issue in file asn1.cpp

>It should be: while ((((static_cast<unsigned long>(integer) & mask)

 

That's overkill.  It should have the types changed:   unsigned int mask;

 

But this probably would defeat this check:  :-(

if (intsize != sizeof (long)) return NULL;

 

>Because this program is Copyright (c) Hewlett-Packard Company

 

But it doesn't list HP-UX as one of the OSs.

Is this part of the openssl package?

 

AlexVi2
Occasional Advisor

Re: "shift count is too large" & little-endian issue in file asn1.cpp

 

-----------------------------

long integer;

unsigned long mask;

 

while((((integer & mask) == 0) || ((integer & mask) == mask)) && intsize > 1) { ... }

------------------------------

 

So, it seems

while((((static_cast<unsigned long>(integer) & mask) == 0) || ((static_cast<unsigned long>(integer) & mask) == mask)) && intsize > 1)

should be OK?

 

> It should have the types changed:   unsigned int mask;

Why?

 

> Is this part of the openssl package?

I don't know. Possibly, we will migrate to newer version of asn1.cpp (in the future)

P.S. Currently we are not using as1.cpp

 

Dennis, thank you.