Maddened by OpenSSL and Crypt::SSLeay make

[Ralph Grothe]

I'm getting furious,
wasting again hours that accumulate to days when simply trying to make crypto libs for HP-UX 11.00 (32 Bit) that build ridiculously easy under Linux.

I need to install the CPAN module Crypt::SSLeay which is required for my Perl UserAgents supporting HTTPS aka SSL.

I tried using various precompiled libs as well as trying to build them from source,
always get errors from compiler/linker (i.e. GNU gcc, sorry no HP-UX ANSI C available, but even then I would doubt that it would build).

As said, compiling the stuff under Linux, FreeBSD is no matter.

Has anyone achieved somehow to build these for HP-UX?

[Steven Gillard_2]

Can you post the errors you're getting from the linker?

Regards,
Steve

[Ralph Grothe]

Hi Steven,

I getting exactly the same as complained about in this thread:

http://forums.itrc.hp.com/cm/QuestionAnswer/1,,0xccb90bce6f33d6118fff0090279cd0f9,00.html

I also found this URL which addresses this issue:

http://www.mail-archive.com/openssl-users@openssl.org/msg23602.html

I did as outlined therein bout still get error messages.

The Makefile of the openssl tarball really sucks.

It always runs a complete recompilation of every source file even though I reached the final linker call before and all the binaries exist already.
Never have seen make behave like this (I haven't changed my system's time between make runs!)

I'm using gcc and +cpd isn't a known flag to gcc

making all in tools...
+ rm -f libcrypto.sl.0
+ rm -f libcrypto.sl
+ rm -f libcrypto.sl.0.9.6
libs=''; for i in crypto; do ( set -x; /usr/ccs/bin/ld +vnocompatwarnings -L/opt/build/openssll-0.9.6c +cpd /opt/build/openssll-0.9.6c:/opt/openssl/lib -b -z -o lib$i.sl.0.9.6 +h lib$i.sl.0.9.6 -Fl lib$i.a $libs -ldld -lc ) || exit 1; libs="$libs -L. -l$i"; done
+ /usr/ccs/bin/ld +vnocompatwarnings -L/opt/build/openssll-0.9.6c +cpd /opt/buil
d/openssll-0.9.6c:/opt/openssl/lib -b -z -o libcrypto.sl.0.9.6 +h libcrypto.sl.0
.9.6 -Fl libcrypto.a -ldld -lc
/usr/ccs/bin/ld: Unrecognized argument: +cpd
/usr/ccs/bin/ld: Usage: /usr/ccs/bin/ld [options] [flags] files
*** Error exit code 1

Stop.
*** Error exit code 1

Stop.
*** Error exit code 1

[Steven Gillard_2]

Are you using the HP supplied make or GNU make? I've found gmake much more reliable.

Your other problem is simply a typo - the option is +cdp, not +cpd. It's actually an argument to the linker, man ld for details.

Regards,
Steve

[Ralph Grothe]

Steven,

thank you for the hint regarding my typo.

I also downloaded and installed GNU make, and tried a new gmake run, this time with the +cdp flag.

Despite, I still get errors, but this time with regard to the alegedly missing libcrypto (which has been compiled and is present):

+ rm -f libssl.sl.0.9.6
libs='-lcrypto'; for i in ssl; do ( set -x; /usr/ccs/bin/ld +vnocompatwarnings -L/opt/build/openssll-0.9.6c +cdp /opt/build/openssll-0.9.6c:/opt/openssl/lib -b -z -o lib$i.sl.0.9.6 +h lib$i.sl.0.9.6 -Fl lib$i.a $libs -ldld -lc ) || exit 1; libs="$libs -L. -l$i"; done
+ /usr/ccs/bin/ld +vnocompatwarnings -L/opt/build/openssll-0.9.6c +cdp /opt/buil
d/openssll-0.9.6c:/opt/openssl/lib -b -z -o libssl.sl.0.9.6 +h libssl.sl.0.9.6 -
Fl libssl.a -lcrypto -ldld -lc
/usr/ccs/bin/ld: Can't find library: "crypto"
*** Error exit code 1

Stop.
*** Error exit code 1

Stop.
gmake: *** [sub_all] Error 1


Would you have another hint what flags to supply to the linker?

[H.Merijn Brand (procura]

First of all, lets congratulate you with your warm head :) Fits you nice, but be aware, I'm chasing you!

use Perl or die;

If you know where libcrypto.a or libcrypto.sl resides, you can pass the location to ld with -L, but for the .sl it might be worth testing if setting $LD_LIBRARY_PATH and/or $SHLIB_PATH

I don't have openssl installed, so I cannot help you with some succeeded builds

[Steven Gillard_2]

It's now complaining that it can't find libcrypto.a - someone posted the same problem yesterday but it looks like you're already using that solution:

http://forums.itrc.hp.com/cm/QuestionAnswer/1,,0xccb90bce6f33d6118fff0090279cd0f9,00.html

Check for the real location of the libcrypt.a library, then add that path as a -L argument. You might also want to correct the path in the +cdp argument to the same.

Regards,
Steve

[Ralph Grothe]

procura, Steven,

thanks for the continuing help. :-)

Meanwhile I wanted to do a gmake distclean, but the Makefile seems only to have a target for clean:

I think the whole Makefile produced by config is rotten.

Even now where I use gmake instead of make the whole thing isn't mtime-stamp-aware.
After having run a gmake and (just for a lark) running a gmake install it completely starts anew despite the existence of lots of object files and libraries from the previous gmake run.
This is very time consuming and frustrating.
A working make should skip everything which is up to date, but not here :-(

The OpenSSL developers should revise the make targets at least for HP-UX.
The whole nuisance seems to be a well known issue for frustrated HP-UX users
because google spits out many threads from mailing lists and forums that deal about it.

At the moment I am giving this recipe a try:

http://www.mail-archive.com/openssl-users@openssl.org/msg23556.html

Regards
Ralph

[Ralph Grothe]

Nope, same story:


+ rm -f libssl.sl.0.9.6
libs='-lcrypto'; for i in ssl; do ( set -x; /usr/ccs/bin/ld +vnocompatwarnings -b -z -o lib$i.sl.0.9.6 +h lib$i.sl.0.9.6 -Fl lib$i.a $libs -ldld -lc ) || exit 1; libs="$libs -L. -l$i"; done
+ /usr/ccs/bin/ld +vnocompatwarnings -b -z -o libssl.sl.0.9.6 +h libssl.sl.0.9.6
-Fl libssl.a -lcrypto -ldld -lc
/usr/ccs/bin/ld: Can't find library: "crypto"
*** Error exit code 1

Stop.

*** Error exit code 1

Stop.
gmake: *** [sub_all] Error 1


libcrypto is present,
here from my build directory

# pwd
/opt/build/openssl-0.9.6c
# find . -name libcrypto\* -exec ll {} \;
-rw-r--r-- 1 root sys 1740224 Mar 12 15:06 ./libcrypto.a
-rwxr-xr-x 1 root sys 1597440 Mar 12 15:15 ./libcrypto.sl.0.9.6
lrwxr-xr-x 1 root sys 18 Mar 12 15:15 ./libcrypto.sl.0 -> li
bcrypto.sl.0.9.6
lrwxr-xr-x 1 root sys 14 Mar 12 15:15 ./libcrypto.sl -> libc
rypto.sl.0

[H.Merijn Brand (procura]

+ rm -f libssl.sl.0.9.6
libs='-lcrypto'; for i in ssl; do ( set -x; /usr/ccs/bin/ld +vnocompatwarnings -b -z -o lib$i.sl.0.9.6 +h lib$i.sl.0.9.6 -Fl lib$i.a $libs -ldld -lc ) || exit 1; libs="$libs -L. -l$i"; done
+ /usr/ccs/bin/ld +vnocompatwarnings -b -z -o libssl.sl.0.9.6 +h libssl.sl.0.9.6
-Fl libssl.a -lcrypto -ldld -lc
/usr/ccs/bin/ld: Can't find library: "crypto"
*** Error exit code 1

I don't see the '-L.' return in that 'ld' command.

Try typing it by hand (or whatever cut'n'paste method is available) and add a -L. in

[Shannon Petry]

I have all of this workin in both 10.20 and 11.x. I found the steps to be identical to what I had to do in Linux. Here's the steps I took.

1. DO NOT USE HP-ANSI/C!
* Instead, use gcc2.95 which is available from the porting center!
2. DO NOT USE STANDARD PERL!
* Instead use perl5.6.1. I downloaded from scratch at perl.org and used gcc to compile my own handy copy with all the goodies I like :)
3. As a rule, I never download and build my own perl mods, instead use CPAN to do this for you. It handles dependancies and crud I plain old dont have time for in Solaris, HP-UX, AIX, Irix, Linux, Sco....you get the idea :)
If you do not have the ability to run cpan, then.....welll....I make no assumptions.
4. I downloaded zlib, openssl, and openssh. In the same order, (zlib, openssl, openssh) run:
> ./configure --prefix /opt
> make
> make install

On openssl, I did have to specify my zlib directory because I did not use the default "/usr/local/lib", and ditto with openssh and openssl. If you are stuck, there's plenty of options given by running
./configure --help (works on all gnu configure commands).

There are linker configs, and you should only be linking openssl to /usr/lib (ld.so should find everything it needs there). Since this is the default, you should have to touch nothing. If you have a big mailbox, I can email you my compiled stuff, but it's about 60MB total including openssh.

While I was blabbing, I tested a box with nothing on it, and it compiled fine with all the defaults (except for specifying previous compile dirs...)

I used to have wierd problems like yours when I would download other peoples pre-compiled stuff. Nowdays, I only download pre-compiled gcc and binutils!

[Ralph Grothe]

Whoa, at least build, tests, and install worked with the interspersed -L. (:-)

installing ssl...
installing rsaref...
installing apps...
installing openssl
installing CA.sh
installing CA.pl
installing der_chop
installing test...
installing tools...
installing libcrypto.a
installing libssl.a
installing libcrypto.sl.0.9.6
installing libssl.sl.0.9.6
+ ln -f -s libcrypto.sl.0.9.6 libcrypto.sl.0
+ ln -f -s libcrypto.sl.0 libcrypto.sl
+ ln -f -s libssl.sl.0.9.6 libssl.sl.0
+ ln -f -s libssl.sl.0 libssl.sl


As I knew I was using the do_hpux_shared: target I sneaked in (is this comprehensible in (D)English?) the missing '-L.' in the ld invocation line within the for loop.

# diff Makefile Makefile.dist
368c368
< ( set -x; /usr/ccs/bin/ld -L. +vnocompatwarnings ---
> ( set -x; /usr/ccs/bin/ld +vnocompatwarnings

To make me an ultimative happy man I only need Crypt::SSLeay getting built as well:


# gmake test
PERL_DL_NONLAZY=1 /usr/bin/perl -Iblib/arch -Iblib/lib -I/opt/perl5/lib/5.6.1/PA
-RISC2.0 -I/opt/perl5/lib/5.6.1 -e 'use Test::Harness qw(&runtests $verbose); $v
erbose=0; runtests @ARGV;' t/*.t
t/ssl_context......./usr/lib/dld.sl: Can't open shared library: ./libcrypto.sl.0
.9.6
/usr/lib/dld.sl: No such file or directory
Can't load 'blib/arch/auto/Crypt/SSLeay/SSLeay.sl' for module Crypt::SSLeay: No
such file or directory at /opt/perl5/lib/5.6.1/PA-RISC2.0/DynaLoader.pm line 206
.
at blib/lib/Crypt/SSLeay/CTX.pm line 2
Compilation failed in require at blib/lib/Crypt/SSLeay/CTX.pm line 2.
Compilation failed in require at blib/lib/Crypt/SSLeay/MainContext.pm line 8.
Compilation failed in require at t/ssl_context.t line 3.
BEGIN failed--compilation aborted at t/ssl_context.t line 3.
t/ssl_context.......dubious
Test returned status 255 (wstat 65280, 0xff00)
FAILED--1 test script could be run, alas--no output ever seen
gmake: *** [test_dynamic] Error 2



When being asked by the MakeMaker module where my openssl library resides I fed /opt/openssl which is a symlink to the directory where the above "gmake install" installed my openssl stuff to

# ll /opt/openssl
lrwxr-xr-x 1 root sys 25 Mar 12 14:08 /opt/openssl -> gnu/op
enssl-0.9.6c-source


procura, many thanks, but for now I will still refrain from giving you 10 pts to prevent from the popping up of the magic rabbit icon.

[H.Merijn Brand (procura]

Finally an easy one :)

# gmake test
PERL_DL_NONLAZY=1 /usr/bin/perl -Iblib/arch -Iblib/lib -I/opt/perl5/lib/5.6.1/PA
-RISC2.0 -I/opt/perl5/lib/5.6.1 -e 'use Test::Harness qw(&runtests $verbose); $v
erbose=0; runtests @ARGV;' t/*.t
t/ssl_context......./usr/lib/dld.sl: Can't open shared library: ./libcrypto.sl.0
.9.6
/usr/lib/dld.sl: No such file or directory

ln -s libcrypto.sl libcrypto.sl.0

or whatever direction is needed
(you should count the symlinks on my system :)

[Ralph Grothe]

Hi Shannon,

I've been following the same philosophy about only downloading precompiled
binaries for GNU's binutils and gcc.

And, of course I *did* have my Perl built from the sources (i.e. stable.tar.gz), because my only one trial with the precompiled Perl binary made every CPAN module installation which wasn't pure Perl but needed to be linked against libraries, or included some C code, hopelessly impossible.

After having found a way how to get through our HTTP/FTP proxy (needed some tweaking of the hash %NetConfig from Net::Config, I do have almost all my modules installed with the CPAN module, and I am a happy camper.

But even the CPAN module couldn't cope with Crypt::SSLeay and insisted on manual attention.

This has been basically what I've been trying for the last two days, having this damned module installed, so that my LWP user agents support HTTPS like they do HTTP.

Have you managed to get the HTTPS aka SSL/TLS support for the LWP::UserAgent module?

If so your recipe would be most welcome.

[H.Merijn Brand (procura]

Shannon, for once I fully disgree here

1. **ALLWAYS** use HP C-ANSI-C
2. Use your own perl build
3. Build all your modules yourself

1 for better performance
2 so you know what type of perl you've got (and of course it's now built with HPc, so the modules will build out of the box)
3 So you learn from the errors and you can feedbach specific hpux problems to the author of the module, so it /will/ build out of the box next time

[Ralph Grothe]

procura,

the "gmake install" of openssl produced some ugly "cross-referential" symlinks:

# ll /opt/openssl/lib/
total 7888
-rw-r--r-- 1 root sys 1740224 Mar 12 16:20 libcrypto.a
lrwxr-xr-x 1 root sys 14 Mar 12 16:20 libcrypto.sl -> libcry
pto.sl.0
lrwxr-xr-x 1 root sys 18 Mar 12 16:20 libcrypto.sl.0 -> libc
rypto.sl.0.9.6
-r-xr-xr-x 1 root sys 1597440 Mar 12 16:20 libcrypto.sl.0.9.6
-rw-r--r-- 1 root sys 338560 Mar 12 16:20 libssl.a
lrwxr-xr-x 1 root sys 11 Mar 12 16:20 libssl.sl -> libssl.sl
.0
lrwxr-xr-x 1 root sys 15 Mar 12 16:20 libssl.sl.0 -> libssl.
sl.0.9.6
-r-xr-xr-x 1 root sys 352256 Mar 12 16:20 libssl.sl.0.9.6

[H.Merijn Brand (procura]

Ralph, I should have read the error better. What happens is that the test suite changes directory to t where it cannot find ./lib.sl.

Move the lib.sl to a location that is in the standard search path (/usr/lib), symlink it to the t directory, or re-read my first reply in this thread setting $LD_LIBRARY_PATH and/or $SHLIB_PATH"

Shannon, I have 4 gcc versions here, I don't think any module would appreciate a gcc-3.0.2/64 build against a gcc-3.0.4/32 object library :)

This morning I've posted a bug report to GNU about building gcc-3.0.4/64 with gcc-3.0.2/64 and binutils-2.12/64 and hope to get some answers soon.

[Ralph Grothe]

procura,

chatr doesn't honour SHLIB_PATH in the current setting of the built lib, but I think one can change this, was it the +s flag?

# chatr /opt/openssl/lib/libcrypto.sl
/opt/openssl/lib/libcrypto.sl:
shared library
shared library dynamic path search:
SHLIB_PATH disabled second
embedded path disabled first Not Defined
internal name:
libcrypto.sl.0.9.6
shared library list:
dynamic /usr/lib/libdld.2
dynamic /usr/lib/libc.2
shared vtable support disabled
static branch prediction disabled
executable from stack: D (default)
kernel assisted branch prediction enabled
lazy swap allocation disabled
text segment locking disabled
data segment locking disabled
third quadrant private data space disabled
fourth quadrant private data space disabled
data page size: D (default)
instruction page size: D (default)

[H.Merijn Brand (procura]

Nothing ugly about those symlinks. There's nothing wrong there. There's even a *huge* advantage in doing it this way. Applications will search for libwhatever.sl, which is a symlink to the real library. Now if I want to update that library, but some application has it open/in use, I cannot replace it with a newer version (say I want to replace the current libz.sl with the just released libz.sl version 1.1.4). Now that is is a symlink, it's easy as pie, the symlink can be removed, the new library can be installed without problem, because it has the version number in it's name, and thus cannot clash with already installed libraries, and the symlink can be manually restored.

I admit that lots of installation processes will fail because they `forget' to remove the existing symlink first.

[Ralph Grothe]

I made a distclean, supplied a symlink to /usr/lib (there was already a static version of libcrypt resident :-(
reran perl Makefile.PL, and gmake, but still get same error when running "gmake test"

Here is my symlink creation following your suggestion since exporting SHLIB_PATH seems useless to me:

# ll /usr/lib/libcrypt.*
-r--r--r-- 12 bin bin 1248 Oct 27 1997 /usr/lib/libcrypt.a
lrwxr-xr-x 1 root sys 35 Mar 12 17:31 /usr/lib/libcrypt.sl.0
.9.6 -> /opt/openssl/lib/libcrypto.sl.0.9.6

[H.Merijn Brand (procura]

Maybe now the -L. haunts you back, because it expects the lib to be in '.'

Just for a test, symlink the missing lib to t and test again.

If that succeeds, rebuild openssl (or whatever package defined the missing library), but now try to not add the -L (it should now be found because of the symlink on /usr/lib

Life is hard and then you die

[Ralph Grothe]

Yeah, I was well aware of the benefits from symlinks with respect to version control of library installations.
Was only joking when using "ugly" ;-)

[Steven Gillard_2]

Ralph,

The library should be "crypto", not "crypt"! You're missing the 'o' in your symlink. :)

Cheers,
Steve

[Ralph Grothe]

Doesn't work either,

I set a symlink to the module's test subdir (see below)

# ll t/
total 2
lrwxr-xr-x 1 root sys 35 Mar 12 17:48 libcrypto.sl.0.9.6 ->
/opt/openssl/lib/libcrypto.sl.0.9.6
-rwxr-xr-x 1 500 500 121 Jun 28 2001 ssl_context.t


Actually, the test script is rediculously terse

# cat t/ssl_context.t
print "1..1\n";

use Crypt::SSLeay::MainContext qw(main_ctx);

print "not " unless main_ctx() =~ /CTX/;
print "ok 1\n";

[H.Merijn Brand (procura]

it requires the *exact* name, so the symlink you made won't get accessed nor searched for

You have to symlink

# cd t
# ln -s /opt/openssl/lib/libcrypto.sl.0.9.6 libcrypto.sl.0

BTW My /real/ name is H.Merijn Brand (call me Merijn) and the company I work for is PROCURA