Linking problem while building XML::Parser against libexpat

[Ralph Grothe]

Hi,

I need to install the CPAN module XML::Parser as a prerequisite for SOAP::Lite, as I wish to use this toolkit for scripting an XMLRPC Webservice.
Well, the issue rose as a side-effect during the installation of SOAP::Lite and all dependencies through the CPAN interface.
This usually goes automagically but the process stops if CPAN.pm cannot run the make test of each module on the way successfully and bails out for manual intervention.
I knew I would need to install the libexpat first which is used by XML::Parser
(although SOAP::Lite would run its Perl-only XML parser if didn't find libexpat, so it's said).
The first hurdle was getting libexpat.
At Sourceforge I only found a CVS repository.
This is usually no problem as you would only have to login and check out the source tree with a cvs client.
Unfortunately I'm sitting behind a proxy, and I haven't found any hints in the cvs manpage how to make the cvs client use a proxy (like environment settings such as http_proxy etc. with wget, lynx, LWP etc.)
Luckily I found an older source tarball of libexpat, grabbed that and compiled it with HP ANSI C.
As it looks the build resulted in 32Bit shared libs

[root@gouda:/root/.cpan/build/XML-Parser-2.34]
# ldd /usr/local/lib/libexpat.sl.4.0
/usr/lib/libc.2 => /usr/lib/libc.2
/usr/lib/libdld.2 => /usr/lib/libdld.2
/usr/lib/libc.2 => /usr/lib/libc.2


My problems start when I try to make the XML::Parser (where CPAN.pm bailed out).
During the make of libexpat this note got displayed amongst the usual output:

----------------------------------------------------------------------
Libraries have been installed in:
/usr/local/lib

If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the `-LLIBDIR'
flag during linking and do at least one of the following:
- add LIBDIR to the `SHLIB_PATH' environment variable
during execution
- use the `-Wl,+b -Wl,LIBDIR' linker flag

See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.
----------------------------------------------------------------------


Hm, so I tried passing the latter as additional LDDLFLAGs to Perl MakeMaker

# perl Makefile.PL LDDLFLAGS="-Wl,+b -Wl,/usr/local/lib"
Checking if your kit is complete...
Looks good
Writing Makefile for XML::Parser::Expat
Writing Makefile for XML::Parser

# grep LDDLFLAGS Makefile
# MakeMaker ARGV: (q[LDDLFLAGS=-Wl,+b -Wl,/usr/local/lib])
LDDLFLAGS = -Wl,+b -Wl,/usr/local/lib
$(PERLRUN) Makefile.PL "LDDLFLAGS=-Wl,+b -Wl,/usr/local/lib"
LDDLFLAGS='-Wl,+b -Wl,/usr/local/lib'


But when I run the make I got this 32-64Bit mismatch error from the linker

LD_RUN_PATH="/usr/local/lib" /usr/bin/ld -b +vnocompatwarnings -L/usr/local/lib -L/lib/pa20_6
4 Expat.o -o ../blib/arch/auto/XML/Parser/Expat/Expat.sl -lexpat
ld: Mismatched ABI (not an ELF file) for -lexpat
Fatal error.
*** Error exit code 1


I also changed the LIBC not to use the pa20_64 subdir, and any +DD64 from the CCFLAGS
as opposed to these

# grep -E '^(LDFLAGS|LDDLFLAGS|LIBC|CCFLAGS)' Makefile
LDDLFLAGS = -Wl,+b -Wl,/usr/local/lib
LDFLAGS = +DD64 -L/usr/local/lib -L/lib/pa20_64
LIBC = /lib/pa20_64/libc.sl
CCFLAGS = +z -Ae -D_HPUX_SOURCE -Wl,+vnocompatwarnings +DD64 -I/usr/local/include -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64


But all was in vain, and now I'm a bit stuck

Rgds
Ralph

[Ralph Grothe]

Hm, no one here who tried to build libexpat with HP ANSI cc and ran into the same trouble when linking XML::Parser?

I found several postings in other forums on the WWW regarding this issue, but all remained unanswered.
This is so frustrating.

To top all this there is a release note concerning HP's built of Perl 5.8.2 for HP-UX 11i where they laconically state that if you want XML::Parser you just need to install Expat before.
Ha, Ha...

http://h21007.www2.hp.com/dspp/tech/tech_TechSoftwareDetailPage_IDX/1,1703,5672,00.html

This may work with a gcc build?
But I don't wan't to carry around several Perl builds just to please singular modules and use up filesystem space.

[harry d brown jr]

Have you tried the expat package from http://hpux.cs.utah.edu/hppd/hpux/Development/Tools/expat-1.95.8/

I just installed it and the XML::Parser last week without difficulties.


live free or die
harry d brown jr

[Ralph Grothe]

Hello Harry,

yes I did, though I used the German mirror of the HP-UX Porting Centre for the download.
But even at the URL you posted it says in the description that the depot was built with GCC.
Since the Perl depot released from HP also is a GCC built I can imagine that the installation under these circumstances is pretty seamless.

Looks I will have to spare a little more space in /opt to install Merijn's GCC build of Perl (he even may have already included XML::Parser)

[harry d brown jr]

Yup, gcc is required.

Also, read the HPUX.install document and make sure you set your LDOPTS, usually something like export LDOPTS="+s -s -L/usr/local/lib -L/usr/local/lib/hpux32"


live free or die
harry d brown jr

[Ralph Grothe]

Harry,

thanks for the hint.
I untarred the docs from the HP Porters' depot (after already having swremoved this expat) just to revisit HPUX.Install where this is quoted.

I used this linker options for a new build of libexpat, and I also used the linker options libexpat's Makefile issued during make install



Libraries have been installed in:
/usr/local/lib

If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the `-LLIBDIR'
flag during linking and do at least one of the following:
- add LIBDIR to the `SHLIB_PATH' environment variable
during execution
- use the `-Wl,+b -Wl,LIBDIR' linker flag

See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.




I think to have tried all quirks known to me but the mismatch persists.

I can even trace down the problematic object files as the files' magic No. identifies,
both the XML::Parser's object files and the libexpat identify differently


# file Expat/Expat.o /usr/local/lib/libexpat.sl.5.0
Expat/Expat.o: ELF-64 relocatable object file - PA-RISC 2.0 (LP64)
/usr/local/lib/libexpat.sl.5.0: PA-RISC2.0 shared library

Do you have any ideas what compiler flags I need to pass or place in the Makefile so that Expat.o won't be compiled as an ELF-64 object?
If I could make it into a PA-RISC2.0 like libexpat than I think the linker wouldn't moan.

[H.Merijn Brand (procura]

/usr/local/lib/libexpat.sl.5.0: PA-RISC2.0 shared library

That's NOT a 64bit lib

So far I've not (yet) seen libexpat available as a binary 64bit port, so you will

1. have to port it yourself

or

2. ask me nicely to give it to you (I already compiled it for 64bit on 11.11)

r3:/wrk/pa20_64/lib 103 > ll libexpat*
988 -rw-r--r-- 1 merijn softwr 400516 Sep 6 14:04 libexpat.a
986 lrwxrwxrwx 1 merijn softwr 15 Sep 6 14:04 libexpat.sl -> libexpat.sl.5.0
985 lrwxrwxrwx 1 merijn softwr 15 Sep 6 14:04 libexpat.sl.5 -> libexpat.sl.5.0
984 -r-xr-xr-x 1 merijn softwr 229160 Sep 6 14:04 libexpat.sl.5.0
r3:/wrk/pa20_64/lib 104 >

Enjoy, Have FUN! H.Merijn

[Ralph Grothe]

Hello Merijn,

I would like to take your generous offer, and ask if you could supply me with the libs for 64Bit 11.00?

If the 11.11 libs cannot be used on 11.00 then don't bother.
Soap::Lite is clever enough to use a pure Perl parser for the XML if a libexpat isn't available.
It should just cost a little performance.

Finally there would be another option for me
to download your gcc Perl builds.
Other people who use these don't seem to run into this problem.

[H.Merijn Brand (procura]

For /you/, I built it :)

rw-rw-rw- 1 merijn softwr 222364 Jan 20 11:32 expat-1.95.8-elf64.tgz

-rwxrwxrwx merijn/softwr 43648 2005-01-20 11:30:37 /usr/local/pa20_64/bin/xmlwf
-rw-r--r-- merijn/softwr 40115 2005-01-20 11:30:37 /usr/local/pa20_64/include/expat.h
-r-xr-xr-x merijn/softwr 229272 2005-01-20 11:30:37 /usr/local/pa20_64/lib/libexpat.sl.5.0
lrwxrwxrwx merijn/softwr 0 2005-01-20 11:30:37 /usr/local/pa20_64/lib/libexpat.sl.5 -> libexpat.sl.5.0
lrwxrwxrwx merijn/softwr 0 2005-01-20 11:30:37 /usr/local/pa20_64/lib/libexpat.sl -> libexpat.sl.5.0
-rw-rw-rw- merijn/softwr 769 2005-01-20 11:30:37 /usr/local/pa20_64/lib/libexpat.la
-rw-r--r-- merijn/softwr 400628 2005-01-20 11:30:37 /usr/local/pa20_64/lib/libexpat.a
-rw-r--r-- merijn/softwr 7900 2005-01-20 11:30:37 /usr/local/pa20_64/man/man1/xmlwf.1

libexpat.a: archive file
libexpat.la: ascii text
libexpat.sl: ELF-64 shared object file - PA-RISC 2.0 (LP64)
libexpat.sl.5: ELF-64 shared object file - PA-RISC 2.0 (LP64)
libexpat.sl.5.0: ELF-64 shared object file - PA-RISC 2.0 (LP64)

http://www.cmve.net/~mbrand/expat-1.95.8-elf64.tgz

tomorrow also on the mirrors

Enjoy, Have FUN! H.Merijn

[Ralph Grothe]

Many thanks Merijn,

I downloaded the libs and untarred them beneath /usr/local

Then I remade the Makefile of XML::Parser,
with these additional args


# CC=/opt/ansic/bin/cc perl Makefile.PL INC=-I/usr/local/pa20_64/include LDDLFLAGS=-L/usr/local/pa20_64/lib

But the make yields the same error from the linker

# make
rm -f ../blib/arch/auto/XML/Parser/Expat/Expat.sl
LD_RUN_PATH="/usr/local/lib" /usr/bin/ld -b +vnocompatwarnings -L/usr/local/lib -L/lib/pa20_64
Expat.o -o ../blib/arch/auto/XML/Parser/Expat/Expat.sl -lexpat
ld: Mismatched ABI (not an ELF file) for -lexpat
Fatal error.
*** Error exit code 1

Stop.
*** Error exit code 1

Stop.


Well it still was supplied with the same /usr/local/lib path with the invalid libexpat
It must have prepended the path from Perl's config.
In the %Config has I found these keys to hold the wrong search path for libraries


# perl -MConfig -e 'print join"\n",@Config{qw(ccflags cppflags ldflags lddlflags lipth)},"\n"'
+z -Ae -D_HPUX_SOURCE -Wl,+vnocompatwarnings +DD64 -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OF
FSET_BITS=64
-Aa -D__STDC_EXT__ -D_HPUX_SOURCE +z -Ae -D_HPUX_SOURCE -Wl,+vnocompatwarnings +DD64 -I/usr/local/inclu
de
+DD64 -L/usr/local/lib -L/lib/pa20_64
-b +vnocompatwarnings -L/usr/local/lib -L/lib/pa20_64


Should I delete (or rename) the Expat stuff in /usr/local/lib so that the linker is forced to look in /usr/local/pa20_64?

[H.Merijn Brand (procura · Accepted solution]

don't REmove. just REname for now, but yes, that should help

# mv /usr/local/lib /usr/local/_lib_

You could also change the LD= line to have -L/usr/local/pa20_64/lib in front of the list, something like

LD=$(LD) -L/usr/local/pa20_64/lib

Enjoy, Have FUN! H.Merijn

[Ralph Grothe]

Ah, moving of the wrong libexpat files out of /usr/local/lib revealed an interesting error message.

# CC=/opt/ansic/bin/cc perl Makefile.PL INC=-I/usr/local/pa20_64/include LDDLFLAGS=-L/usr/local/pa20>
Note (probably harmless): No library found for -lexpat

Expat must be installed prior to building XML::Parser and I can't find
it in the standard library directories. You can download expat from:

http://sourceforge.net/projects/expat/

If expat is installed, but in a non-standard directory, then use the
following options to Makefile.PL:

EXPATLIBPATH=... To set the directory in which to find libexpat

EXPATINCPATH=... To set the directory in which to find expat.h

For example:

perl Makefile.PL EXPATLIBPATH=/home/me/lib EXPATINCPATH=/home/me/include

Note that if you build against a shareable library in a non-standard location
you may (on some platforms) also have to set your LD_LIBRARY_PATH environment
variable at run time for perl to find the library.


So the Module provides an extra EXPATLIBPATH and EXPATINCPATH environment variable.
I thought I read the README?
To my apology, the README doesn't mention :-(

Hey, but this works

# CC=/opt/ansic/bin/cc perl Makefile.PL EXPATINCPATH=/usr/local/pa20_64/include EXPATLIBPATH=/usr/lo>
Checking if your kit is complete...
Looks good
Writing Makefile for XML::Parser::Expat
Writing Makefile for XML::Parser
[root@gouda:/root/.cpan/build/XML-Parser-2.34]
# make
cp Parser/Encodings/x-sjis-cp932.enc blib/lib/XML/Parser/Encodings/x-sjis-cp932.enc
cp Parser/Encodings/iso-8859-7.enc blib/lib/XML/Parser/Encodings/iso-8859-7.enc
cp Parser/Style/Tree.pm blib/lib/XML/Parser/Style/Tree.pm
cp Parser/Encodings/iso-8859-9.enc blib/lib/XML/Parser/Encodings/iso-8859-9.enc
cp Parser/Encodings/x-euc-jp-unicode.enc blib/lib/XML/Parser/Encodings/x-euc-jp-unicode.enc
cp Parser/Encodings/README blib/lib/XML/Parser/Encodings/README
cp Parser/Encodings/euc-kr.enc blib/lib/XML/Parser/Encodings/euc-kr.enc
cp Parser/Encodings/windows-1250.enc blib/lib/XML/Parser/Encodings/windows-1250.enc
cp Parser/Encodings/windows-1252.enc blib/lib/XML/Parser/Encodings/windows-1252.enc
cp Parser/Encodings/big5.enc blib/lib/XML/Parser/Encodings/big5.enc
cp Parser/Encodings/iso-8859-3.enc blib/lib/XML/Parser/Encodings/iso-8859-3.enc
cp Parser/Encodings/Japanese_Encodings.msg blib/lib/XML/Parser/Encodings/Japanese_Encodings.msg
cp Parser/Style/Subs.pm blib/lib/XML/Parser/Style/Subs.pm
cp Parser/Encodings/iso-8859-4.enc blib/lib/XML/Parser/Encodings/iso-8859-4.enc
cp Parser/Encodings/iso-8859-8.enc blib/lib/XML/Parser/Encodings/iso-8859-8.enc
cp Parser/Encodings/x-euc-jp-jisx0221.enc blib/lib/XML/Parser/Encodings/x-euc-jp-jisx0221.enc
cp Parser/Encodings/iso-8859-2.enc blib/lib/XML/Parser/Encodings/iso-8859-2.enc
cp Parser/Encodings/x-sjis-jdk117.enc blib/lib/XML/Parser/Encodings/x-sjis-jdk117.enc
cp Parser/Encodings/x-sjis-unicode.enc blib/lib/XML/Parser/Encodings/x-sjis-unicode.enc
cp Parser/LWPExternEnt.pl blib/lib/XML/Parser/LWPExternEnt.pl
cp Parser/Style/Objects.pm blib/lib/XML/Parser/Style/Objects.pm
cp Parser.pm blib/lib/XML/Parser.pm
cp Parser/Style/Debug.pm blib/lib/XML/Parser/Style/Debug.pm
cp Parser/Encodings/x-sjis-jisx0221.enc blib/lib/XML/Parser/Encodings/x-sjis-jisx0221.enc
cp Parser/Style/Stream.pm blib/lib/XML/Parser/Style/Stream.pm
cp Parser/Encodings/iso-8859-5.enc blib/lib/XML/Parser/Encodings/iso-8859-5.enc
cp Expat.pm ../blib/lib/XML/Parser/Expat.pm
/usr/bin/perl /opt/cpan/lib/5.8.6/ExtUtils/xsubpp -noprototypes -typemap /opt/perl5/lib/5.8.6/E
xtUtils/typemap -typemap typemap Expat.xs > Expat.xsc && mv Expat.xsc Expat.c
cc -c -I/usr/local/pa20_64/include +z -Ae -D_HPUX_SOURCE -Wl,+vnocompatwarnings +DD64 -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 +O2 +Onolimit -DVERSION=\"2.34\" -DXS_VERS
ION=\"2.34\" +Z "-I/opt/perl5/lib/5.8.6/PA-RISC2.0-LP64/CORE" Expat.c
Running Mkbootstrap for XML::Parser::Expat ()
chmod 644 Expat.bs
rm -f ../blib/arch/auto/XML/Parser/Expat/Expat.sl
LD_RUN_PATH="/usr/local/pa20_64/lib" /usr/bin/ld -b +vnocompatwarnings -L/usr/local/lib -L/lib
/pa20_64 Expat.o -o ../blib/arch/auto/XML/Parser/Expat/Expat.sl -L/usr/local/pa20_64/lib -lexpat
chmod 755 ../blib/arch/auto/XML/Parser/Expat/Expat.sl
cp Expat.bs ../blib/arch/auto/XML/Parser/Expat/Expat.bs
chmod 644 ../blib/arch/auto/XML/Parser/Expat/Expat.bs
Manifying ../blib/man3/XML::Parser::Expat.3
Manifying blib/man3/XML::Parser::Style::Objects.3
Manifying blib/man3/XML::Parser::Style::Debug.3
Manifying blib/man3/XML::Parser.3
Manifying blib/man3/XML::Parser::Style::Subs.3
Manifying blib/man3/XML::Parser::Style::Tree.3
Manifying blib/man3/XML::Parser::Style::Stream.3

# make test
rm -f ../blib/arch/auto/XML/Parser/Expat/Expat.sl
LD_RUN_PATH="/usr/local/pa20_64/lib" /usr/bin/ld -b +vnocompatwarnings -L/usr/local/lib -L/lib
/pa20_64 Expat.o -o ../blib/arch/auto/XML/Parser/Expat/Expat.sl -L/usr/local/pa20_64/lib -lexpat
chmod 755 ../blib/arch/auto/XML/Parser/Expat/Expat.sl
cp Expat.bs ../blib/arch/auto/XML/Parser/Expat/Expat.bs
chmod 644 ../blib/arch/auto/XML/Parser/Expat/Expat.bs
PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0, 'blib/lib', 'bl
ib/arch')" t/*.t
t/astress.........ok
t/cdata...........ok
t/decl............ok
t/defaulted.......ok
t/encoding........"my" variable $p masks earlier declaration in same scope at t/encoding.t line 94.
t/encoding........ok
t/external_ent....ok
t/file............ok
t/finish..........ok
t/namespaces......ok
t/parament........ok
t/partial.........ok
t/skip............ok
t/stream..........ok
t/styles..........ok
All tests successful.
Files=14, Tests=130, 4 wallclock secs ( 2.56 cusr + 0.67 csys = 3.23 CPU)


You saved my day :-)