CPAN issues with proxy server

[chindi]

Hi ,

We need to install perl modules in our setup.

We dont have direct internet connection.

11iv3 host

when we are trying through our proxy we are getting the fllowing issues ;

cpan[2]> install Rex::Task
Warning: Cannot install Rex::Task, don't know what it is.
Try the command

i /Rex::Task/

to find objects with matching identifiers.

cpan[3]> i /Rex::Task/
No objects found of any type for argument /Rex::Task/

cpan[4]> install IO::All::FTP
Warning: Cannot install IO::All::FTP, don't know what it is.
Try the command

i /IO::All::FTP/

to find objects with matching identifiers.

cpan[5]> o conf
$CPAN::Config options from '/opt/perl_32/lib/5.8.8/CPAN/Config.pm':
commit [Commit changes to disk]
defaults [Reload defaults from disk]
help [Short help about 'o conf' usage]
init [Interactive setting of all options]

applypatch []
auto_commit [0]
build_cache [100]
build_dir [/.cpan/build]
build_dir_reuse [0]
build_requires_install_policy [ask/yes]
bzip2 [/usr/local/bin/bzip2]
cache_metadata [1]
check_sigs [0]
colorize_debug undef
colorize_output [0]
colorize_print undef
colorize_warn undef
commandnumber_in_prompt [1]
commands_quote undef
connect_to_internet_ok [0]
cpan_home [/.cpan]
curl [/usr/bin/curl]
dontload_hash undef
dontload_list undef
ftp []
ftp_passive [1]
ftp_proxy [33.xx.xx.xx:8080]
ftpstats_period undef
ftpstats_size undef
getcwd [cwd]
gpg []
gzip [/usr/contrib/bin/gzip]
halt_on_failure [0]
histfile [/.cpan/histfile]
histsize [500]
http_proxy [3.xx.xx.xx:8080]
inactivity_timeout [0]
index_expire [1]
inhibit_startup_message [0]
keep_source_where [/.cpan/sources]
load_module_verbosity [v]
lynx []
make [/usr/bin/make]
make_arg []
make_install_arg []
make_install_make_command [/usr/bin/make]
makepl_arg []
mbuild_arg []
mbuild_install_arg []
mbuild_install_build_command [./Build]
mbuildpl_arg []
ncftp []
ncftpget []
no_proxy []
pager [/usr/bin/more]
password undef
patch [/usr/bin/patch]
perl5lib_verbosity [v]
prefer_installer [MB]
prefs_dir [/.cpan/prefs]
prerequisites_policy [ask]
proxy_pass undef
proxy_user []
randomize_urllist undef
scan_cache [atstart]
shell [/sbin/sh]
show_unparsable_versions [0]
show_upload_date [0]
show_zero_versions [0]
tar [/usr/bin/tar]
tar_verbosity [v]
term_is_latin [1]
term_ornaments [1]
test_report [0]
trust_test_report_history [0]
unzip [/oracle/app/oracle/product/10.2.0/bin/unzip]
urllist
0 [http://perlmirror.indialinks.com/]
use_sqlite [0]
username undef
wait_list undef
wget [/usr/local/bin/wget]
yaml_load_code [0]
yaml_module [YAML]

[Matti_Kurkela]

In another thread, we saw that your proxy at 33.xx.xx.xx:8080 is really a HTTP proxy.

You should specify the "ftp_proxy" setting as "http://33.xx.xx.xx:8080".

The "http://" prefix tells the download utilities that the proxy is a HTTP proxy and FTP-over-HTTP technique should be used.

[chindi]

Hi Matti ,

Even after putting ftp_proxy its the same ;

cpan[4]> o conf
$CPAN::Config options from '/opt/perl_32/lib/5.8.8/CPAN/Config.pm':
commit [Commit changes to disk]
defaults [Reload defaults from disk]
help [Short help about 'o conf' usage]
init [Interactive setting of all options]

applypatch []
auto_commit [0]
build_cache [100]
build_dir [/.cpan/build]
build_dir_reuse [0]
build_requires_install_policy [ask/yes]
bzip2 [/usr/local/bin/bzip2]
cache_metadata [1]
check_sigs [0]
colorize_debug undef
colorize_output [0]
colorize_print undef
colorize_warn undef
commandnumber_in_prompt [1]
commands_quote undef
connect_to_internet_ok [0]
cpan_home [/.cpan]
curl [/usr/bin/curl]
dontload_hash undef
dontload_list undef
ftp []
ftp_passive [1]
ftp_proxy http://33.xx.xx.xx:8080
ftpstats_period undef
ftpstats_size undef
getcwd [cwd]
gpg []
gzip [/usr/contrib/bin/gzip]
halt_on_failure [0]
histfile [/.cpan/histfile]
histsize [500]
http_proxy http://33.xx.xx.xx:8080
inactivity_timeout [0]
index_expire [1]
inhibit_startup_message [0]
keep_source_where [/.cpan/sources]
load_module_verbosity [v]
lynx []
make [/usr/bin/make]
make_arg []
make_install_arg []
make_install_make_command [/usr/bin/make]
makepl_arg []
mbuild_arg []
mbuild_install_arg []
mbuild_install_build_command [./Build]
mbuildpl_arg []
ncftp []
ncftpget []
no_proxy []
pager [/usr/bin/more]
password undef
patch [/usr/bin/patch]
perl5lib_verbosity [v]
prefer_installer [MB]
prefs_dir [/.cpan/prefs]
prerequisites_policy [ask]
proxy_pass undef
proxy_user []
randomize_urllist undef
scan_cache [atstart]
shell [/sbin/sh]
show_unparsable_versions [0]
show_upload_date [0]
show_zero_versions [0]
tar [/usr/bin/tar]
tar_verbosity [v]
term_is_latin [1]
term_ornaments [1]
test_report [0]
trust_test_report_history [0]
unzip [/oracle/app/oracle/product/10.2.0/bin/unzip]
urllist
0 [http://perlmirror.indialinks.com/]
use_sqlite [0]
username undef
wait_list undef
wget [/usr/local/bin/wget]
yaml_load_code [0]
yaml_module [YAML]

=========================================

cpan[5]> install CAM::PDF
CPAN: Storable loaded ok (v2.15)
Going to read /.cpan/Metadata
Warning: Cannot install CAM::PDF, don't know what it is.
Try the command

i /CAM::PDF/

to find objects with matching identifiers.
CPAN: Time::HiRes loaded ok (v1.86)

cpan[6]> install Tk::Text
Warning: Cannot install Tk::Text, don't know what it is.
Try the command

i /Tk::Text/

to find objects with matching identifiers.

cpan[7]> install Tk::Text
Warning: Cannot install Tk::Text, don't know what it is.
Try the command

i /Tk::Text/

to find objects with matching identifiers.

cpan[8]> install Text::Reform
Warning: Cannot install Text::Reform, don't know what it is.
Try the command

i /Text::Reform/

to find objects with matching identifiers.

why am not able to download any module ???

[chindi]

Hi Matti ,

We ned to change Config.pm right ?

[chindi]

Hi Matti ,

From the proxy configuration support we got below reply ;

Force users to use FTP via browser (ftp over http) only, that way you’ll force the content to be scanned by the gateway security server.

Using the FW You should block all ftp outgoing traffic except for the Gateway Security – with such users will be forced using the Gateway Security proxy for FTP using a browser over port 8080

[Matti_Kurkela]

> cpan[5]> install CAM::PDF
> CPAN: Storable loaded ok (v2.15)
> Going to read /.cpan/Metadata
> Warning: Cannot install CAM::PDF, don't know what it is.

I guess that your CPAN shell previously attempted to download the metadata (= the list of available modules), and got an empty file. Now it thinks it has valid metadata, but actually it has nothing at all. Assuming that your proxy configuration is now correct, you would need to make the CPAN shell retry loading the metadata before you can do anything else.

Hmm... Looking at the output of the "help" command of the CPAN shell, I think you should try the "reload index" command first. If that does not help, exit the CPAN shell and remove /.cpan/Metadata; the CPAN shell should reload it automatically when it does not exist.

Edit:

Your situation seems to be exactly like the "http firewall" case in the CPAN shell documentation:

http://perldoc.perl.org/CPAN.html#WORKING-WITH-CPAN.pm-BEHIND-FIREWALLS

 Your proxy at port 8080 is apparently some sort of a security/anti-virus filter.

The answer from the proxy configuration support looks like the support person was thinking about "workstation-like" use only. "FTP using a browser" means ftp-over-http, exactly what we are trying to do by setting the ftp_proxy to a http:// prefixed value.

[chindi]

Hi Matti ,

root #/ >rm -rf /.cpan/Metadata
root #/ >cpan
Terminal does not support AddHistory.

cpan shell -- CPAN exploration and modules installation (v1.9301)
ReadLine support available (maybe install Bundle::CPAN or Bundle::CPANxxl?)

cpan[1]> reload index
CPAN: Storable loaded ok (v2.15)
CPAN: LWP::UserAgent loaded ok (v5.819)
CPAN: Time::HiRes loaded ok (v1.86)
LWP not available

Trying with "/usr/bin/curl -L -f -s -S --netrc-optional -U ":" -x "http://33.xx.:8080"" to get
ftp://cpan.mirror.ac.za/authors/01mailrc.txt.gz
curl: (52) Empty reply from server

System call "/usr/bin/curl -L -f -s -S --netrc-optional -U ":" -x "http://33.xx.:8080" "ftp://cpan.mirror.ac.za/authors/01mailrc.txt.gz" > /.cpan/sources/authors/01mailrc.txt.tmp23723"
returned status 52 (wstat 13312)
Warning: expected file [/.cpan/sources/authors/01mailrc.txt.gz.tmp23723] doesn't exist

Trying with "/usr/local/bin/wget -O /.cpan/sources/authors/01mailrc.txt.tmp23723" to get
ftp://cpan.mirror.ac.za/authors/01mailrc.txt.gz
ftp://cpan.mirror.ac.za/authors/01mailrc.txt.gz: Unsupported scheme.

System call "/usr/local/bin/wget -O /.cpan/sources/authors/01mailrc.txt.tmp23723 "ftp://cpan.mirror.ac.za/authors/01mailrc.txt.gz" "
returned status 1 (wstat 256)
Warning: expected file [/.cpan/sources/authors/01mailrc.txt.gz.tmp23723] doesn't exist
LWP not available

Trying with "/usr/bin/curl -L -f -s -S --netrc-optional -U ":" -x "http://33.xx.xx.xx8080"" to get
http://mirror.is.co.za/authors/01mailrc.txt.gz
curl: (52) Empty reply from server

System call "/usr/bin/curl -L -f -s -S --netrc-optional -U ":" -x "http://33.xx.xx.:8080" "http://mirror.is.co.za/authors/01mailrc.txt.gz" > /.cpan/sources/authors/01mailrc.txt.tmp23723"
returned status 52 (wstat 13312)
Warning: expected file [/.cpan/sources/authors/01mailrc.txt.gz.tmp23723] doesn't exist

Trying with "/usr/local/bin/wget -O /.cpan/sources/authors/01mailrc.txt.tmp23723" to get
http://mirror.is.co.za/authors/01mailrc.txt.gz
http://mirror.is.co.za/authors/01mailrc.txt.gz: Unsupported scheme.

System call "/usr/local/bin/wget -O /.cpan/sources/authors/01mailrc.txt.tmp23723 "http://mirror.is.co.za/authors/01mailrc.txt.gz" "
returned status 1 (wstat 256)
Warning: expected file [/.cpan/sources/authors/01mailrc.txt.gz.tmp23723] doesn't exist
LWP not available

Trying with "/usr/bin/curl -L -f -s -S --netrc-optional -U ":" -x "http://33.xx.xx.:8080"" to get
http://ftp.rediris.es/authors/01mailrc.txt.gz
curl: (22) The requested URL returned error: 404 Not Found

System call "/usr/bin/curl -L -f -s -S --netrc-optional -U ":" -x "http://33.xx.xx.xx:8080" "http://ftp.rediris.es/authors/01mailrc.txt.gz" > /.cpan/sources/authors/01mailrc.txt.tmp23723"
returned status 22 (wstat 5632)
Warning: expected file [/.cpan/sources/authors/01mailrc.txt.gz.tmp23723] doesn't exist

Trying with "/usr/local/bin/wget -O /.cpan/sources/authors/01mailrc.txt.tmp23723" to get
http://ftp.rediris.es/authors/01mailrc.txt.gz
--10:34:38-- http://ftp.rediris.es/authors/01mailrc.txt.gz
=> `/.cpan/sources/authors/01mailrc.txt.tmp23723'
Connecting to 33.xx.xx.xx:8080... connected.
Proxy request sent, awaiting response... 404 Not Found
10:34:39 ERROR 404: Not Found.

[chindi]

Hi Matti ,

cpan[1]> reload index
CPAN: Storable loaded ok (v2.15)
Going to read /.cpan/Metadata
CPAN: LWP::UserAgent loaded ok (v5.819)
CPAN: Time::HiRes loaded ok (v1.86)
LWP not available
LWP not available

Trying with "/usr/bin/curl -L -f -s -S --netrc-optional -U ":" -x "http://33.xx.x.x:8080"" to get
ftp://cpan.mirror.ac.za/authors/01mailrc.txt.gz
CPAN: Compress::Zlib loaded ok (v2.015)
Going to read /.cpan/sources/authors/01mailrc.txt.gz
............................................................................DONE
LWP not available
LWP not available

why am getting LWP not available ??????

[Matti_Kurkela]

I tried to access the mirror.is.co.za and ftp.rediris.es CPAN mirrors you used in your previous message, and neither of them seemed to be good. Apparently you noticed that on your own, and switched to cpan.mirror.ac.za instead. If that mirror works for you, that's OK.

LWP (also known as libwww-perl) is a perl module that can be used to access HTTP and FTP servers without using an external utility like wget or curl.

The "LWP not available" message means that the LWP module is not working in your system for some reason.

In that case, the CPAN shell uses other methods instead. In this case, it has apparently used the curl utility, successfully.

If the CPAN shell is now working with your proxy, you might want to try CPAN commands "install LWP" or "upgrade LWP".

The first one should install LWP if it is not installed; the second one will try to upgrade LWP, which might help if you have a version of LWP installed but it does not work.

[chindi]

Hi Matti ,

cpan[2]> install LWP
CPAN: Storable loaded ok (v2.15)
Going to read /.cpan/Metadata
LWP is up to date (5.820).

cpan[3]> upgrade LWP
All modules are up to date for LWP