Nagios NRPE agent on IA64 VMS 8.31h1

[Brent Weaver_1]

I see where people have gotten this working. I compiled the EXE's myself using the BUILD.COM file provided with Kit 2.0B. I am having the following issue:

$ check_nrpe "-H" 127.0.0.1 "-c" check_test
CHECK_NRPE: Error - Could not complete SSL handshake.
%NONAME-E-NOMSG, Message number 00000002

When I try to use nossl (-n) I get this message:

$ check_nrpe "-H" 127.0.0.1 -n
CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages.

My config is as follows:

$ ty nrpe.cfg
#############################################################################
# Sample NRPE Config File
# Written by: Ethan Galstad (nagios@nagios.org)
#
# Last Modified: 12-30-2002
#
# NOTES:
# This is a sample configuration file for the NRPE daemon. It needs to be
# located on the remote host that is running the NRPE daemon, not the host
# from which the check_nrpe client is being executed.
#############################################################################



# PORT NUMBER
# Port number we should wait for connections on.
# NOTE: This must be a non-priviledged port (i.e. > 1024).
# NOTE: This option is ignored if NRPE is running under either inetd or xinetd

server_port=5666



# SERVER ADDRESS
# Address that nrpe should bind to in case there are more than one interface
# and you do not want nrpe to bind on all interfaces.
# NOTE: This option is ignored if NRPE is running under either inetd or xinetd

#server_address=10.0.0.1



# ALLOWED HOST ADDRESSES
# This is a comma-delimited list of IP address of hosts that are allowed
# to talk to the NRPE daemon.
#
# NOTE: The daemon only does rudimentary checking of the client's IP
# address. I would highly recommend adding entries in your
# /etc/hosts.allow file to allow only the specified host to connect
# to the port you are running this daemon on.
#
# NOTE: This option is ignored if NRPE is running under either inetd or xinetd

#allowed_hosts=127.0.0.1

# DEBUGGING OPTION
# This option determines whether or not debugging messages are logged to the
# syslog facility.
# Values: 0=debugging off, 1=debugging on

debug=1



# COMMAND TIMEOUT
# This specifies the maximum number of seconds that the NRPE daemon will
# allow plugins to finish executing before killing them off.

command_timeout=60

dont_blame_nrpe=1

# COMMAND DEFINITIONS
# Command definitions that this daemon will run. Definitions
# are in the following format:
#
# command[]=
#
# When the daemon receives a request to return the results of
# it will execute the command specified by the argument.
#
# Unlike Nagios, the command line cannot contain macros - it must be
# typed exactly as it should be executed.
#
# Note: Any plugins that are used in the command lines must reside
# on the machine that this daemon is running on!
# Note that you will have to modify the definitions below
# to match the argument format the plugins expect. Remember, these are
# examples only!

# VMS Specif:
# Note the use of the "@" for the DCL procedures. This are executed
# by the user running the auxiliary server. You may need to check the
# privileges of the account in sysuaf and maybe the quotas.
# All this plug-ins are provided in the scripts directory.
# $ symbol is for parameters. So be sure to use double $ ($$) for logical
# names, device names and directories.
#

command[check_test]=@nrpe$scripts:check_test.com
command[test0]=@nrpe$scripts:test0.com
command[test1]=@nrpe$scripts:test1.com
command[test2]=@nrpe$scripts:test2.com
command[check_testp]=@nrpe$scripts:check_testp.com $ARG1$
command[check_disk]=@nrpe$scripts:check_disk.com $ARG1$
command[check_dkc100]=@nrpe$scripts:check_disk.com dkc100
command[check_dkc300]=@nrpe$scripts:check_disk.com dkc300
command[check_memory]=@nrpe$scripts:check_memory.com
command[check_cpu]=@nrpe$scripts:check_system.com CPU
command[check_prc]=@nrpe$scripts:check_system.com PRC
command[check_pfr]=@nrpe$scripts:check_system.com PFR
command[check_prr]=@nrpe$scripts:check_system.com PRR
command[check_fpl]=@nrpe$scripts:check_system.com FPL
command[check_mpl]=@nrpe$scripts:check_system.com MPL
command[check_dio]=@nrpe$scripts:check_system.com DIO
command[check_bio]=@nrpe$scripts:check_system.com BIO

$

I am able to execute @nrpe$scripts:check_test.com:

$ @nrpe$scripts:check_test.com
Test of NRPED is ok

Here is what I have installed:

$ prod sho hist
------------------------------------ ----------- ----------- --- -----------
PRODUCT KIT TYPE OPERATION VAL DATE
------------------------------------ ----------- ----------- --- -----------
HP I64VMS C X7.3-289 Full LP Install Val 21-OCT-2010
HP I64VMS VMS831H1I_UPDATE V6.0 Patch Install Val 20-OCT-2010
HP I64VMS VMS831H1I_PCSI V2.0 Patch Install Val 20-OCT-2010
HP I64VMS AVAIL_MAN_BASE V8.3-1H1 Full LP Install (U) 19-OCT-2010
HP I64VMS CDSA V2.3-306 Full LP Install Val 19-OCT-2010
HP I64VMS DWMOTIF_SUPPORT V8.3-1H1 Full LP Install (U) 19-OCT-2010
HP I64VMS KERBEROS V3.1-152 Full LP Install Val 19-OCT-2010
HP I64VMS OPENVMS V8.3-1H1 Platform Install Sys 19-OCT-2010
HP I64VMS SSL V1.3-284 Full LP Install Val 19-OCT-2010
HP I64VMS TCPIP V5.6-9ECO2 Full LP Install Val 19-OCT-2010
HP I64VMS TDC_RT V2.3-1 Full LP Install Val 19-OCT-2010
HP I64VMS VMS V8.3-1H1 Oper System Install Sys 19-OCT-2010
HP I64VMS WBEMCIM V2.61-A070728 Full LP Install Val 19-OCT-2010
HP I64VMS WBEMPROVIDERS V1.5-31 Full LP Install Val 19-OCT-2010
------------------------------------ ----------- ----------- --- -----------
14 items found.

[Hoff]

Google for "CHECK_NRPE: Error - Could not complete SSL handshake." finds various topics that appear potentially relevant.

Here's the Nagios FAQ, with some references:
http://support.nagios.org/knowledgebase/faqs/?section_id=4&expand=true&showdesc=true

and here:
http://support.nagios.org/knowledgebase/faqs/index.php?option=com_content&view=article&id=52&catid=35&faq_id=191&expand=false&showdesc=true

Or here:
http://www.siamkia.com/open-source-help/how-to-fix-check-nrpe-error-could-not-complete-ssl-handshake.html

Or here:
http://ubuntuforums.org/archive/index.php/t-498435.html

And here are some potentially relevant set-up steps for Nagios NRPE on VMS:
http://nrpevms.dbaalacarte.com/

And for giggles, I might well test the addresses for allowed_hosts and server_address within the config.

[Brent Weaver_1]

This is what I am getting now. Each check I try logs in the nrpe$log directory. The following is created:

SYS$SYSDEVICE:[NRPE.LOG]NRPE.LOG;2

TRACEBACK Error: failure on output lineTRACEBACK Error: failure on output lineTRACEBACK Error: failure on output lineTRACEBACK Error
: failure on output lineTRACEBACK Error: failure on output lineTRACEBACK Error: failure on output lineTRACEBACK Error: failure on ou
tput lineTRACEBACK Error: failure on output lineTRACEBACK Error: failure on output line

SYS$SYSDEVICE:[NRPE.LOG]NRPE.LOG;1

$ define sys$error nrpe$log:nrpe.log
$ nrpe_aux="$nrpe$:nrpe_aux.exe"
$ nrpe_aux -c nrpe$:nrpe.cfg -d -n
nrped: opening log file....
20: Added command[check_test]=(null)=

20: Added command[test0]=(null)=

20: Added command[test1]=(null)=

20: Added command[test2]=(null)=

20: Added command[check_testp]=(null)=

20: Added command[check_disk]=(null)=

20: Added command[check_dkc100]=(null)=

20: Added command[check_dkc300]=(null)=

20: Added command[check_memory]=(null)=

20: Added command[check_cpu]=(null)=

20: Added command[check_prc]=(null)=

20: Added command[check_pfr]=(null)=

20: Added command[check_prr]=(null)=

20: Added command[check_fpl]=(null)=

20: Added command[check_mpl]=(null)=

20: Added command[check_dio]=(null)=

20: Added command[check_bio]=(null)=

0: INFO: SSL/TLS NOT initialized. Network encryption DISABLED.
20: Connection from 3.230.76.66 port 17694735
20: Host address checks out ok
20: Handling the connection...
99: Could not read request from client, bailing out...
%SYSTEM-F-ACCVIO, access violation, reason mask=00, virtual address=0000000000000020, PC=0000000000089140, PS=0000001B
%TRACE-F-TRACEBACK, symbolic stack dump follows
image module routine line rel PC abs PC
SSL$LIBSSL_SHR32 0 0000000000053140 0000000000089140
NRPE_AUX NRPE_AUX handle_connection 54489 0000000000002712 0000000000032712
NRPE_AUX NRPE_AUX wait_for_connections
54398 0000000000002292 0000000000032292
NRPE_AUX NRPE_AUX main 54079 0000000000000CA2 0000000000030CA2
NRPE_AUX NRPE_AUX __main 53975 0000000000000232 0000000000030232
0 FFFFFFFF80B9EE92 FFFFFFFF80B9EE92
DCL 0 000000000006BD22 000000007AE27D22
%TRACE-I-END, end of TRACE stack dump

Any advice?

[Hoff]

This is going to take some debugging, presuming somebody hasn't already resolved and ported this code.

As for the specific error in the logs, here is an article describing how to debug an access violation (ACCVIO) error:

http://labs.hoffmanlabs.com/node/800

Also look to shut off SSL within the source build, given the references to SSL in the ACCVIO traceback, and given the HP SSL code has itself been somewhat of a moving target recently. (That may not fix this, but it's worth a try.)

The IP port value shown in the dump is unexpected.

The C compiler version looks slightly dodgy, too; V7.3 would be expected.

Could well be collateral from another corruption, but that's going to need some debugging to determine that.

[john Dite]

Hi Brent,

one of our customers who is using Nagios as well, who noticed the difference in behaviour between Alpha and Itanium has an escalation running against SSL. I'm not in the office at present, so I can't give you details of the call. As soon as I have more infos I'll post this.

[Brent Weaver_1]

I upgraded SSL to 1.4 this am and now have:

$ ty NRPE.LOG;3
$ define sys$error nrpe$log:nrpe.log
$ nrpe_aux="$nrpe$:nrpe_aux.exe"
$ nrpe_aux -c nrpe$:nrpe.cfg -d -n
nrped: opening log file....
20: Added command[check_test]=(null)=

20: Added command[test0]=(null)=

20: Added command[test1]=(null)=

20: Added command[test2]=(null)=

20: Added command[check_testp]=(null)=

20: Added command[check_disk]=(null)=

20: Added command[check_dkc100]=(null)=

20: Added command[check_dkc300]=(null)=

20: Added command[check_memory]=(null)=

20: Added command[check_cpu]=(null)=

20: Added command[check_prc]=(null)=

20: Added command[check_pfr]=(null)=

20: Added command[check_prr]=(null)=

20: Added command[check_fpl]=(null)=

20: Added command[check_mpl]=(null)=

20: Added command[check_dio]=(null)=

20: Added command[check_bio]=(null)=

0: INFO: SSL/TLS NOT initialized. Network encryption DISABLED.
20: Connection from 127.0.0.1 port 17694735
20: Host address checks out ok
20: Handling the connection...
20: Host is asking for command 'check_test' to be run...
20: Running command: @nrpe$scripts:check_test.com$
%SYSTEM-F-ACCVIO, access violation, reason mask=00, virtual address=0000FFFF0000
0000, PC=FFFFFFFF84C1EFA0, PS=0000001B
%TRACE-F-TRACEBACK, symbolic stack dump follows
image module routine line rel PC abs PC
DECC$SHR C$STRLEN strlen 3694 0000000000000180 FFFFFFFF84C1EFA0
DECC$SHR C$TXDOPRINT convert_spec 42850 000000000000BED2 FFFFFFFF84D6CF32
DECC$SHR C$TXDOPRINT decc$$txdoprint
43462 000000000000D772 FFFFFFFF84D6E7D2
DECC$SHR C$TXDOPRINT sprintf 43807 0000000000012282 FFFFFFFF84D732E2
NRPE_AUX CUSTOM syslog 1866 0000000000000132 0000000000036552
NRPE_AUX NRPE_AUX handle_connection 60222 0000000000002DA2 0000000000032DA2
NRPE_AUX NRPE_AUX wait_for_connections
60030 0000000000002292 0000000000032292
NRPE_AUX NRPE_AUX main 59711 0000000000000CA2 0000000000030CA2
NRPE_AUX NRPE_AUX __main 59607 0000000000000232 0000000000030232
0 FFFFFFFF80B9EE92 FFFFFFFF80B9EE92
DCL 0 000000000006BD22 000000007AE27D22
%TRACE-I-END, end of TRACE stack dump

I verified that the path to the COM file is accurate:
$ @nrpe$scripts:check_test.com
Test of NRPED is ok

Am I getting anywhere? I had to recompile nrpe with new SSL lib's.

[Hoff]

There's an application error (corrupt pointer, uninitialized pointer, logic bug) in, near or somewhere upstream from image NRPE_AUX module CUSTOM routine syslog circa line 1866.

[Brent Weaver_1]

Hoff - Thank so much for your time on this issue. Do you know how to resolve this bug? I am not a C programmer by any means.

[JohnDite]

Brent,

the case number is 4621089462.

There is a provisonal patch that has been produced:

tcpip$ssh_sshd2.exe
"V5.7-ECO1B"
21-OCT-2010 13:39:34.96

I don't know as yet whether this has resolved the customer's issue.

I'll update this as soon as we have had feedback.

[Richard Brodie_1]

It's hard to see (at least for me) how a new sshd is going to solve an SSL problem.