ISEE test email syntax

[Bonnie Temple]

I want to test the HP 9000, hp-ux 11.0 server to be sure ISEE is sending mail to external address particularly HP. I can get to the URL for isee.americas.hp.com and I can get to :5060.start.html. I can successfully send email to an internal address via sendmail, but I want to send a test to HP as if through ISEE and see the open issue when I logon to :5060.start.html to confirm it made it.
What is the correct syntax or method for sending a test?
Thanks.

[Bonnie Temple]

I think I found the answer to my own question.
This must be the test syntax.
/opt/resmon/bin/send_test_event disk_em

tail of callout.log shows entry as below for this test at 4:10pm on 1/19. But I don't see anything in my isee open cases when I log on to :5060.start.html

#tail callout.log
Getting maximum callout log file size
Maximum callout log file size is 100000
Determining length of current Log File
Length of current Log File is 17800
Returned from trimCalloutLog
Attempting to load the list of allowed scripts
Loading the list of allowed scripts
RemoteSupport script firstSync will be allowed to run when called
PreSync: contactIDsBONNIETEMPLEBTEMPLE@abc.COMISEE.AMERICAS.HP.COM=BONNIETEM
PLEBTEMPLE@abc.COMISEE.AMERICAS.HP.COM

[Bonnie Temple]

After test send I see this in mad.log
190105 16:27:50.550 ERROR,[WebServer] WebServer exception:
190105 16:27:50.551 ERROR,[WebServer] NetworkException (1): socket.cpp:374 (232)
Connection reset by peer Socket::read(): recv() failed [[172.30.14.46]]

190105 16:27:52.019 ERROR,[WebServer] WebServer exception:
190105 16:27:52.019 ERROR,[WebServer] NetworkException (15): unexpected eof

Where do I go from here?

[Liem Nguyen]

Bonnie,
1. You cannot send mail for testing, ISEE doesn't send mail to HP, it only sends the incidents to HP through port 80 (http).

2. You can send /opt/resmon/bin/send_test_event disk_em, but in your case, it doesn't seem to work. Could this be a firewall issue?

When you logon to :5060.start.html, click on Configuration options, then System, do you see the Entitlement information?

If you don't see this, then isn't working yet and most likely, it's with your firewall/proxy.

[Frank Alden Smith]

Bonnie,

Forgive me if you have already verified this, but as a first step to diagnose the issue I would suggest that you browse to the ISEE content server directly from the HP9000 host.

The URL would be:

http://isee..hp.com/mots/motserv

Where region is: americas, europe, or asiapac.

Note: If the host does not support a browser then you can use a browser on a pc located on the SAME subnet as the host.

If you can't reach the content server the next thing to do is try nslookup -- again from the host. Frequently, the issue can be traced to name resolution. If the name doesn't resolve make an entry in the /etc/hosts file.

If nslookup succeeds I would suspect proxy issues.

Take care,
frank

[Bonnie Temple]

We can reach that URL. I am on a PC on the same subnet and see the page as - (I saw somewhere in ITRC) - that we should see.

Thanks,

[Bonnie Temple]

Liem - I can see the entitlement information - my system handle and contact information. All are correct. Out in configuration, it will show me the details of our system. All looks normal there.

[Bonnie Temple]

We don't have a proxy. We do have a firewall. If ISEE is trying to send out port 80 (http) & sendmail from Unix system is able to send to my external email address for example, ISEE should be able to send out as well, correct?

[Frank Alden Smith · Accepted solution]

Bonnie,

The next thing to do is test the ISEE client software itself. Use the following test script, which should generate a "Testing connectivity" incident:

# /opt/hpservices/RemoteSupport/bin/iseeConnectivityTest.sh

1545807970.6@rtcuss

Note: If the command generated a test incident you should see an incident key similar to 1545807970.6@yourhostname

Browse back to the server and verify you can see the incident in ISEE.

If not I would return to the server and do an nslookup on isee.americas.hp.com.

Also a ps -ef | grep mad to establish the client software is running.

Take care,
frank

[Bonnie Temple]

Awesome. That worked. I can see the test incident. That's what I wanted to confirm.
I guess I was running the wrong test with the opt/resmon/...teset_disk_em ?

Thanks Frank!

Bonnie

[Frauke Denker_2]

Hello Bonnie,
the iseeConnectivityTest.sh test wether ISEE is able to send event to HP. What you do not test is wether EMS can talk to ISEE and that is as well a very important thing as the diagnostics detect the error and if EMS is not able to talk to ISEE you will not get an ISEE incident for a hardware problem. So please test that with the send_test_event disk_em (or what onitor you like). If you do not get an ISEE incident, checked wether the test event is reported in the /var/opt/resmon/log/rst.log. If not try to reenbale EMS, therefor start /etc/opt/but resmon/lbin/monconfig and type "e" ((E)nable Monitoring). Then send the test event again!
If the incident is reported in the rst.log but no ISEE case is generated check wether the system is able to resolve localhost and check the api.log, client.log and registrar.log in the /var/opt/resmon/log/ directory for any error messages.
Hope this helps.
Regards
Frauke

[Frank Alden Smith]

Bonnie,

I agree totally with Frauke Denker! There are three components that ALL must work together to transmit a hardware incident to the support center. My previous suggestions had you test connectivity to the isee.americas.hp.com content server and then the ISEE client software As each test succeeds we need to proceed further upstream to test the other two components, namely the event handler and health agent (AKA the rstlistener process and EMS respectively).

So, the next step I would take is to check that the event handler process is running. To do this enter:

# ps -ef | grep rstl

If the process is running then try to:

# send_test_event -av memory_ia64

I use memory instead of embedded disks because every server has to have memory.

Take care,
frank

[Bonnie Temple]

After following steps from Frank Denker * Frank Smith, where I'm at is: I don't see rstl listener running and send_test_event for the memory returned an error.
EMS is enabled.
test TestConnectivity.sh event shows up on my browser page, but it soon closes automatically. When I send a disk_em test it shows in logs all except the api.log

Doesn't it sound like there is connectivity but EMS is not really communicating with ISEE?

How do I start rstl ?
Below are tails from logs. My comments are surrounded by BT> Thanks,
Bonnie

# ps -ef | grep ems
root 1813 1 0 Jan 23 ? 0:00 /etc/opt/resmon/lbin/emsagent
root 4680 17301 2 12:53:16 pts/tCc 0:00 grep ems

BT>I went to /etc/opt/resmon/lbin/monconfig and started EMS anyway and it said it was already enabled.
After send_test disk_em
here is rst.logNotification Time: Thu Jan 27 16:09:42 2005

glantz sent Event Monitor notification information:
/storage/events/disks/default/8_0.5.0 is >= 1.
Its current value is INFORMATION(1).

Event data from monitor:

Event Time..........: Thu Jan 27 16:09:42 2005
Severity............: INFORMATION
Monitor.............: disk_em
Event #.............: 103
System..............: glantz.nglantz.com
Summary:
Disk at hardware path 8/0.5.0 : Test event

Description of Error:
This is a test message from the monitor to test the communication path
from the monitor to the notification mechanism.

Probable Cause / Recommended Action:

No action required.

Additional Event Data:
System IP Address...: 172.40.10.3
Event Id............: 0x41f9589600000002
Monitor Version.....: B.01.00
Event Class.........: I/O
Client Configuration File...........:
/var/stm/config/tools/monitor/rst_disk_em.clcfg
Client Configuration File Version...: H.01.00
Qualification criteria met.
Number of events..: 1
Associated OS error log entry id(s):
None
Additional System Data:
System Model Number.............: 9000/851
OS Version......................: B.11.00
System Firmware Version.........: 36.34
System Serial Number............: unavailable
System Software ID..............: 2013884922
EMS Version.....................: A.03.20
STM Version.....................: A.38.00
System Current Product Number...: unavailable
System Original Product Number..: unavailable
Latest information on this event:
http://docs.hp.com/hpux/content/hardware/ems/disk_em.htm#103

v-v-v-v-v-v-v-v-v-v-v-v-v D E T A I L S v-v-v-v-v-v-v-v-v-v-v-v-v

Component Data:
Physical Device Path...: 8/0.5.0
Device Class...........: Disk
Inquiry Vendor ID......: SEAGATE
Inquiry Product ID.....: ST39236LC
Firmware Version.......: HP05
Serial Number..........: 3BN06Q8Z000010350MKN
>---------- End Event Monitoring Service Event Notification ----------<

BT>System resolved when #ping localhost
No entry for these tests in the api.log

tail of event.log>------------ Event Monitoring Service Event Notification ------------<

Notification Time: Thu Jan 27 16:09:42 2005

glantz sent Event Monitor notification information:

/storage/events/disks/default/8_0.5.0 is >= 1.
Its current value is INFORMATION(1). Event data from monitor:
This is a test message from the monitor to test the communication path
from the monitor to the notification mechanism.

Probable Cause / Recommended Action:
No action required.
Additional Event Data:
System IP Address...: 172.40.10.3
Event Id............: 0x41f9589600000000
Monitor Version.....: B.01.00
Event Class.........: I/O
Client Configuration File...........:
/var/stm/config/tools/monitor/default_disk_em.clcfg
Client Configuration File Version...: A.01.00
Qualification criteria met.
Number of events..: 1
Associated OS error log entry id(s):
None
Additional System Data:
System Model Number.............: 9000/851
OS Version......................: B.11.00
STM Version.....................: A.38.00
EMS Version.....................: A.03.20

Event Time..........: Thu Jan 27 16:09:41 2005
Severity............: INFORMATION
Monitor.............: disk_em
Event #.............: 103
System..............: glantz.nglantz.com

Summary:
Disk at hardware path 8/0.5.0 : Test event


Description of Error:
This is a test message from the monitor to test the communication path
from the monitor to the notification mechanism.

Probable Cause / Recommended Action:

No action required.

Additional Event Data:
System IP Address...: 172.40.10.3
Event Id............: 0x41f9589600000000
Monitor Version.....: B.01.00
Event Class.........: I/O
Client Configuration File...........:
/var/stm/config/tools/monitor/default_disk_em.clcfg
Client Configuration File Version...: A.01.00
Qualification criteria met.
Number of events..: 1
Associated OS error log entry id(s):
None
Additional System Data:
System Model Number.............: 9000/851
OS Version......................: B.11.00
STM Version.....................: A.38.00
EMS Version.....................: A.03.20
Latest information on this event:
http://docs.hp.com/hpux/content/hardware/ems/disk_em.htm#103

v-v-v-v-v-v-v-v-v-v-v-v-v D E T A I L S v-v-v-v-v-v-v-v-v-v-v-v-v



Component Data:
Physical Device Path...: 8/0.5.0
Device Class...........: Disk
Inquiry Vendor ID......: SEAGATE
Inquiry Product ID.....: ST39236LC
Firmware Version.......: HP05
Serial Number..........: 3BN06Q8Z000010350MKN


>---------- End Event Monitoring Service Event Notification ----------<
(EOF):
BT> But on Frank Smithâ s note to see if rstl is runningâ ¦ itâ s not.
# ps -ef | grep rstl
root 16212 6783 2 16:25:20 pts/tLc 0:00 grep rstl


# send_test_event -av memory_ia64

Finding resource name associated with monitor memory_ia64.
send_test_event: Failed to find monitor name in sapcf

[Frauke Denker_2]

Hello Bonnie,
the memory_ia64 is supported for Integrity servers only - and the disk HW Path does not sound like a IA64 path. So you need to use dm_memory and not memory_ia64 in that case.
If the rstlistener is not running, you cannot get any isee events from EMS. Try to start the monitor by /sbin/init.d/rstemsListener start. If this does not start the listener, it might be that some other process already uses the port 1402 that is used by the listener. Check this with netstat -an|grep 1402.
If any other process uses this port, you can switch to another port. Therefor you need to reconfigure the listener and EMS:
1) Listener: change the rstPort to an unused one in the file /etc/opt/hpservices/contrib/emsListener/rstlistener.prop then start the listener again and check wether is runs now.
2) EMS: Edit the file /var/stm/config/tools/monitor/isee.sapcfg and change the TCP Port to the same value. Restart EMS through monconfig.
After that try again a send_test_event -v disk_em and check wether this generates an event now!
Hope this helps
Frauke

[Bonnie Temple]

Thanks Frank. Doesn't look like errors in logs for the dm_memory send. But there is nothing posted on the HP website for an incident test on our server.
Used command to start rstemsListener. Seemed to accept with no errors. But grep on rstl and rstemsListener returned no process. netstat for port 1402 didn't show it in use. What else would prevent the listener from starting?
Thanks. Bonnie

# ./send_test_event -av dm_memory
Finding resource name associated with monitor dm_memory.

Found resource name /system/events/memory
associated with monitor dm_memory.
Creating test file /var/stm/config/tools/monitor/dm_memory.test
for monitor dm_memory.
Making test file /var/stm/config/tools/monitor/dm_memory.test
for monitor dm_memory
indicate send test event for all resources.

Performing resls on resource name /system/events/memory
for monitor dm_memory to cause generation of test event.
Contacting Registrar on glantz
NAME: /system/events/memory
DESCRIPTION: System Memory Monitor

This resource monitors events for system memory. Event monitoring
requests are created using the Monitoring Request Manager. Monitoring
requests to detect changes in device status are created using the
Peripheral Status Monitor (psmmon(1m)) and Event Monitoring Service (EMS).

For more information see the monitor man page, (dm_memory(1m)).
TYPE: /system/events/memory is a Resource Class.

There is one resource configured below /system/events/memory:
Resource Class
/system/events/memory/49
#

# ./rstemsListener start
# ps -ef | grep rstl
root 27710 16959 2 17:47:40 pts/tVc 0:00 grep rstl
# netstat -an|grep 1402
#

TAIL OF RST.LOG
>------------ Event Monitoring Service Event Notification ------------<

Notification Time: Fri Jan 28 17:43:59 2005

glantz sent Event Monitor notification information:

/system/events/memory/49 is >= 1.
Its current value is INFORMATION(1).

Event data from monitor:

Event Time..........: Fri Jan 28 17:43:59 2005
Severity............: INFORMATION
Monitor.............: dm_memory
Event #.............: 103
System..............: glantz.nglantz.com

Summary:
Memory Event Type : Test event

This is a test message from the monitor to test the communication path from
the monitor to the notification mechanism.

Probable Cause / Recommended Action:
No action required.
Additional Event Data:
System IP Address...: 172.40.10.3
Event Id............: 0x41fac02f00000002
Monitor Version.....: B.01.00
Event Class.........: I/O
Client Configuration File...........:
/var/stm/config/tools/monitor/rst_dm_memory.clcfg
Client Configuration File Version...: H.01.00
Qualification criteria met.
Number of events..: 1
Associated OS error log entry id(s):
None
Additional System Data:
System Model Number.............: 9000/851
OS Version......................: B.11.00
System Firmware Version.........: 36.34
System Serial Number............: unavailable
System Software ID..............: 2013884922
EMS Version.....................: A.03.20
STM Version.....................: A.38.00
System Current Product Number...: unavailable
System Original Product Number..: unavailable
Latest information on this event:
http://docs.hp.com/hpux/content/hardware/ems/dm_memory.htm#103

v-v-v-v-v-v-v-v-v-v-v-v-v D E T A I L S v-v-v-v-v-v-v-v-v-v-v-v-v


>---------- End Event Monitoring Service Event Notification ----------<
(EOF):

[Frauke Denker_2]

Hello Bonnie,
without the listener running you don´t need to try a send_test_event because the listener will not get the event if the process does not run!
The question is why we have no listener. Check the logfile:
/var/opt/hpservices/contrib/emsListener/listener.trace
For any error messages. Try to use another port for the listener (wait the reconfiguring EMS until it is sure it is needed).
Regards
Frau

[Frank Alden Smith]

Bonnie,

I apologize for not replying sooner; unfortunately I wasn't being notified of your replies to this thread.

Now, the next two steps to sort out your problem, would be:

1)Verify the rstlistener executable (/opt/hpservices/contrib/emsListener/bin/rstlistener) exists.

2)Start the listener process manually like this:

# /sbin/init.d/rstemsListener start

Check to make sure rstlistener process is running:

# ps -ef | grep rstl

If so then you can do another send_test_event.

Take care,
frank