HPE Community read-only access December 15, 2018
This is a maintenance upgrade. You will be able to read articles and posts, but not post or reply.
Hours:
Dec 15, 4:00 am to 10:00 am UTC
Dec 14, 10:00 pm CST to Dec 15, 4:00 am CST
Dec 14, 8:00 pm PST to Dec 15, 2:00 am PST
ProLiant Servers (ML,DL,SL)
cancel
Showing results for 
Search instead for 
Did you mean: 

HP ProLiant DL360 Gen10 with SLES11 SP4 : hp-health fail to start: check_ilo2: BMC Returned Error

 
GregCabaj
Occasional Advisor

HP ProLiant DL360 Gen10 with SLES11 SP4 : hp-health fail to start: check_ilo2: BMC Returned Error

Hi All, Dear Community, 

I rarely use forums, but finally I hit the problem I cannot find solution to, and after spending a few days of troubleshooting I thought to ask for help here. Hopefully someone went through a similar issue and will be able to share the knowledge. 

After successful installation of SLES11 SP4 on HP DL360 Gen10, I was trying to install hp-health and hp-snmp-agents. Unfortunately hp-health service hangs up indefinitely during the startup. 

The errors in /var/log/messages

hpasmlited[10046]: check_ilo2: BMC Returned Error: ccode 0x0, Req. Len: 15, Resp. Len: 21
hpasmlited[10046]: The Integrated Lights-Out Management Processor is not responding!

System details: 
HP ProLiant DL360 Gen10
OS: SLES11 SP4
Kernel: 3.0.101-107-default
Kernel modules:
lsmod | grep hp
hpwdt 14212 0
hpilo 17496 0
ILO5: version 1.20 (also tested with 1.15 with same result)
HP packages: 
hp-health-10.70-1846.3.sles11
hp-snmp-agents-10.70-2962.5.sles11
(also tested with older versions with the same result)

The ILO5 looks healthy and is responsive. I just cannot access it from OS side. What else should I check? IPMI libraries are there: 

/lib/modules/3.0.101-107-default/kernel/drivers/char/ipmi/

Has anyone came across same problem? 

I would appreciate some hints, hence currently it is not possible to monitor the server with HPE OIDs. 

Many thanks,
Greg

9 REPLIES
JJSS
Occasional Visitor

Re: HP ProLiant DL360 Gen10 with SLES11 SP4 : hp-health fail to start: check_ilo2: BMC Returned Err

Hi,

You are not alone.

We are also having the same problem using SLES 12 SP3.

hp-health-10.70-1846.2.sles12.x86_64

 

 

Jimmy Vance
HPE Pro

Re: HP ProLiant DL360 Gen10 with SLES11 SP4 : hp-health fail to start: check_ilo2: BMC Returned Err

The health/snmp functionality was moved to iLO on HPE ProLiant Gen10 servers. hp-health, hp-snmp-agents, hp-smh* and hp-ams are for Gen9 servers and earlier.

Gen10 ProLiant use a new AMS (Agentless Management Service) service along with SMA (System Managmenet Assistance)

Refer to these documents

HPE Integrated Lights Out (iLO 5) for Gen10 Servers - Configuring iLO Management Settings
https://support.hpe.com/hpsc/doc/public/display?docId=emr_na-a00026111en_us&docLocale=en_US#N10256

HPE iLO 5 User Guide Starting on page 285
https://support.hpe.com/hpsc/doc/public/display?sp4ts.oid=1008876525&docLocale=en_US&docId=emr_na-a00039434en_us

 




__________________________________________________
No support by private messages. Please ask the forum!      I work for HPE

If you feel this was helpful please click the KUDOS! thumb below!   
GregCabaj
Occasional Advisor

Re: HP ProLiant DL360 Gen10 with SLES11 SP4 : hp-health fail to start: check_ilo2: BMC Returned Err

Hi Jim, 

Many thanks for the update. Appreciate it very much. I will go through the documents and give feedback here. 

Regards.

GregCabaj
Occasional Advisor

Re: HP ProLiant DL360 Gen10 with SLES11 SP4 : hp-health fail to start: check_ilo2: BMC Returned Err

Hi Jim, 

I have posted here a reply, which I cannot see anymore. 

We used hp-health package to automate some tasks ie. marking IML logs as resolved with hpasmcli. 

Unfortunately, the links you provided do not cover the subject on how IML logs could be marked as "resolved" via ILO's CLI. 

Thanks.

 

GregCabaj
Occasional Advisor

Re: HP ProLiant DL360 Gen10 with SLES11 SP4 : hp-health fail to start: check_ilo2: BMC Returned Err

 

Below is the example, showing that with ILO5 CLI setting severity to "Repaired" is not supported. The hpasmcli command on earlier ProLiant generations was allowing for this. 

mgmtserver:~ # ssh hpgen10ilo -l Administrator "show system1/log1/record25"
Administrator@hpgen10ilo's password:
show system1/log1/record25
status=0
status_tag=COMMAND COMPLETED
Mon Apr 16 13:53:59 2018

 

/system1/log1/record25
Targets
Properties
number=25
severity=Caution
date=04/16/2018
time=09:12:28
description=IMPORTANT: The Standard Boot Order (IPL) has been detected as corrupted and has been restored to default values.
Verbs
cd version exit show

################### Fix Attempt ######################################

mgmtserver:~ # ssh hpgen10ilo -l Administrator "set system1/log1/record25 severity=Repaired"
Administrator@hpgen10ilo's password:
set system1/log1/record25 severity=Repaired
status=2
status_tag=COMMAND PROCESSING FAILED
error_tag=COMMAND ERROR-UNSPECIFIED
Mon Apr 16 13:56:20 2018

Command is not supported for this property.

 

 

Jimmy Vance
HPE Pro

Re: HP ProLiant DL360 Gen10 with SLES11 SP4 : hp-health fail to start: check_ilo2: BMC Returned Err

I don't think you can mark repaired through the iLO CLI, using the CLI about all you can do with IML is clear the entire log. After checking around I have been told the HPE RESTful interface tool 'ilorest' can perform this function.  Here is the link to the tools documetnation. I'm not well versed with the tool so I can't provide and example for your task

https://hewlettpackard.github.io/python-redfish-utility/#overview

 

 




__________________________________________________
No support by private messages. Please ask the forum!      I work for HPE

If you feel this was helpful please click the KUDOS! thumb below!   
GregCabaj
Occasional Advisor

Re: HP ProLiant DL360 Gen10 with SLES11 SP4 : hp-health fail to start: check_ilo2: BMC Returned Err

The ilorest does not seem to be very helpful here and "clearlog" is not an option. Historical information on hardware is critical for infrastructure teams. 

iLOrest > serverlogs --selectlog=IML --repair
Usage: serverlogs [LOG_SELECTION] [OPTIONS]

Download the AHS logs from the logged in server.
example: serverlogs --selectlog=AHS

Clear the AHS logs from the logged in server.
example: serverlogs --selectlog=AHS --clearlog

Download the IEL logs from the logged in server.
example: serverlogs --selectlog=IEL -f IELlog.txt

Clear the IEL logs from the logged in server.
example: serverlogs --selectlog=IEL --clearlog

Download the IML logs from the logged in server.
example: serverlogs --selectlog=IML -f IMLlog.txt

Clear the IML logs from the logged in server.
example: serverlogs --selectlog=IML --clearlog

(IML LOGS ONLY FEATURE)
Insert entry in the IML logs from the logged in server.
example: serverlogs --selectlog=IML -m "Text message for maintenance"

Download logs from multiple servers
example: serverlogs --mpfile mpfilename.txt -o outputdirectorypath --mplog=IEL,IML

Note: multiple server file format (1 server per new line)
--url <iLO url/hostname> -u admin -p password
--url <iLO url/hostname> -u admin -p password
--url <iLO url/hostname> -u admin -p password

Insert customized string if required for AHS
example: serverlogs --selectlog=AHS --customiseAHS "from=2014-03-01&&to=2014-03-30"

(AHS LOGS ONLY FEATURE)
Insert the location/path of directory where AHS log needs to be saved.
example: serverlogs --selectlog=AHS --directorypath=C:\Python27\DataFiles

ilorest: error: no such option: --repair

Also when trying to use get or list command, it does not seem to be possible to even filter with "repaired" property. 

If this command works fine:

iLOrest > list Message --filter "OemRecordFormat=Hpe-IML"

why below returns error??

iLOrest > list Message --filter "/Oem/Hpe/Repaired=False"
Full data retrieval enabled. You may experience longer download times.
No type currently selected. Please use the 'types' command to
get a list of types, or input your type by using the '--selector' flag.

iLOrest > list Message --filter "/Oem/Hpe/Repaired=False" --selector=log
Full data retrieval enabled. You may experience longer download times.
Unable to locate instance for 'log' and filter '/Oem/Hpe/Repaired=False'

iLOrest > list Message --filter "Members/Oem/Hpe/Repaired=False" --selector=log
Full data retrieval enabled. You may experience longer download times.
Unable to locate instance for 'log' and filter 'Members/Oem/Hpe/Repaired=False'

 

HPE decisions on software moves and server managment limitations are very disappointing. Documentation for iloRest is at least poor, with examples for windows only??? What world do you guys live in?? 

https://hewlettpackard.github.io/python-redfish-utility/#overview

The vision of ordering another 300 Gen10 DL360s is at least worrying. 

Jimmy Vance
HPE Pro

Re: HP ProLiant DL360 Gen10 with SLES11 SP4 : hp-health fail to start: check_ilo2: BMC Returned Err

As a follow-up., after discussing what you were previously doing with the old tools, and that there is a now a gap in functionality that the older tools have been retired. The RESTful team is working to add the marked repaired option into the RESTful tool. 

 




__________________________________________________
No support by private messages. Please ask the forum!      I work for HPE

If you feel this was helpful please click the KUDOS! thumb below!   
GregCabaj
Occasional Advisor

Re: HP ProLiant DL360 Gen10 with SLES11 SP4 : hp-health fail to start: check_ilo2: BMC Returned Err

Jim , 

There have been two releases of restful api since we spoke last time and I still do not see the IML logs repair feature. 

Could you please update everyone here? 

Thanks.