iLO 4 (DL360p Gen 8) is unusable out of the box due to the CLI being disabled

[anthony11]

https://skitch.com/anthonyeleven/erihf/terminal-ssh-135x48

Brand new DL360p Gen 8 right out of the box.  Plug in the serial console, and ESC( doesn't work to get an iLO prompt.  I manage to hit ESC 8 at the right split second to get into the [nearly useless] text-men iLO config utility and under Settings -> CLI I see this:

https://skitch.com/anthonyeleven/erihf/terminal-ssh-135x48

So it would seem that iLO4 has quietly made a major leap backwards compared to iLO2 and iLO3 in that the serial CLI is disabled by default.  This means that one needs

1) Local hands

2) That speak English

3) and can follow directions

4) and have a legacy VGA monitor

5) and a USB keyboard

in order to use something with "LIghts Out" right in the name.  I consider this nothing short of fraud.  Guess it's time to go back to using Sun/Oracle systems that understand the difference between a rackmount server and a desktop.

[anthony11]

Oh, and before someone tells me to change it and hit F10, think about it.  F10 is not an ASCII character.  It can't be sent over the serial console.  F10 translates to "ESC[21~", and the ESC exits out of the menu, so it is STILL impossible to save any changes.

And, no, I can't just plug in a legacy monitor and keyboard.  The system is thousands of miles away.

And, no, I can't just use the HTTPS interface because, again, it's a new system and the reason I'm trying to get into it is to configure the IP address/netmask/gateway.

[yoshac]

Ref the RBSU manual:

---

@anthony11 wrote:

F10 is not an ASCII character.  It can't be sent over the serial console.  F10 translates to "ESC[21~", and the ESC exits out of the menu, so it is STILL impossible to save any changes. 

---

Keystroke emulation is supposed to have a filter timer on the Esc key. Does that not work?

Ref the RBSU user guide, BIOS Serial Console (CLI) section, Escape sequences

BIOS Serial Console enables you to manually input unsupported keystrokes by entering escape

sequences. Each character must be entered within 2 seconds of pressing and holding down the

Esc

key to emulate the escape sequence.

<ESC>1 F1

<ESC>2 F2

<ESC>3 F3

<ESC>4 F4

<ESC>5 F5

<ESC>6 F6

<ESC>7 F7

<ESC>8 F8

<ESC>9 F9

<ESC>0 F10

<ESC>! F11

<ESC>@ F12

<ESC>h Home

<ESC>k End

<ESC>+ Insert

<ESC>- Delete

<ESC>? Page Up

<ESC>/ Page Down

<ESC>R<ESC>r<ESC>R System Reset

[Jimmy Vance]

I just checked several Gen8 systems that I know are still at default settings and CLI is enabled on all of them

[anthony11]

---

@yoshac wrote:

Ref the RBSU manual:

---

@anthony11 wrote:

F10 is not an ASCII character.  It can't be sent over the serial console.  F10 translates to "ESC[21~", and the ESC exits out of the menu, so it is STILL impossible to save any changes. 

---

Keystroke emulation is supposed to have a filter timer on the Esc key. Does that not work?

Ref the RBSU user guide, BIOS Serial Console (CLI) section, Escape sequences

---

I tried it about five times and each time it exited the menu.  Re the RBSU user guide, I had not thought to look there, since RBSU isn't involved.  I've managed to mostly avoid RBSU on new systems, though I had to subject myself to it to fix a number of used G6 systems we were stuck with, in order to change the NIC that was used for booting.  I found that the delete key didn't work, and that I couldn't cut/paste commands or random characters would be dropped. 

[anthony11]

---

@Jimmy Vance wrote:

I just checked several Gen8 systems that I know are still at default settings and CLI is enabled on all of them

---

Indeed, it was on some that we got, but the last *four* we've unboxed have had it disabled and the system name set to "linux" in iLO.  Two of those had no tag or sticker indicating the factory Administrator password.  Had they been shipped to a site where I couldn't arrange for capable remote hands (which tend to cost $300+ each visit) and a monitor/keyboard, they'd be rather expensive doorstops.  Note that these were shipped from HP,  and purchased through an authorized reseller.

I had to have someone go onsite to dig up a legacy keyboard and monitor to get me into iLO. Attached is a screen capture of the HTTP interface showing the disabled setting.  Note that this system also came out of the box with the iLO domain name set to "Gromit.com"

I've also attached a capture of the serial CLI on an earlier G8 system we received, showing the CLI disabled in the power-on iLO configuration utility.

On another system:

</>hpiLO-> reset map1
status=2
status_tag=COMMAND PROCESSING FAILED
error_tag=COMMAND ERROR-UNSPECIFIED
Sat Feb 25 05:28:22 2012

User not authorized.

</map1/dhcpendpt1>hpiLO-> set EnabledState=no
status=2
status_tag=COMMAND PROCESSING FAILED
error_tag=COMMAND ERROR-UNSPECIFIED
Tue Feb 21 13:11:23 2012

User not authorized.

</map1/enetport1/lanendpt1>hpiLO-> set ipendpt1 ipv4address=xxx.xx.x.xx subnetmask=255.255.255.0
status=2
status_tag=COMMAND PROCESSING FAILED
error_tag=COMMAND ERROR-UNSPECIFIED
Tue Feb 21 13:13:12 2012

User not authorized.


ipv4address: User not authorized.
subnetmask: User not authorized.

None of this crap ever happened to me on Sun hardware.  The serial console Just Works out of the box -- and for years has been delivered via an RJ45 on the chassis, not a stupid DB9 throwback to the 1990's.

Two offshored "experts" within HP told me to just plug in a keyboard and monitor, despite me having describe that this was as infeasible as it should be unnecessary.  The second, described as "ISS GCC & SW Complex Problem Resolution & Quality, Technology Services, Enterprise Group" claimed that it worked for her, though she refused to try it on a factory-fresh system, which is the problem.  She stopped responding then someone else closed the ticket.

[taylorb]

You said you bought them from a reseller.  Are we sure they are 100% factory fresh?   The fact that they had "Gromit.com" in there makes me think they were likely demo or returned servers.   The guys at the HP factory don't have time to mess around with stuff like that.    Since you can't physically see the servers, I guess you wouldn't be able to look for other signs of non-newness. 

[Oscar A. Perez]

Actually, "Gromit" is the project name used inside HP for the iLO Chip.  My guess is that the factory did not clear the NVRAM after running tests on this particular iLO4.

[anthony11]

---

@taylorb wrote:

You said you bought them from a reseller.  Are we sure they are 100% factory fresh?   The fact that they had "Gromit.com" in there makes me think they were likely demo or returned servers.   The guys at the HP factory don't have time to mess around with stuff like that.    Since you can't physically see the servers, I guess you wouldn't be able to look for other signs of non-newness. 

 Actually, "Gromit" is the project name used inside HP for the iLO Chip.  My guess is that the factory did not clear the NVRAM after running tests on this particular iLO4.

---

The reseller swears that they are factory-fresh and they appear to have shipped directly from HP.   Demo or returned was my first thought, but note that only one had Gromit.com -- but all had the server name "linux" set in iLO, and the serial CLI disabled.  Good to know that Gromit is an internal codename -- thanks! 

Most of our systems get shipped to a location in a state different from where I sit, and a local skilled & clued team member wrangles them.  Were there obvious signs of the systems not being new he would notice.  Systems shipped there he can plug legacy kbd+display into to fix, which is annoying but doable.  I'm worried about upcoming shipments, especially in other countries, where that won't be feasible.  There are rumors that HP has a service where they will configure systems before shipping, which I would gladly engage for overseas systems, but I can't get anyone to tell me how to arrange that.  I would gladly pay for a known iLO password to be set and for (system-specific) iLO IP/netmask/gateway to be preconfigured.  And of course the serial CLI being enabled, but that's supposed to be the factory default.  The widespread mistaking of servers for desktops has surprised me since we've given up trying to get systems from Sun/Oracle -- especially Cisco's UCS models.  Cisco has been making gear with serial consoles for decades, but when we talked to them it it was as though they were channeling Nigel Tufnel, not understanding why we can't just plug in a monitor when a system is deployed on the other side of the planet.  Oh, you don't have a monitor?  You can plug in a laptop instead.

It'd also be nice if ORCA would let me in instead of having to PXE boot the offline ACU .iso over the network, but I digress.

[Jimmy Vance]

Anthony, did you receive the private forum message I sent?