Deploy CustomOS fails: There are no MAC addresses registered for this system

[Craig Gilmore]

The Custom OS is CentOS 5.1.

This is a bare metal system. We discovered the system using the PXE boot and loading the pxelinux.bin file from the DHCP server that points to the CMS to grab the

We populated the repository with the CentOS files from the .iso.

However trying to deploy always returns this error.

We are using the demo licenses, and we licensed this new server before trying to deploy.

I've searched the Users Guide, the only troubleshooting comment is close is 17.11 talking about getting MAC address from iLO. Checking, we have both the Management Processor and the DHCP addresses discovered.

The DHCP server is not the CMS. The DHCP server does point to the CMS to get the RAM Disk boot file.

Where have I mis-stepped? What do I need to correct in my configuration to get the Deploy to work?

[Mitchell Kulberg]

Craig,

The "no mac addresses" error is a classic SNMP agent problem.

Although you put the PSP on the managed node, are you sure it's working properly?

Try the standard debugging routines for bad agent config:

run 'configure or repair agents" against the managed server and see if that helps.

If that doesn't work, also try running "data collection" against that node.

There is an snmp command you can run to check if the agents are running properly. I will look for that and get back to you.

Mitch

[Mitchell Kulberg]

Craig,

You can check the SNMP agents with the 'snmpwalk' command.

Run this command on the CMS against the managed node
putting the managed node name where it says
and putting the community string where it says

snmpwalk -Os -c -v 1 if

This command will return all the interface
related information including the MAC
accresses. If you get a valid response
then please verify that SIM is using the
right community string when contacting
the node. If that is not the problem,
we'll have to look elsewhere.

Side note: the above command works on RedHat
but I don't have a CentOS system to test on.

Mitch

[Craig Gilmore]

The target system does not have the PSP installed, as there is no OS yet.

Did you mean that the CMS Management server is missing the SNMP setup? Although we installed the PSP on the CMS, we found that several parts of the PSP were missing from the system.

We've now wiped the CMS, and are installing the CMS from scratch again.

[Mitchell Kulberg]

Oh, forgive me. You did say bare metal, but I was too focused on the error message and forgot about that little fact :(

I was talking about checking SNMP on the managed node. Although you want the PSP on the CMS, I didn't mean to check that and I hope you're not reinstalling for no reason :(

Anyway, back to your problem:

What I said earlier was true. The error message you got reflects the fact that SIM no longer is able to contact the agents and read the MAC addresses through SNMP.

HOWEVER, on a bare metal system, this is caused by different circumstances.

On a bare metal system, SIM gets the information about the node from the ICE-Linux RAMdisk, which DOES run the required PSP agents. The first time you PXE boot the node, the RAMdisk boots and contacts the CMS, and SIM discovers the node and is able to collect all the relevant information including MAC addresses.

After the system is fully discovered, ICE-Linux instructs the system to reboot. On a system like yours, with nothing on it, it tries to boot the local hard drive, but fails, so it tries PXE booting again. Since SIM already knows about the node, it does NOT rediscover, but instead, tells the node to boot from its local hard drive. This of course fails, and it tries to PXE boot again. This loop goes on forever.

Unfortunately, while all this is going on, SIM may try to contact the node and update its information. It sees the node because it's trying to PXE, but when it asks the agents for info, it gets nothing, since the system is not up. This is where the information on the MAC addresses gets wiped out.

This is obviously behavior we don't want.

The solution to this problem is to have the system POWER DOWN after completing the bare metal discovery. There is a release note about this. The release note simply says to watch the console of the server and when you see it starting to reboot, power it off.

The very next version or patch of this product will chance the default behavior to power down after discovery to avoid the problem.

So, when the rebuild of the CMS is complete, run the bare metal discovery, but power off after it is complete. then you should be able to at least TRY your CentOS deployment.

Mitch

[Mitchell Kulberg · Accepted solution]

If you want to have your CMS automatically power down nodes after they are discovered, which will avoid this and other problems, you can perform this simple procedure:


*** WARNING ***

This procedure has you modify a file that is not normally meant to be modified by users, in a directory that is not normally meant to be accessed by users. Making changes to ANY files in this directory other then the exact change listed here could be VERY BAD.

Not only might it break your CMS, but the people trying to help you won't think to look here, since the files are not meant to be modified. So any problems you cause might take a very long time to troubleshoot.

In other words, don't poke around and hack in here.
---------------------------------------

To power down nodes after bare metal discovery, modify the file

/opt/repository/taskchain/Discovery.xml

Change the word "ResetServer" to "PowerOffServer"

That's it.

[Craig Gilmore]

The CMS is reinstalled. Booting the bare metal blade shows the same behavior as on CentOS. There is no recorded MAC address.

However, the snmpwalk times out with the customer read community string.

Checking the PXE booted blade, the snmpd.conf file has the read community string set to "public". However, "public" is not the customer read community string. Doing the snmpwalk from the CMS using "public" as the community string, does report the four MAC address.

We've added the "public" string to the SIM server.

Now, it appears that we have properly discovered the Blade server. At least, we have files in /opt/repository/boot/pxelinux.cfg/ that appear to be the needed MAC address files.

We are now attempting a deploy. More soon.

[Mitchell Kulberg]

Excellent!

This 'feature' is also detailed in the release notes. The community string "public" is hard coded into the ramdisk, so that needs to at least be on the list of strings that are checked, even if it's not the main string in use.

[Craig Gilmore]

GREAT! We are past this problem.

It is important to note that the community string "public" must be one of the recognized SNMP strings on the SIM CMS. If you don't set the rocommunity string to "public" on the initial SIM install, you must go in and add the "public" string after install.

When the SNMP timeout occurs, the relationship between the Management Processor and the Server is not made. This is another way to notice that something happened with the SNMP discovery.

Also, it is important to note that, it does take several minutes for the post-discovery reboot to occur. And, the reboot only happens after the system is successfully discovered.

[Channing Benson]

I had this same problem. I removed the discovered bare metal systems from the list of servers and attempted rediscovery after tweaking Discovery.xml. Strangely enough though, I still had files in /opt/repository/boot/pxelinux.cfg that corresponded to the MAC addresses of the discovered systems, so the rediscovery failed with a weird "No Discovery. No Volume Groups Found" error.

I had to remove the files in pxelinux.cfg to get discovery to work properly again.