Weird NIC enumeration issue

[chuckk281]

Nick had the infamous NIC enumeration issue:

*************************

Hi All,

I have a few BL460c G6s that I am building out with Windows Server 2008 R2 Enterprise and have NIC enumeration issue I just can’t fathom.

So far I have looked at 3 blades, all identical except for the amount of memory.

The enclosure is running VC-ENET 1/10 (no Flex-10), no VC-FC

Each blade has;

• The 2 embedded LOMS (seen as 2 x 1GB NICs)
• 1 x HP NC550m Dual Port Flex-10 10GbE BL-c Adapter in Mezz. 1 (again seen as 2 x 1GB NICs)
• 1 x QLogic QMH2562 8Gb FC HBA in Mezz.2

2 of the blades have 18GB, one has 36GB

The issue is that on the 36GB blade the NICs were enumerated in the order of the profile “1, 2, 3, 4”, but on the 18GB blades the second Embedded NIC is found first, I.e. 2, 1, 3, 4.

I have looked in the BIOS and can see no reason for this (everything PCI related seems to be set the same), the only difference I can find is the larger memory on the 36GB blade.

Can different amounts of memory change NIC enumeration, or is there something more fundamental wrong?

********************

Martin also had the same issue:

I have just seen the same thing. I had to manually install 4 DL380 G7 servers yesterday. The servers had 12 NIC’s in total (4 build in NICs and 2*4 port PCI NIC’s). Temporarily I had to assign a static IP to one of the NICs, which would physically be NIC 1 on the server. However it was only on two of the servers that the NIC enumeration were the same.

I have also heard from a customer of mine that they have seen the same problem and it didn’t matter if they installed manually or deployed using RDP. They had actually experienced that the NIC enumeration would change from time to time when they installed the servers.

Michael provided some context:

This situation has existed ever since Microsoft introduced “Plug-and-Play”.  There is no way to control the discovery/enumeration behavior that I am aware of.  I could not find any recent Microsoft KB article about this, but here is one describing the PnP behavior in Windows 2003/XP (PnP in Windows 2008 is essentially the same):

http://support.microsoft.com/kb/825668

"...this behavior can occur on computers that have multiple logical disks on one Small Computer System Interface (SCSI) hard disk controller or on computers that have multiple network adapters that use the same driver."

And here is an additional article for Windows 2008 that only speaks of disk enumeration, but has a pointer back to the above article:

http://support.microsoft.com/kb/937251

Nick’s discovery that the change appeared to be triggered by a difference in memory is also significant.  This is definitely enough of a difference to cause the PnP discovery/enumeration to behave differently.  Even so, two servers that are as identical as possible can still have different discovery/enumeration results.

***************

Any other input or comments?