HP NC532i NICS

[srussell]

I have two BL460c G6 servers that both have NC532i Dual port 10GbE Multifunction adapters. When these adapters are enabled (alone or in a team) they consume on initial connection 145MB nonpaged pool memory and then over time gradually climb to 200MB+
When I disable them the nonpaged pool returns to normal around 30MB
I have updated the drivers (v5.0.13.0), I have updated the firmware (v4.8.0), I have updated the NCU (v9.70.0.0), I have PSP 8.15A installed on server. The servers are clustered SQL2005 but are not in production so there are no connections being made to them. Please help shed some light on why these adapters consume so much non paged memory. I also have a BL685c server running 64bit 2003 Enterprise with the same adapter. It also shows considerable non paged pool usage by the adapters.

[JonathanT]

The reserved (non-paged memory) probably is there to handle DMA transfers between the card in memory. These are very sophisticated cards with TOE offload, etc. To me a card that can transfer 1100MB/sec (theoretical limit of 10GB/ethernet) aren't being wasteful in their reservation of 200MB or so of RAM. Load that blade down with 64GB of RAM and unleash it's potential!

[Cederberg]

Hi!

This i would think is normal as above writer is saying.

The same chip but mezzanine card has a note in the product bulletin.
"NOTE: The NC532m requires a minimum of 2 GB of server memory"

if i interpret this correctly that means that ~1gb for OS and ~1 gb for use of the nic. If this is not accurate, we can conclude that the nic needs alot of memory.

[rick jones]

I cannot speak to the specifics of the Windows driver for the NC532i (or NC532m for that matter) but in general, a driver and NIC will have a receive queue (rx queue) between them. The driver posts buffers to that queue and the NIC DMAs inbound packets into them. Those will be non pageable memory (it would be bad news to try to DMA to a page that wasn't there :) and the memory used will be a function of the size of the buffers and the size of the ring. The size of the buffers posted to the rx ring will depend on the MTU (to a first order anyway)

Some NICs might employ multiple rings - especially NICs which have "multi queue" support.

If there are ways to set the rx ring size, that might be a way to alter the quantity of memory consumed by the driver - at the risk of making the rx ring too small (remember, this is a 10 gigabit per second interface) and losing performance.

[srussell]

Thanks for your responses. I opened a support case with HP and they gave me the form email telling me to upgrade firmware and drivers (which I already informed them I had done in the support ticket)

So if these adapters require a lot of non paged memory specifically because they are 10GB then the issue now becomes the limitation of windows 2003 32 bit non paged memory at 256MB. These servers are not even in production yet and running idle the adapters are using 180MB. We have one other server with these adapters. It is 64bit and in production. The non paged memory peaks at 300MB (which is safe for 64bit and more) but for 32bit this will kill my servers and prevent things like TCP connections and system services that require non paged mem from working. Each of my servers has 24G of memory so there is no shortage of physical memory to the OS. Can you please elaborate on what is meant by RX queue and reconfiguring the size.

[rick jones]

rx queue - the "receive" queue I mentioned - it is a set of buffers shared between the card and the driver which the card uses to hold the packets coming in off the network. While I do not know if it is configurable under Windows (I have limited Windows experience), but on "other stacks" it is possible to tune the size of the receive (rx) queues.