Simpler Navigation for Servers and Operating Systems - Please Update Your Bookmarks
Completed: a much simpler Servers and Operating Systems section of the Community. We combined many of the older boards, so you won't have to click through so many levels to get at the information you need. Check the consolidated boards here as many sub-forums are now single boards.
If you have bookmarked forums or discussion boards in Servers and Operating Systems, we suggest you check and update them as needed.
BladeSystem Server Blades
cancel
Showing results for 
Search instead for 
Did you mean: 

2GB memory required for 10/20GB adapters.

chuckk281
Trusted Contributor

2GB memory required for 10/20GB adapters.

Keith had a customer question regarding using high bandwidth adapters:

 

************

 

Cust is running on older blades….BL460c G6 with minimal memory – I think 12GB.  They just added an additional workload and are working through some issues.  They uncovered this note in the BL460c G6 QuickSpecs relating to 10Gb adapters:

 

NOTE: Each 10 Gigabit Ethernet adapter requires a minimum of 2GB of server

memory.

 

When they run tools like ‘ps aux’ and ‘top’, they’re not seeing anything specifically for the adapter.  Is it part of the RHEL kernel when it loads?  Pretty sure it is.  Just want to confirm and get some context.

 

***********

 

Reply from Richard:

 

*************

 

Others may have a veto, but indeed, it is memory consumed in the kernel. The 10 Gbit NICs tend to have rx multiqueue enabled (multiple receive queues), and between that and enabling JumboFrames, and perhaps tweaking the size of the receive queues (for those NICs where it is possible at least (... and your inability to tune receive queue sizes...) and you can have a fair bit of memory tied-up in receive buffers for the NIC to DMA inbound traffic to a memory location.

 

Whether adding a 10Gb NIC to an exsiting system will increase performance will depend of course on whether or not the 1 Gbit NIC was actually a bottleneck... In the broadest handwaving terms, the CPU cycles to send/recieve a frame over a 10 GbE network is no different than a 1 GbE Network, so if the workload is not all small packets, odds are quite good there certainly won't be a huge improvement. But getting the multi queue support going can help those things - assuming it isn't a single stream/flow of small packets...

 

************

 

Other comments?