ProLiant Servers (ML,DL,SL)
cancel
Showing results for 
Search instead for 
Did you mean: 

Device is ineligible for IOMMU domain attach due to platform RMRR requirement

 
pharidos
Visitor

Device is ineligible for IOMMU domain attach due to platform RMRR requirement

Hi Folks

Is PCI passthrough supported on DL380 Gen 9 server? I have a 2 port 10 gig X540-AT2 and a 2 port 10G 82599ES  linecard and I am not able to pass through these devices to a quest

 

I get the error message "Device is ineligible for IOMMU domain attach due to platform RMRR requirement. Contact your platform vendor." in /var/log /messages on trying to passthrough any of the above mentioned 10G port

 

Regards

Prem

 

Logs

====

pharidos@uks9:~$ lspci -vvv -s 05:00.0 | grep driver
Kernel driver in use: pci-stub
pharidos@uks9:~$

pharidos@uks9:~$ sudo qemu-system-x86_64 -enable-kvm -nographic -smp 2 -m 4G -cdrom /home/pharidos/ubuntu15.04.serial.iso -device pci-assign,host=0000:05:00.0,id=eth0
qemu-system-x86_64: -device pci-assign,host=0000:05:00.0,id=eth0: Failed to assign device "eth0": Operation not permitted
qemu-system-x86_64: -device pci-assign,host=0000:05:00.0,id=eth0: Device 'kvm-pci-assign' could not be initialized
pharidos@uks9:~$
pharidos@uks9:~$ sudo tail -n1 /var/log/messages
Jun 2 14:41:46 uks9 kernel: [533287.588971] pci-stub 0000:05:00.0: Device is ineligible for IOMMU domain attach due to platform RMRR requirement. Contact your platform vendor.
8 REPLIES
pharidos
Visitor

Re: Device is ineligible for IOMMU domain attach due to platform RMRR requirement

More info: Same issue is seen on trying to passthrough the devices using "vfio-pci" option instead of "pci-assign"

 

pharidos@uks9:~$ lspci -vvv -s 05:00.0 | grep driver
Kernel driver in use: vfio-pci

pharidos@uks9:~$
pharidos@uks9:~$ sudo qemu-system-x86_64 -enable-kvm -nographic -smp 2 -m 4G -cdrom /home/pharidos/ubuntu15.04.serial.iso -device vfio-pci,host=0000:05:00
.0,id=eth0
qemu-system-x86_64: -device vfio-pci,host=0000:05:00.0,id=eth0: vfio: failed to set iommu for container: Operation not permitted
qemu-system-x86_64: -device vfio-pci,host=0000:05:00.0,id=eth0: vfio: failed to setup container for group 20
qemu-system-x86_64: -device vfio-pci,host=0000:05:00.0,id=eth0: vfio: failed to get group 20
qemu-system-x86_64: -device vfio-pci,host=0000:05:00.0,id=eth0: Device initialization failed
qemu-system-x86_64: -device vfio-pci,host=0000:05:00.0,id=eth0: Device 'vfio-pci' could not be initialized
pharidos@uks9:~$ 

pharidos@uks9:~$ sudo tail -n1 /var/log/messages
Jun 2 14:55:34 uks9 kernel: [534115.142496] vfio-pci 0000:05:00.0: Device is ineligible for IOMMU domain attach due to platform RMRR requirement. Contact your platform vendor.
pharidos@uks9:~$ 
Suman_1978
HPE Pro

Re: Device is ineligible for IOMMU domain attach due to platform RMRR requirement

Hi,

 

I guess this issue could be with the Adapters or OS/Kernel you are using rather than a server.

 

Check this HP advisory for example.

Check this Bug report from RedHat.  Check comment 10.

 

You can refer to this document on VFIO.

http://vfio.blogspot.com/2014/08/iommu-groups-inside-and-out.html

 

Thank You!
I am a HP employee
__________________________________________
Was the post useful? Click on the white KUDOS! Thumb below to say Thank You!

Useful Links for ProLiant Servers

 

Self Solve Options for ProLiant Servers

aw___
Occasional Visitor

Re: Device is ineligible for IOMMU domain attach due to platform RMRR requirement

As the error message indicates, the problem is with the platform, ie. the server and the firmware running on it.  Current Linux kernels no longer allow assignment of devices associated with VT-d Reserved Memory Reporting Regions (RMRRs).  These regions are used by some platform vendors for, among other things, side-band communication between I/O devices and the management controller.  Unfortunately the existence of these regions is fundamentally incompatible with PCI device assignment.  PCI device assignment requires full control of the device's I/O virtual address space, yet the RMRR imposes a platform requirement to reserve and identity map a vendor defined range of memory.  Honoring the RMRR would require the reserved region be imposed on the VM and prevent hot-plug, ignoring the RMRR risks that the device continues to do DMA to the RMRR defined memory, putting the integrity of the VM at risk.

 

It's my understand that HP has BIOS utilities that can disable these RMRRs on a per-slot basis.  This issue is not related to PCIe ACS isolation issues of the specific adapter or the resulting IOMMU groups.

aw___
Occasional Visitor

Re: Device is ineligible for IOMMU domain attach due to platform RMRR requirement

Here's a Red Hat white paper for anyone that may be seeing this issue on Red Hat kernels or anywhere else: https://access.redhat.com/articles/1434873

pharidos
Visitor

Re: Device is ineligible for IOMMU domain attach due to platform RMRR requirement

This is definitely a HP issue. The same adapter I am using (INtel 82599ES and Intel X540-AT2) works fine on other servers with the same OS.

Please have a look at what Redhat has reported under https://access.redhat.com/articles/1434873

moarph
Occasional Visitor

Re: Device is ineligible for IOMMU domain attach due to platform RMRR requirement

Hello,

I have the same problem. I applied the HP advisory.

http://h20564.www2.hpe.com/hpsc/doc/public/display?docId=emr_na-c04781229&sp4ts.oid=5249566

but no luck.

Am I correct: kernelflag has to be iommu=on andf nothing more?

Slot number in this advisory is the same slot number as shwon in the iLO device infroation tab?

If above questions are to be answered with yes, I have no idea how to get pci passthrough working with HP servers

Jimmy Vance
HPE Pro

Re: Device is ineligible for IOMMU domain attach due to platform RMRR requirement


pharidos wrote:

This is definitely a HP issue. The same adapter I am using (INtel 82599ES and Intel X540-AT2) works fine on other servers with the same OS.

Please have a look at what Redhat has reported under https://access.redhat.com/articles/1434873


This advisory explains how to configure the server for HPE and 3rd party adpaters

Advisory: Linux - KVM Virtual Machine Does Not Start and Displays "Device is ineligible for IOMMU domain attach due to platform RMRR requirement" Message on HP ProLiant Systems with an Intel Processor
http://h20564.www2.hpe.com/hpsc/doc/public/display?docId=emr_na-c04781229&sp4ts.oid=5249566

 




__________________________________________________
No support by private messages. Please ask the forum!      I work for HPE

If you feel this was helpful please click the KUDOS! thumb below!   
Jimmy Vance
HPE Pro

Re: Device is ineligible for IOMMU domain attach due to platform RMRR requirement


moarph wrote:

Hello,

I have the same problem. I applied the HP advisory.

http://h20564.www2.hpe.com/hpsc/doc/public/display?docId=emr_na-c04781229&sp4ts.oid=5249566

but no luck.

Am I correct: kernelflag has to be iommu=on andf nothing more?

Slot number in this advisory is the same slot number as shwon in the iLO device infroation tab?

If above questions are to be answered with yes, I have no idea how to get pci passthrough working with HP servers


More information please

Whate model server server and NIC are you working with and what section of th advisory did you apply? 

 

Have you opened a support case with HPE?

 




__________________________________________________
No support by private messages. Please ask the forum!      I work for HPE

If you feel this was helpful please click the KUDOS! thumb below!