topic Re: Device is ineligible for IOMMU domain attach due to platform RMRR requirement in ProLiant Servers (ML,DL,SL)

Device is ineligible for IOMMU domain attach due to platform RMRR requirement

pharidos — Tue, 02 Jun 2015 13:45:29 GMT

Hi Folks

Is PCI passthrough supported on DL380 Gen 9 server? I have a 2 port 10 gig X540-AT2 and a 2 port 10G 82599ES linecard and I am not able to pass through these devices to a quest

I get the error message "Device is ineligible for IOMMU domain attach due to platform RMRR requirement. Contact your platform vendor." in /var/log /messages on trying to passthrough any of the above mentioned 10G port

Regards

Prem

Logs

====

pharidos@uks9:~$ lspci -vvv -s 05:00.0 | grep driver
Kernel driver in use: pci-stub
pharidos@uks9:~$

pharidos@uks9:~$ sudo qemu-system-x86_64 -enable-kvm -nographic -smp 2 -m 4G -cdrom /home/pharidos/ubuntu15.04.serial.iso -device pci-assign,host=0000:05:00.0,id=eth0
qemu-system-x86_64: -device pci-assign,host=0000:05:00.0,id=eth0: Failed to assign device "eth0": Operation not permitted
qemu-system-x86_64: -device pci-assign,host=0000:05:00.0,id=eth0: Device 'kvm-pci-assign' could not be initialized
pharidos@uks9:~$
pharidos@uks9:~$ sudo tail -n1 /var/log/messages
Jun 2 14:41:46 uks9 kernel: [533287.588971] pci-stub 0000:05:00.0: Device is ineligible for IOMMU domain attach due to platform RMRR requirement. Contact your platform vendor.

Re: Device is ineligible for IOMMU domain attach due to platform RMRR requirement

pharidos — Tue, 02 Jun 2015 14:00:08 GMT

More info: Same issue is seen on trying to passthrough the devices using "vfio-pci" option instead of "pci-assign"

pharidos@uks9:~$ lspci -vvv -s 05:00.0 | grep driver
Kernel driver in use: vfio-pci

pharidos@uks9:~$
pharidos@uks9:~$ sudo qemu-system-x86_64 -enable-kvm -nographic -smp 2 -m 4G -cdrom /home/pharidos/ubuntu15.04.serial.iso -device vfio-pci,host=0000:05:00
.0,id=eth0
qemu-system-x86_64: -device vfio-pci,host=0000:05:00.0,id=eth0: vfio: failed to set iommu for container: Operation not permitted
qemu-system-x86_64: -device vfio-pci,host=0000:05:00.0,id=eth0: vfio: failed to setup container for group 20
qemu-system-x86_64: -device vfio-pci,host=0000:05:00.0,id=eth0: vfio: failed to get group 20
qemu-system-x86_64: -device vfio-pci,host=0000:05:00.0,id=eth0: Device initialization failed
qemu-system-x86_64: -device vfio-pci,host=0000:05:00.0,id=eth0: Device 'vfio-pci' could not be initialized
pharidos@uks9:~$ 

pharidos@uks9:~$ sudo tail -n1 /var/log/messages
Jun 2 14:55:34 uks9 kernel: [534115.142496] vfio-pci 0000:05:00.0: Device is ineligible for IOMMU domain attach due to platform RMRR requirement. Contact your platform vendor.
pharidos@uks9:~$

Re: Device is ineligible for IOMMU domain attach due to platform RMRR requirement

Suman_1978 — Wed, 05 Apr 2023 06:50:15 GMT

Hi,

I guess this issue could be with the Adapters or OS/Kernel you are using rather than a server.

Check this HP ~~advisory~~ for example.

[Moderator edit: Removed the broken link]

Check this Bug report from RedHat. Check comment 10.

You can refer to this document on VFIO.

http://vfio.blogspot.com/2014/08/iommu-groups-inside-and-out.html

Thank You!
I am a HP employee
__________________________________________
Was the post useful? Click on the white KUDOS! Thumb below to say Thank You!

Useful Links for ProLiant Servers

Self Solve Options for ProLiant Servers

Re: Device is ineligible for IOMMU domain attach due to platform RMRR requirement

aw___ — Wed, 03 Jun 2015 04:38:30 GMT

As the error message indicates, the problem is with the platform, ie. the server and the firmware running on it. Current Linux kernels no longer allow assignment of devices associated with VT-d Reserved Memory Reporting Regions (RMRRs). These regions are used by some platform vendors for, among other things, side-band communication between I/O devices and the management controller. Unfortunately the existence of these regions is fundamentally incompatible with PCI device assignment. PCI device assignment requires full control of the device's I/O virtual address space, yet the RMRR imposes a platform requirement to reserve and identity map a vendor defined range of memory. Honoring the RMRR would require the reserved region be imposed on the VM and prevent hot-plug, ignoring the RMRR risks that the device continues to do DMA to the RMRR defined memory, putting the integrity of the VM at risk.

It's my understand that HP has BIOS utilities that can disable these RMRRs on a per-slot basis. This issue is not related to PCIe ACS isolation issues of the specific adapter or the resulting IOMMU groups.

Re: Device is ineligible for IOMMU domain attach due to platform RMRR requirement

aw___ — Tue, 18 May 2021 12:55:00 GMT

Here's a Red Hat technical paper for anyone that may be seeing this issue on Red Hat kernels or anywhere else: https://access.redhat.com/articles/1434873

Re: Device is ineligible for IOMMU domain attach due to platform RMRR requirement

pharidos — Mon, 02 Nov 2015 15:53:47 GMT

This is definitely a HP issue. The same adapter I am using (INtel 82599ES and Intel X540-AT2) works fine on other servers with the same OS.

Please have a look at what Redhat has reported under https://access.redhat.com/articles/1434873

Re: Device is ineligible for IOMMU domain attach due to platform RMRR requirement

moarph — Wed, 05 Apr 2023 06:51:56 GMT

Hello,

I have the same problem. I applied the HP advisory.

~~http://h20564.www2.hpe.com/hpsc/doc/public/display?docId=emr_na-c04781229&sp4ts.oid=5249566~~

but no luck.

Am I correct: kernelflag has to be iommu=on andf nothing more?

Slot number in this advisory is the same slot number as shwon in the iLO device infroation tab?

If above questions are to be answered with yes, I have no idea how to get pci passthrough working with HP servers

[Moderator edit: The above link is no longer valid. Please refer to support.hpe.com]

Re: Device is ineligible for IOMMU domain attach due to platform RMRR requirement

Jimmy Vance — Wed, 05 Apr 2023 06:53:47 GMT

@pharidos wrote:

This is definitely a HP issue. The same adapter I am using (INtel 82599ES and Intel X540-AT2) works fine on other servers with the same OS.

Please have a look at what Redhat has reported under https://access.redhat.com/articles/1434873

This advisory explains how to configure the server for HPE and 3rd party adpaters

Advisory: Linux - KVM Virtual Machine Does Not Start and Displays "Device is ineligible for IOMMU domain attach due to platform RMRR requirement" Message on HP ProLiant Systems with an Intel Processor
~~http://h20564.www2.hpe.com/hpsc/doc/public/display?docId=emr_na-c04781229&sp4ts.oid=5249566~~

[Moderator edit: Removed the broken link. Please refer to https://support.hpe.com/]

Re: Device is ineligible for IOMMU domain attach due to platform RMRR requirement

Jimmy Vance — Mon, 30 Nov 2015 01:28:17 GMT

@moarph wrote:
Hello,
I have the same problem. I applied the HP advisory.
http://h20564.www2.hpe.com/hpsc/doc/public/display?docId=emr_na-c04781229&sp4ts.oid=5249566
but no luck.
Am I correct: kernelflag has to be iommu=on andf nothing more?
Slot number in this advisory is the same slot number as shwon in the iLO device infroation tab?
If above questions are to be answered with yes, I have no idea how to get pci passthrough working with HP servers

More information please

Whate model server server and NIC are you working with and what section of th advisory did you apply?

Have you opened a support case with HPE?

Re: Device is ineligible for IOMMU domain attach due to platform RMRR requirement

homer1990 — Tue, 04 Apr 2023 20:55:52 GMT

I know this thread is really old, but the advisory is not visible anymore. Can I find this somewhere else?

Re: Device is ineligible for IOMMU domain attach due to platform RMRR requirement

Sunitha_Mod — Wed, 05 Apr 2023 06:55:43 GMT

Hello @homer1990,

You can try find the information here: https://support.hpe.com/