I relize this is not a new problem, but it's unsolvable without at least a minimal HPE help. I know HPE may not want to officially want to engage in support for MS G8 as we have G10 & G10+ but seeing as iLO4 updates are still realsed I'm writing here in a hope that someone with insider's knowledge can help.
Cutting to the chase:
- PCI passthru is impossible on Linux-based systems for any device while working perfectly in ESXi even for built-in Smart Array controller
- Linux throws "DMAR: Device is ineligible for IOMMU domain attach due to platform RMRR requirement. Contact your platform vendor." as iLO reports all device's memory as protected
- HPE published advisory recommending connrep disabling RMRR for physical PCI slots but unfortunately while Microserver Gen8 is affected, the method doesn't work for it, and it only applies to 3rd-party devices in a slot and not to e.g. built-in ethernets or P120i
Workarounds:
- Community tried hacking Linux kernel removing RMRDS / RMRR check which appears to work, but only HPE knows what implications this may have (i.e. we cannot be sure if iLO is just reserving these regions for monitoring tools "just-in-case" or actively doing something with them)
- The advisory specifies "fix" for physical PCI slots, but for example P120i doesn't have such designation. I tried putting 0xFF in NVRAM 0x26A and 0x26B in a hope that engineers maybe used different undocummented masks for internal PCIe but it obviously doesn't work.
- Final workaround: using ESXi. As it's propriatary we don't know what is it doing that makes PCI passthru work (maybe it's just YOLOing the DMA and ignoring RMRDS / RMRR all together?)
Help needed:
While I relize this is a community forum I know people from HPE and isnide contacts are reading and responding. I hope that HPE can just provide an access/information about a flag which can disable "HP Shared Memory" / RMRDS / RMRR for internal devices as it's definitelly safe to do so if ESXi is doing this.