Vt-d issue on z600

[dave319]

Hi all,

I've just bought a used Z600 with the intention of installing QubesOS, which requires VT-d (IOMMU) for Xen Hypervisor. After hitting problems with VT-d during the install, I'm now aware of some issues with the Intel 5500/5520 chipset, but can't be sure if my system is affected. Has anybody some experience with this?

I have a Boot Block Date in BIOS of 01/07/10. According to the note at the bottom of page 11 of QuickSpecs document (https://content.etilize.com/User-Manual/1019458200.pdf), this means I have C2 revision of the 5520 chipset, so that should be good news, but VT-d is still not working for me.

For those with the older B3 stepping, the following might apply: https://www.suse.com/de-de/support/kb/doc/?id=7014344 mentions that erratas 47 & 53 were fixed by BIOS update. [edit: Not fixed, but rather disabled reporting so that the sytem didn't try to use faulty IOMMU]

Anyway, I have the latest BIOS 786G4 v03.61. I have enabled VTx and VT-d2 in Security>System Security. I have 2 CPUs (X5675) with all RAM sockets filled, and have disabled all of the settings in advanced chipset options (Memory Mode Interleaving, PCI SERR# Generation, VGA Palette Sniffing, NUMA split mode) but I don't entirely understand if these settings are relevant. 

Probably the best lead so far is from some instructions on the Citrix website (https://support.citrix.com/article/CTX136517_ga), and from Linux, I've get a return for 8086:342e (rev 13) which is an affected chipset. Does this mean I can't use VT-d at all on this machine???

My motherboard sticker says SP#591184-001 / AS#460840-003, and here is the full list of issues in the Intel 5520 datasheet.

Any ideas? Thanks for any help/experience

[DGroves]

in the bios, go to the "system security" section

note, you may have to replace the 55xx series xeons with 56xx models, read the QubesOS application section for supported CPU's however you need a 2010 boot block date which confirms it is a "version 2" Z600 motherboard in order to run the 56xx series xeons

here's a link to all the system options/features

https://support.hp.com/us-en/document/c01718461

[dave319]

Hi DGroves, thanks for the quick reply. I confirm I'm using 56xx CPUs and have a 2010 boot blcok date. Still no VT-d 😞

Anyone managed to get this to work?

[Hiatus]

I ran into some problems with virtualization and some other settings in BIOS after the last update. Not sure if I was extra confused that specific day, but to enable virtualization and hyperthreading, I had to have those options disabled in the BIOS.

Whenever I had hyperthreading enabled in BIOS - 6 cores. With hyperthreading disabled - 12 cores

Same with virtualization.

To get the Windows Hypervisor running I had to disable all virtualization options in BIOS. With all virtualization options enabled, I could not enable Hyper-V in Windows...

Somehow I got the feeling that "disable/enable" meant "disabled/enabled" in some of the settings after the last BIOS update...

A "d" makes a lot of difference.

As I said, I may be extra confused on this one. No one else seems to have ran in to this issue...

but worth trying disabling everything that you think you need to have enabled just no make sure.

Hiatus

[DGroves]

rather than changing bios settings you may not fully understand

please try setting the bios to "cmos" defaults and then try enabling VT-D

[dave319]

Thanks again for the replies. I've been investigating the BIOS options you both mentioned. I've tried 3 different BIOS versions (starting with default settings) and have not managed to get VT-d working, with settings as expected or reversed.

I installed MS Windows in order to use the HP Performance Advisor Software. It confirms I have the updated motherboard (0B54h) which tallys with the BIOS block date, and from the HP Specs in my intial post, this SHOULD mean that I have the C2 stepping. 

However, it turns out that Xen identifies the 5520 chipset as faulty on boot and disables Interrupt Remapping. I also confirmed in my dmesg log after booting Xen, that indeed, I appear to have the 13h version, which is the B3 stepping. 

I rather suspect that's the end of the line as far as using this Z600 for virtualisation. This 55x0 chipset issue seems to affect the whole range of Z400/600/800 machines as far as I've read, and I can't find any evidence at all that anybody has had success with Vt-d on any of these machines.

I've got QubesOS v4 working with PV instead of Vt-d, but that wasn't really the goal. 

Good luck to anybody who stumbles upon this thread following the same expectiations I had!