Z640 928-Fatal PCIe error.

[dorbd]

Hey everyone,

I am using the HP Z640 workstation for FPGA development at work

in purpose to make it work, i need to work on "bcdedit.exe - set TESTSIGNING ON", because the driver is unsigned,

Xilinx PCI Express DMA Drivers for windows.

but when i do, the driver installation is cut in the middle, the computer crash and display the error.

I was looking over the solutions in the forum but nothing worked.

i was already updated the bios, but still nothing.

[DGroves]

windows 7 has the ability to run a virtual XP, i recomend you try this

https://www.microsoft.com/en-us/download/details.aspx?id=8002

[BambiBoomZ]

Dorb,

That is an a very technical issue and this is outside my direct experience. To me, it seems that the maker of the board should provide technical support for the installation.

However, here are a couple of wild guesses:

1_   "The TESTSIGNING Boot Configuration Option" :

https://docs.microsoft.com/en-us/windows-hardware/drivers/install/the-testsigning-boot-configuration...

>"The TESTSIGNING boot configuration option determines whether Windows Vista and later versions of Windows will load any type of test-signed kernel-mode code. This option is not set by default, which means test-signed kernel-mode drivers will not load by default on 64-bit versions of Windows Vista and later versions of Windows.

Note  For 64-bit versions of Windows Vista and later versions of Windows, the kernel-mode code signing policy requires that all kernel-mode code have a digital signature. However, in most cases, an unsigned driver can be installed and loaded on 32-bit versions of Windows Vista and later versions of Windows. For more information, see Kernel-Mode Code Signing Policy (Windows Vista and Later).

The TESTSIGNING boot configuration option is enabled or disabled through the BCDEdit command. To enable test-signing, use the following BCDEdit command:

C++

Bcdedit.exe -set TESTSIGNING ON

To disable test-signing, use the following BCDEdit command:

C++

Bcdedit.exe -set TESTSIGNING OFF

Note  After you change the TESTSIGNING boot configuration option, restart the computer for the change to take effect.

Caution  Administrative privileges are required to use BCDEdit to modify BCD. Changing some boot entry options using the BCDEdit /set command could render your computer inoperable. As an alternative, use the System Configuration utility (MSConfig.exe) to change boot settings.

Note  Before setting BCDEdit options you might need to disable or suspend BitLocker and Secure Boot on the computer.

To use BCDEdit, you must be a member of the Administrators group on the system and run the command from an elevated command prompt. To open an elevated Command Prompt window, create a desktop shortcut to Cmd.exe, right-click the Cmd.exe shortcut, and select Run as administrator.

The following screen shot shows the result of using the BCDEdit command-line tool to enable test-signing.

When the BCDEdit option for test-signing is enabled, Windows does the following:

• Displays a watermark with the text "Test Mode" in all four corners of the desktop, to remind users the system has test-signing enabled. Note  Starting with Windows 7, Windows displays this watermark only in the lower left-hand corner of the desktop.

• The operating system loader and the kernel load drivers that are signed by any certificate. The certificate validation is not required to chain up to a trusted root certification authority. However, each driver image file must have a digital signature."<END

_______________________________________________________

2_   And here is the "Xilinx PCI Express DMA Drivers and Software Guide":

https://www.xilinx.com/support/answers/65444.html

___________________________

3_  As mentioned, I'm not certain if the above are addressing the fundamental issue.  The error message suggests something  more along the lines of changing the length of time allowed for completion before the PCIe slot 4 timeout triggers the post halt.  In a z620,  I had repeated Quadro display driver crashes because of a completion timeout error.  This was fixed by altering a registry value:

https://www.drivereasy.com/knowledge/fix-display-driver-stopped-responding-and-has-recovered-error/

> "Solution 3: Increase the GPU Processing Time by adjusting the Timeout Detection and Recovery registry value

Note: This method will tell you how to modify the registry. It may help fix the problem. However, serious problems might occur if you modify the registry incorrectly. Therefore, make sure that you follow these steps carefully. It is recommended to back up the registry before you modify it so you can restore the registry if a problem occurs.

1) On your keyboard, press the Windows logo key  and R at the same time to invoke the Run box.

Type regedit and click OK to open Registry Editor.

2) Browse to and then click the following registry subkey:
  HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\GraphicsDrivers

3) On the Edit menu in the right pane, right-click on the blank place. Click New, and then select the following registry value from the drop-down menu specific to your version of Windows.

If your PC is running 32-bit operating system, follow these steps:

a) Select DWORD (32-bit) Value.

b) Type TdrDelay as the Name and click Enter.

c) Double-click TdrDelay and add “8” for the Value data and click OK.

If your PC is running 64-bit operating system, follow steps below:

a) Select QWORD (64-bit) Value.

b) Type TdrDelay as the Name and click Enter.

c) Double-click TdrDelay and add “8” for the Value data and click OK.

4) Restart your PC and check to see if the problem is resolved.

After modifying the registry here, if a problem occurs and you are not sure how to restore the registry, you can delete the added TdrDelay Name and restart your PC." <END

This worked: changing the timeout value stopped the driver crashes due to completion timeout.

These may be well off the mark but my best guess is that the problem is something similar to these two conditions in combination .

Let us know what you find out.

BambiBoomZ

[dorbd]

Hey DGroves, BambiBoomZ

thanks for ur answer

i followed ur instructions, ->

• the XPmode didnt help,

• i run testsigning mode,

  i increased the GPU processing time as u wrote,

  i downloaded the driver u gave,

  but when i install the driver, the computer is still crushing giving the same error.

i would like to hear some more suggestions if u have, i already followed every suggestion on the internet.

Thanks, Dor

[BambiBoomZ]

Dor,

The 928-Fatal PCIe error_Completion timeout may indicate a hardware problem:

Have a look at:

Z420 Workstation - "928-Fatal PCIe error... ...Completion timeout" GPU passthrough with r9 380, kvm

‎

09-16-2017 01:58 PM

Product: HP Z420 Workstation

Operating System: Linux

I am currently using an HP z420 workstation, with a xeon e5 2660 CPU, and gentoo linux installed. I have the latest BIOS and firmware. I have two video cards installed: the NVIDIA GeForce GT 310 that came with the system and an AMD R9 280 2GB. I am trying to use a qemu/kvm virtual machine and passthrough the r9 380 to a windows 10 guest. When I started the virtual machine at first, it worked with functioning output. I proceeded to install the drivers for the card in the virtual machine. Then, the whole system instantly reset and presented the following error during bootup:

928-Fatal PCIe error. 
    PCIe error detected Slot 2
    Completion timeout

I dont think I can even start that vm anymore without this happening, and I also attempted passing through the video card to a linux guest and the same error occured, before i even installed any drivers.

The r9 380 appears to function fine in a host machine. The error also occurs with an rx 560.

Is this a problem with the motherboard? Is there a potential solution to this?"

________________________________________________

"Hi yes it is the issue with the system board and the only fix is to replace the system board. Please contact Hp Technical support to check and replace the system board.

______________________________________________________________

"09-21-2017 03:53 AM

Thank you for posting on HP Forums,

installs Red Hat Enterprise Linux 6 (x86_64) on an HP Z workstation with two or more NVIDIA graphics cards configured, the system may reboot automatically and have following error during POST.

Resolutions:
To resolve this issue, select " Install system with basic video driver" on the boot menu, or use the nomodeset xdriver=vesa boot option at the boot prompt.

928 - Fatal PCIe Error
PCIe error detected slot 5
Completion Timeout

Debashish
I am an HP Employee
If the information I've provided was helpful,
give us some reinforcement by clicking the solution and kudos buttons,"

_________________________________________________________________

"09-21-2017 08:55 PM

Ok - i guess ill just get a new board - probably a different one i know will work for hardware virtualization. Thank you."

<END

_________________________________________________________________

In the example above, the completion timeout error was related to a (GPU) driver installation and in the context of hardware virtualization (qemu/kvm virtual machine).   As FPGA development is a form of hardware virtualization, that seems to be a very similar situation.

The two replies by HP employees indicate changing the motherboard.

What device is in Slot 4 (z640 Slot 4 = PCIe3 x8) in your system?  Is it an Xilink (https://www.xilinx.com/products/boards-and-kits/fmc-cards.html) ?  If so,

https://www.xilinx.com/support.html

has many resources  and the forum:

https://forums.xilinx.com

seem to be very active.

If you ar using Xilinx, have a look at this page:

Xilinx PCI Express - FAQs and Debug Checklist

https://www.xilinx.com/support/answers/69751.html

This is a difficult one!

BambiBoomZ

HP z620_2 (2017) (R7) > Xeon E5-1680 v2 (8-core@ 4.3GHz) / z420 Liquid Cooling / 64GB DDR3-1866 ECC Reg / Quadro P2000  5GB + GTX 1070 Ti 8GB / HP Z Turbo Drive M.2 256GB AHCI + Samsung 860 EVO 500GB + HGST 7K6000 4TB / Focusrite Scarlett 2i4 sound interface > 2X Mackie MR824 / 825W PSU /> Windows 7 Prof.’l 64-bit  > 2X Dell Ultrasharp U2715H  (2560 X 1440)

[BambiBoomZ]

I am currently using an HP z420 workstation, with a xeon e5 2660 CPU, and gentoo linux installed. I have the latest BIOS and firmware. I have two video cards installed: the NVIDIA GeForce GT 310 that came with the system and an AMD R9 280 2GB. I am trying to use a qemu/kvm virtual machine and passthrough the r9 380 to a windows 10 guest. When I started the virtual machine at first, it worked with functioning output. I proceeded to install the drivers for the card in the virtual machine. Then, the whole system instantly reset and presented the following error during bootup:

928-Fatal PCIe error. 
    PCIe error detected Slot 2
    Completion timeout

I dont think I can even start that vm anymore without this happening, and I also attempted passing through the video card to a linux guest and the same error occured, before i even installed any drivers.

The r9 380 appears to function fine in a host machine. The error also occurs with an rx 560.

Is this a problem with the motherboard? Is there a potential solution to this?"

________________________________________________

"Hi yes it is the issue with the system board and the only fix is to replace the system board. Please contact Hp Technical support to check and replace the system board.

______________________________________________________________

"09-21-2017 03:53 AM

Thank you for posting on HP Forums,

installs Red Hat Enterprise Linux 6 (x86_64) on an HP Z workstation with two or more NVIDIA graphics cards configured, the system may reboot automatically and have following error during POST.

Resolutions:
To resolve this issue, select " Install system with basic video driver" on the boot menu, or use the nomodeset xdriver=vesa boot option at the boot prompt.

928 - Fatal PCIe Error
PCIe error detected slot 5
Completion Timeout

Debashish
I am an HP Employee
If the information I've provided was helpful,
give us some reinforcement by clicking the solution and kudos buttons,"

_________________________________________________________________

"09-21-2017 08:55 PM

Ok - i guess ill just get a new board - probably a different one i know will work for hardware virtualization. Thank you."

<END

_________________________________________________________________

In the example above, the completion timeout error was related to a (GPU) driver installation and in the context of hardware virtualization (qemu/kvm virtual machine).   As FPGA development is a form of hardware virtualization, that seems to be a very similar situation.

The two replies by HP employees indicate changing the motherboard.

What device is in Slot 4 (z640 Slot 4 = PCIe3 x8) in your system?  Is it an Xilink (https://www.xilinx.com/products/boards-and-kits/fmc-cards.html) ?  If so,

https://www.xilinx.com/support.html

has many resources  and the forum:

https://forums.xilinx.com

seem to be very active.

If you ar using Xilinx, have a look at this page:

Xilinx PCI Express - FAQs and Debug Checklist

https://www.xilinx.com/support/answers/69751.html

This is a difficult one!

BambiBoomZ

HP z620_2 (2017) (R7) > Xeon E5-1680 v2 (8-core@ 4.3GHz) / z420 Liquid Cooling / 64GB DDR3-1866 ECC Reg / Quadro P2000  5GB + GTX 1070 Ti 8GB / HP Z Turbo Drive M.2 256GB AHCI + Samsung 860 EVO 500GB + HGST 7K6000 4TB / Focusrite Scarlett 2i4 sound interface > 2X Mackie MR824 / 825W PSU /> Windows 7 Prof.’l 64-bit  > 2X Dell Ultrasharp U2715H  (2560 X 1440)

[BambiBoomZ]

Thank you for posting on HP Forums,

installs Red Hat Enterprise Linux 6 (x86_64) on an HP Z workstation with two or more NVIDIA graphics cards configured, the system may reboot automatically and have following error during POST.

Resolutions:
To resolve this issue, select " Install system with basic video driver" on the boot menu, or use the nomodeset xdriver=vesa boot option at the boot prompt.

928 - Fatal PCIe Error
PCIe error detected slot 5
Completion Timeout

Debashish
I am an HP Employee
If the information I've provided was helpful,
give us some reinforcement by clicking the solution and kudos buttons,"

_________________________________________________________________

"09-21-2017 08:55 PM

Ok - i guess ill just get a new board - probably a different one i know will work for hardware virtualization. Thank you."

<END

_________________________________________________________________

In the example above, the completion timeout error was related to a (GPU) driver installation and in the context of hardware virtualization (qemu/kvm virtual machine).   As FPGA development is a form of hardware virtualization, that seems to be a very similar situation.

The two replies by HP employees indicate changing the motherboard.

What device is in Slot 4 (z640 Slot 4 = PCIe3 x8) in your system?  Is it an Xilink (https://www.xilinx.com/products/boards-and-kits/fmc-cards.html) ?  If so,

https://www.xilinx.com/support.html

has many resources  and the forum:

https://forums.xilinx.com

seem to be very active.

If you ar using Xilinx, have a look at this page:

Xilinx PCI Express - FAQs and Debug Checklist

https://www.xilinx.com/support/answers/69751.html

This is a difficult one!

BambiBoomZ

HP z620_2 (2017) (R7) > Xeon E5-1680 v2 (8-core@ 4.3GHz) / z420 Liquid Cooling / 64GB DDR3-1866 ECC Reg / Quadro P2000  5GB + GTX 1070 Ti 8GB / HP Z Turbo Drive M.2 256GB AHCI + Samsung 860 EVO 500GB + HGST 7K6000 4TB / Focusrite Scarlett 2i4 sound interface > 2X Mackie MR824 / 825W PSU /> Windows 7 Prof.’l 64-bit  > 2X Dell Ultrasharp U2715H  (2560 X 1440)

[BambiBoomZ]

Ok - i guess ill just get a new board - probably a different one i know will work for hardware virtualization. Thank you."

<END

_________________________________________________________________

In the example above, the completion timeout error was related to a (GPU) driver installation and in the context of hardware virtualization (qemu/kvm virtual machine).   As FPGA development is a form of hardware virtualization, that seems to be a very similar situation.

The two replies by HP employees indicate changing the motherboard.

What device is in Slot 4 (z640 Slot 4 = PCIe3 x8) in your system?  Is it an Xilink (https://www.xilinx.com/products/boards-and-kits/fmc-cards.html) ?  If so,

https://www.xilinx.com/support.html

has many resources  and the forum:

https://forums.xilinx.com

seem to be very active.

If you ar using Xilinx, have a look at this page:

Xilinx PCI Express - FAQs and Debug Checklist

https://www.xilinx.com/support/answers/69751.html

This is a difficult one!

BambiBoomZ

HP z620_2 (2017) (R7) > Xeon E5-1680 v2 (8-core@ 4.3GHz) / z420 Liquid Cooling / 64GB DDR3-1866 ECC Reg / Quadro P2000  5GB + GTX 1070 Ti 8GB / HP Z Turbo Drive M.2 256GB AHCI + Samsung 860 EVO 500GB + HGST 7K6000 4TB / Focusrite Scarlett 2i4 sound interface > 2X Mackie MR824 / 825W PSU /> Windows 7 Prof.’l 64-bit  > 2X Dell Ultrasharp U2715H  (2560 X 1440)

[Z440Roger]

From the "Maintenance and Service Guide" for the Z440/Z640/Z840 the solution is: "Move the card to a different slot. If the problem persists, replace the card."

Start with reseating the card and make sure that it´s properly seated in the PCIe socket.