Z8 dual M.2 hp-custom PCI adapter (4xNVME) follow-up

[UserTwo]

Hi,

the top-slot 933576 card has several jumpers. The AIC card L32646 is identical and
the manual at least documents:

If the computer has one card, connect the P127 2-pin header to the system

board HDD LED header with the 2-wire cable.

Ig the computer has multiple cards, connect thh P127 2-pin header to anither card`s
P128 header... This daisy chains teh signals together.
The E1 and E2 jumpers are not documented. What are they used for?

[SDH]

That is the Z Turbo Drive Dual Pro (ZTD DP) you're posting about. It is in the line of Z Turbo Drives HP has been producing for years now that allow use of very fast M.2 SSD sticks to be used in HP workstations and some of the HP business class PCs. There now are two Dual Pro versions, the original v1 and the new v2 (released July last year). The v1 has a sturdy die-cast black aluminum alloy heatsink and the v2 mainly added more cooling surface area comprised of thin (but easy to bend) black aluminum fins to provide even better cooling. That change was made by HP because the v2 is rated to run two PCIe Gen4 (and even two Gen5) M.2 SSDs. In our testing the original v1 card can also run fine with two Gen4 M.2 sticks without thermal throttling. The v2 can also run PCIe Gen3-controller M.2 sticks fine, as expected. Remember that the controller physically present on the M.2 stick is key to the higher speeds (but faster still means hotter).

The same basic Dual Pro card can come with a conventional vertical metal backplane plate screwed on, or with the top two black rotating locking levers used on the Z8 workstations. There are lots of HP part numbers for all these... option P/N, Assembly P/N, Spares P/N, Promotion P/N, newer L P/N, and on and on. If the device comes with HP M.2 sticks in place it gets other P/Ns. These also run fine with non-HP sticks, and they even run well with two of our spare HP SM951 AHCI-controller sticks. They seem to take any capacity M.2 stick on the market and reportedly will run fine with even higher capacity (and longer) future ones too. Here's the ZTD Dual Pro v1:

Original Z Turbo Drive Dual Pro

You or the HP manual are a little incorrect regarding the card's LED-activity 2-pin headers. Use of this feature is optional. Those same two headers are on both the v1 and v2 ZTD Dual Pro cards. P127 is for in-only. P128 is for out-only. If you just have a single ZTD DP installed attach a 2-wire LED-activity cable from the card's P128 out-header down to the motherboard's LED in-header. If you have installed a ZTD G1 (or G2) card plus a ZTD Dual Pro (or Quad Pro) card then attach one 2-wire cable from the out-header of the smaller ZTD G1/G2 card and run it over to the ZTD DP (or QP) P127 in-header. Then run a second 2-wire cable from the ZTD DP (or QP) P128 out-header down to the motherboard's LED in-header. That LED-activity in-header position on the motherboard is shown on a diagram on the inside of your case cover. The display of drive activity works better on more recent workstations compared to the Zx20 family.

It does not matter what the orientation of the cable's plug ends are at the headers (HP engineering on these is not polarized). P127 or P128 is printed in tiny letters on the ZTD Dual Pro v1 and v2 by each header. HDDACT_IN or _OUT is by the corresponding headers on the Quad Pro. These headers have been shifted by HP over to the top middle of the Dual Pro v2 for easier access (see picture below). FYI, StarTech sells inexpensive 12" or 18" female-female plug ended LED-activity cables as "IDC2PIN12" or "IDC2PIN18" which fit on the headers perfectly if you don't have an official HP LED-activity cable (those were also used with HP SAS LSI PCIe cards). They even have a 24" long one if you need that. If you are also running SATA HDD and SSD drives along with your M.2 SSDs the combined activity will all now show up at the front case drive activity indicator... it is quite the light show when all drives are going at once.

The E1 and E2 headers on the ZTD Dual Pro v1 and v2 remain in the same top right area but are less crowded now. Those two 3-pin headers and their 2-hole jumpers control the device number assigned to each ZTD device. More than one ZTD should not be assigned the same device number. Exactly where that device number shows up in BIOS or the OS is still a mystery, and I'd appreciate knowing where to look.  DGroves may know... he has worked with HP on the ZTD cards.

The jumpers come from the factory loaded onto the device assignment headers in a standard default position, and I assume that the default device number that results is the same for a ZTD G1 = ZTD G2 = ZTD DP = ZTD QP. So, if we are running only 1 ZTD device in a workstation we just leave its jumpers at their default positions. If we install 2 or more ZTDs of any type we keep the first at its default and change the jumper positions of the second ZTD enough so that the odds are good for no conflicts. You are right... the HP documentation for the ZTD Dual Pro is truly substandard, even in the new v2 manual (which adds almost no new info). There is a ZTD G2--ZTD Quad Pro jumper position document that is somewhat related to the ZTD Dual Pro that I'll post later, and info on that is in the Z820 Technical and Service Guide... valid still.

One critical thing that HP does not seem to discuss anywhere is the issue of setting PCIe slot bifurcation in BIOS if you're using a ZTD Dual or Quad Pro. Bifurcation has to do with changing a true x8 (or x16) electrical lane PCIe slot into 2 (or 4) separate x4 electrical lane equivalents. Leaving bifurcation in BIOS/ Advanced/ Slot settings at the default of "Auto" is not enough in the current BIOS of at least the Zx40 workstations. Instead, you need to change a x8 slot's bifurcation setting in BIOS to "x4x4" to get a ZTD Dual Pro's two M.2 sticks to be seen and to work in the Zx40s. Or, if you're using a ZTD Quad Pro in a x16 slot you need to change its bifurcation setting in BIOS from the default of "Auto" to "x4x4x4x4" to get all four M.2 sticks to show up and work.

However, in the BIOS of a Z4 G4 Xeon workstation I worked on recently I could just leave its PCIe x8 slot's bifurcation setting at the default of "Auto" and it bifurcated correctly. For now, we will just change that also in BIOS to "x4x4" so our IT guys don't get confused about how to configure things (we use a mix of Zx40 and Zx G4 workstations).

There are a number of relatively poor reviews of the ZTD Dual Pro cards out there because users simply did not know how to set bifurcation properly. Bifurcation does not exist in BIOS before the current Zx40 generation. That capability came first to the Z840 and was added later to the Z440 and Z640 by a BIOS update. It is a sweet feature and very fast. The upper M.2 stick in a DP card is "primary" for your boot/applications fastest M.2 drive, and the adjacent lower "secondary" M.2 stick can be much larger for your documents drive. We don't run any  of the slower SATA HDD or SSD drives in some of our workstations now.

HP is currently clearing out a good number of their brand new ZTD Dual Pro v1 cards on eBay through their resellers... I got a couple recently from the Alan guys for under 50 USD each including tax and shipping. The manual is not included in the sealed antistatic bag so I'll post a PDF of the v1 and v2 manuals here shortly. The newer ZTD Dual Pro v2:

[SDH]

The earlier 2018 and 2022 ZTD DP v1 manuals don't include any information on the use of LED activity cables.

The later 2022 ZTD DP v2 manual does... see section 12, and HP has it correctly described there... attached below.

[SDH]

These two pics should help if there is any confusion remaining... the ZTD Dual Pro shown is a v1.

In from a ZTD G2, out to the motherboard

The big picture...