Z620 Unable to Boot from Z-Turbo Drive

[Z-User]

I want to use a G2 Z-Turbo drive with 512GB NVMe as my boot disk in my first generation Z620. I purchased a single slot, 512GB Z-Turbo G2 drive off of Ebay. This drive included the original, HP branded Samsung SM961 NVMe drive. Per the installation instructions I installed this in slot 3. The jumper configuration is set for a single card (i.e. figure 1 in the Z-Turbo drive installation guide).

Since I want to use this as the boot drive I removed all other drives, booted Ubuntu 20.04.3 LTS, and installed the operating system. No issues were observed with the installation. Upon reboot the system is unable to boot from the drive. Searching online I came across this thread:

The truth about z620/z820 NVMe SSD boot 

I followed the instructions in the first post about enabling UEFI support for the storage devices and restarted. Still not boot. I decided to reinstall the OS just in case this change required as much. Same result.

I have tried everything I've read on the HP forums and I am still unable to get this device to boot. Is it possible to use a Z-Turbo Drive G2 (with its corresponding HP branded NVMe SSD) to boot a first generation Z620. I am running BIOS version J61 v03.96 which I believe supports NVMe booting (which, from the forums, appears was added in 03.84).

Has anyone ever gotten this to work? All the marketing material for the Z-Turbo Drive G2 indicates it is possible but I am unable to get it to work.

[SDH]

"All the marketing material for the Z-Turbo Drive G2 indicates it is possible but I am unable to get it to work."

That statement is incorrect.  HP made the NVMe-controller M.2 stick ZTD G2 for the ZX40 generation of workstations, not the ZX20 generation.  The ZTD G1 was made for your Z620, and there was a BIOS upgrade for that family of workstations which allowed it to work.  There are a lot of posts in this forum about the G1 versus G2 issue... use the search bar and also google to find those.  For all practical purposes the ZTD G2 (made for specific NVMe-controller M.2 HP sticks) will not work on your workstation.  There are a few workarounds mentioned in the posts but IMHO they are not practical.  The physical HP ZTD PCIe card that currently holds your NVMe M.2 stick itself will work fine in your Z620 if you can find an appropriate AHCI-controller M.2 HP stick made for this purpose...so not all is lost. 

Those particular HP G1 M.2 sticks are hard to find but I have built up several Z620s combining a ZTD G2 PCIe card with a 512GB HP ZTD G1 M.2 stick, and those work fine together.  The 265GB HP ZTD G1 M.2 sticks are also available with less searching.  These are not cheap for what you get because they are now rare.  These made-by-Samsung G1 M.2 sticks have specific HP firmware applied... the G2 PCIe card is nice because it includes a heatsink that fits over both the G1 and G2 sticks.  My terminology:  a ZTD G1 or G2 PCIe card without a M.2 stick is called a ZTD card and with a stick it is called a drive.

Considering the costs you might want to build up a Z440 workstation... 

[DGroves · Accepted solution]

there's a lot of misinformation on this subject regarding the Zx20 workstations

here's what i know from personal observations and testing (a short history on this)

1. this workstation line lacks the required "nvme" boot code in it's bios, and HP will not release a updated bios to fix this as these workstations were end of life BEFORE nvme booting was approved by intel and HP 

 nvme boot support was not integrated into most mainstream computers until the x99 and newer chipsets

and on the HP Zx40 and newer workstation models like your z440 (corrected thanks SDH!)

2. as far as i know only the retail/prosumer samsung sm950 NVME ssd contained the necessary nvme oprom in it's firmware allowing it to boot on the Zx20 line of workstations

2A. the HP Z Turbo/Z Turbo "G2" PCI-E cards are exactly the same, only the installed SSD was different  the  G2 card had a nvme sm951(MZHVxxxxxxxx) installed   and the non G card had a SM951 AHCI SSD (MZHPxxxxxxx) or the precursor AHCI XP941

Also note that the sm951 is a OEM non retail drive that was used by HP and Lenovo, however the firmware differs between them,.... both will work in the HP systems but the lenovo firmware has not been designed/tested for HP and vice versa

3. a very few Datacenter/Server grade PCI-E cards can also boot on the Zx20 workstations i have personally tested several

the Seagate Warpdrive, Intel 750 and 3600 (including the OEM P3605) series, and the apple ssd's using the SSUBX/SSUAX based controller and the correct apple custom M.2 style socket to pci-e adapter, OCZ RevoDrive 3 of these only the warpdrive and the intel 750/3x00 ssd's are quick by today's standards the others are slower

there are others like the intel 3500/3700 and the micron 9200/9300 ssd'd but i have not personally tested these models but some people claim they also will boot on the Zx20 workstations

these ssd's require the workstation bios to have the "boot EFI" option enabled and this prevents any other cards bios (such as a raid card) from showing during the bootup

also, pricing on these ssd's are usually quite a bit  higher, compared to SATA based ssd's which will work in any system

there is also a software based NVME boot loader called "Duet/Refind" that i have personally tested that adds the missing nvme boot code via a usb key, using this method has some minor restrictions

bottom line:...................

your current/only choice if you want to boot from the sm961 ssd is to use the software based nvme boot loader

PM me if you have any questions

[SDH]

DGroves has posted in the past that the ZTD G1 used two different M.2 sticks, the most common of which is the XP941 with an AHCI type controller.  The later much less common one (the SM951 with a AHCI controller) is significantly faster.  There also is a SM951 version with a NVMe controller (even faster) so you have to be careful what you're looking at. 

EDIT:  The model alphanumerics are different for the two SM951 types... H means AHCI and V means NVMe when you look in the right spots.  Look at the third alphanumeric in the upper "Part" number and you can also look for H vs V just beyond the hyphen in the "Model" number.  Examples below, using non-HP sticks... I did not have a bunch of the HP version pictures at the time, but I personally only use the HP ones because I know they work (given that they use unique HP firmware versions).  The method works the same HP vs non-HP:

I've posted in the past that HP should have named the first released ZTD the "ZTD G1" but they did correctly name the second the ZTD G2.  The G1 is AHCI controller based only; the G2 is NVMe controller only.  This is where confusion comes in... people don't clearly know the difference, and the BIOS for the ZX20 family only recognizes the AHCI controllers while that for the ZX40 family recognizes both.

Thus, the ZTD G1 also can work fine in the ZX40 next generation of workstations in addition to working in the ZX20 family.

[Z-User]

This is somewhat frustrating as I perform research and in the end what I read ends up being misleading. Everything I read suggests the ZTD G2 is capable of booting an NVMe drive in a Z620 workstation. Now it appears that information is no entirely accurate. It seems I've spent money on a ZTD G2 and SSD which will not function as intended.

As an FYI I do have the system booting from a ZTD G1 with a 256GB SSD (Samsung XP941). I wanted to replace it with a NVMe drive. Looks like that's no longer an option.

Very frustrated with the confusion HP publishes about their workstations. I already experienced this when I purchase a pair of Xeon E5-2697 v2 processors which I researched prior to buying. Document after document after document states these are supported processors. Nowhere did I read they're only supported in the second revision of the Z620 series.

In conclusion it looks like HPs own ZTD G2 products won't support booting in the Zx20 series of workstations...Thank you all for the guidance.

[DGroves]

the HP "quickspecs" documentation on this card plainly states/shows which card works in which systems, however most people don't know about the quickspecs it appears

https://www8.hp.com/h20195/v2/getpdf.aspx/c04200260.pdf?ver=10

again you can use the sm961 -  using Duet/Refind,

personally i did not find it that hard to setup once i fully read the directions

however many people seem to have trouble with it not realising when they need to remove the MS install media during the setup of the OS

[SDH]

I've edited in two pictures into my most recent prior post, above.  The HP and non-HP versions of these sticks have the same type of part/model numbers on the left.  However, on the right there is a HP xxxxxx-xxx Assembly part number and also shown is the unique HP firmware designation.  The different sized sticks can have the same firmware if they're the same type otherwise.  Sometimes on the rear of the stick there also is a HP "Spares" part number label.  Quite a mishmash...

[Z-User]

There's just a lot of confusion and I think it comes from HPs lack of clarity on what works with what. As for using a work around it's not that important to me as I already have a ZTD G1 being used as a boot device.

[SDH]

Well, let me refer you to a bit of little-known info on souping up your ZTD G1 card.  If you're lucky you already have a SM951 AHCI M.2 stick in your ZTD G1 rather than the much more common but slower XP941 AHCI-controllerM.2 stick.  I'd bet you don't based on how few of those were used.  It was only when the XP941 ran out that HP had to switch over to the SM951 AHCI stick.

First tip:  You can use the aluminum heatsink off a ZTD G2 card on a ZTD G1 card... it fits perfectly and adds quite a bit of cooling capacity and thus thermal throttling of performance is less likely no matter what M.2 stick is under the hood.  You can also buy a used ZTD G2 PCIe card with that heatsink that no longer has a M.2 stick on it and add a SM951 AHCI G1 stick of desired capacity.  Make sure if you do that the special small HP black plastic/brass milled screw that holds the stick to the card is included.

Second and most important tip:  Take a quick look at this interesting LINK from QuietPC.com on the differences between the very common older slower XP941 AHCI M.2 card HP used in the vast majority of the ZTD G1 drives, versus the much less common faster cooler-running HP SM951 AHCI-controller M.2 stick you can use instead:

https://www.quietpc.com/samsung-m2-ssds

Note that little review is only comparing the two AHCI-controller M.2 drives HP used in the ZTD G1.  A video link provided there, however, is for the SM951 NVMe M.2 drive that was released shortly thereafter.

The SM951 AHCI is significantly faster than the XP941 because it can use faster PCIe generation 3.0 slots and also 4-lane technology versus the PCIe gen 2.0/2-lane technology that is the best the XP941 M.2 card can utilize. Thus, you would only want to insert a ZTD G1 card using the SM951 AHCI M.2 stick into a ZX20 PCIe 3.0 slot having at least 4 lane capabilities.  The best slot HP advises to use for a ZTD G1 in a Z420 or Z620 is slot #4 (which is PCIe 3.0 with 8 electrical lanes).  Second best slot per HP is #5.  The XP941 can only use 2 of slot 4's 8 electrical lanes but the SM951 can use 4 of them.

As reported in the review, the SM951 AHCI controller also provides better thermal control technology so it runs cooler to reduce thermal throttling and wattage used.  Then, for the G2 ZTD drive, along came the NVMe-controller version of the SM951... even better and faster, but for the ZX40 family.

Only a small percentage of ZTD G1 drives came with a SM951 AHCI M.2 stick installed.  And that is why I hunt them down.

[DGroves]

just a quick bit of info, i've had the chance to test the xp941/sm951 AHCI/SM951 NVME in the same asus x99 deluxe motherboard and on a z820 (using Duet/Refind for nvme boot) under windows 10 and benchmarked using several popular ssd testing programs

the result,.................the xp941 is about 60% faster than a SATA based SSD the AHCI 951 IS ALMOST THE EQUAL of the NVME 951 in most tests except for sequential xfer where the nvme sm951 was about 10-13% faster

bottom line,... the sm951 AHCI is still shockingly fast even when compared to a modern SSD like the Samsung 970 pro

and the intel 750/p3600/p3605 ssd's are in the same class speed wise as the AHCI sm951/NVME SM951 drives

if you can live with the samsung sm951 256/512 capacities' they are well worth buying (even the lenovo ones) if the price is acceptable to you