The truth about z620/z820 NVMe SSD boot

[41C]

Thanks for the tip. I was thinking of an HP P822, but I see that there is more than passing similarity between HP, DELL and Lenovo cards and Adaptec, who I think is probably the original manufacturer for all of them.

I'll look out for one of the suggested Adaptec cards

[41C]

I recently acquired a Z640 and can confirm that the 'BIOS' is full UEFI, rather than just 'EFI Enabled' as on the latest update for the Zx20 series. In short, any PCIe NMVe drive is recognised and usable as a boot disk.  The UEFI 'BIOS' has a completely different look  and interface from the Zx20 series. (On the Zx20 series, the NVMe drive requires a Boot ROM to be able to use it as a boot drive from the BIOS, although any PCIe NMVe drive will be usable from within Windows 10 - just not bootable. This limits choice of drive to a few examples, discussed in the earlier posts, that contain Boot ROMs). 

My conclusion is that if you don't have a Zx40 machine, adding a PCIe NMVe drive for booting is not worth the hassle, unless you happen to have  already one of the few Boot ROM compatible ones. Just add a standard SATA or SAS SSD and once booted the OS will have the drivers for any NVMe drive that you may want for storage. There is really no practical speed difference in booting between NMVe and SATA on these workstations.  There's plenty of space inside these machines for a SATA SSD boot disk.  Additionally, the NMVe cards take up a valuable PCIe 3 slot, which might be needed for something else.  Probably a better bet would be a 12gb/s RAID card (LSI 9361, £100 on eBay) with a 12gb/s SAS SSD (HGST, enterprise surplus, new, unused,  400GB drive  £150 on eBay, not much more than 6gb/s SATA)

As for the RAID cards - yes the HP enterprise cards (P420, P840 etc) can be made to work, by turning off their boot ROMs and configuring from Windows, but are not ideal and run very hot. I concur that the Adaptec or LSI cards are a better choice. If your machine has hot-plug connectors on the drive bays, then you also require a Mini SAS to 4-SATA cable, preferably with a male SATA end to join to exiting SATA type cables to the drive bays, to avoid re-wiring the back-plane connectors in the drive bays. HP part number for this is 483508-003.

[DGroves]

i agree with just about everything in 41c's latest post,..............the only thing i might differ on is that if you like to tinker with or actually need the speed increase a high performance nvme drive delivers you might want to consider the DUET/REFIND method of soft loading the missing nvme boot code on the zX20

for the zX00 systems, a SATA boot SSD using a adaptec ASR-6805T 6GBps card with the optional cache/bbu  module and the HP mini sas to SATA male cable  483508-003 along with a sff-8087 to female "Forward type" cable is the way to go on these systems

[checlever]

Hi @MR_Raven ,

I bought an HP EX920 1TB M.2 drive and am using the HP Z Turbo Drive (G1 and G2) adapter but I can't boot to it. Did you have to make any other changes to the BIOS or install any specific drivers?

Thanks in advanced

[DGroves]

READ THE FULL THREAD, your not going to be able to boot off of any current consumer nvme drive, the out of production 950 pro is one of the few consumer (and data center grade) ssd's that had the missing nvme boot code on the ssd's firmware

your only option for booting a 970 is to use either clover, or DUET/REFIND bootloaders

the z820 bios does not nor will it ever have official HP support for booting nvme, that feature was added to the z840 and newer systems

[Mr_Raven]

---

@41C wrote:

This week I received a 960GB, Kingston KC1000 NVMe drive. I installed it in both a Z420 and Z820 with the following same results:

Installed Win10 from USB. NVMe drive was recognised OK and install completed (Installed on fresh re-format of drive)

However, on reboot, the drive was not found by the BIOS and hence NO BOOT.

This is without changing the settings that work with the Intel P3700 drive, and for that matter no other settings that I subsequently tried would get it to work.

---

What sector size is the drive formatted with?

As far as I remember the Kingston KC1000 does not work as a boot drive on a Mac Pro 5,1 which has NVMe support ieither f it is formatted with 4k sector size. It works  if formatted with 512k

With NVMe drives, sector size is important. On the zx20 not all drives will work with 4k sector size, others won't boot in 512byte mode. 

---

@41C wrote:

I therefore conclude that the latest BIOS versions of the Zx20 series may be UEFI compatible, but probably require to load a UEFI OpROM from the drive itself. If you recall, disabling the loading of OpROM from the P3700 drive would cause it not to BOOT. Either that or some NVMe drives, like the KC1000 don't contain the correct UEFI identifiers. This Kingston KC1000, which I tried, is a pro-level, MLC, drive from c. 2016, and I would guess it might well work as a boot drive in more recent systems than the Zx20.

It would be interesting to learn how the EFI implementation on the Zx20 series actually works, in terms of what generic drivers are built-in and what drivers it need to load from the drive's OpROM. It could even be that it only has built-in drivers for certain drive models, like the intel 750, P3xx0 etc.  I don't know.

Hence, all a bit disappointing, but on the bright side, I see that eBay seems to have a growing number of Intel P3520, P3600 and P3700; some of them new and unused, for reasonable money, comparable to a Samsung 970. 

---

The issue here is NVMe standard. The NVMe loader in the zx20 BIOS only supports NVMe 1.1, and that doesn't mean that drives supporting newer NVMe standards can't possibly work there have been various changes introduced at later NVMe versions which are not supported by the NVMe 1.1 BIOS, some of them (like dedicated boot partitions and the changes in how the drive communicates with the host which were introduced with NVMe 1.3) affecting how the drive boots. 

This problem is not specific to zx20 workstations, it also affects other systems with early generation NVMe boot support like the mentioned Apple Mac Pro 5,1 (which got NVMe support with the last firmware 144.0.0.0) or Dell's Generation 12 PowerEdge servers like the PowerEdge T320 (which got NVMe boot support in a later BIOS update). 

The general rule is that consumer drives are less likely to work than enterprise-class drives, and much more so when it comes to newer drives. 

[APeng]

@Weisswurstsepp thank you so much for sharing this information. I don't have a Z820, but i have a few friends who do and I think they're going to love this information. i came here to post another question and saw this thread and just read your post and some of the other posts in this thread. great stuff!

[Ussmillerco]

Ok, just to bring this full circle and understand what it is I need exactly to make my z820 ( have several) boot from a larger drive than my tiny current ssd drive (951 I think) which is constantly full. So as I understand, I can buy one of the intel 3600 1.2gb u.2 enterprise drives selling on eBay in my neck of the woods (Fremont ca. Which is like right over there...finger pointing) for $116 and change, and get a cheap u.2 to pcie adapter and pop it in to one of the slots (which slot is preferable?) and make a few bios setting changes (again which ones would be required), and then install windows 10.....and I would be in business, and bobs your uncle.....is that correct? Please direct me or correct me if I’m flawed in my understanding here , cause I’m ready to not have to worry about my scrawny boot drive ever again in my main computer. I’ll use these scrawny little drives in the other z820s I have, for a video server and/or a cctv security dvr. So will these u.2 drives work and what else do I need to do? 

PS. Absolutely thank you all for the updates, I used the earlier (several years back) info to get decent speed of the boot drive, albeit smaller drives, and these recent updates  using enterprise drives is such a leap forward. My compliments to those who made this possible. Thank you. 

[marsMAK]

Hi DGroves,

I can confirm that z820 do boot from my Samsung 970EVO Plus 500GB m2 NVME SSD finely. However, after I have mistakenly upgraded my BIOS ver. to v3.96 (the latest one. Previous working version 3.94), it can no longer do so - from BIOS settings I cannot even find this SSD.

I attempted to refresh its ver. back to a modified BIOS (got from website) via BIOS settings, but ended up with a failure - it says my BIOS file stored on a USB drive is either missing or corrupted - for the checksum matter maybe.

Now, other than Clover, is there any chance I could refresh my BIOS version back to whichever version that supports NVME? I tried to get BIOS such as v3.94 from internet but always no luck to find it.

Thank you in advance!

[DGroves]

as far as i'm aware there is no way to flash a modified bios on these workstations due to hp checksums, so short of programing the bios chip in a programer and then installing (soldering) it on your motherboard the answer is no

where/who modified the bios that you had?