The truth about z620/z820 NVMe SSD boot

[41C]

I became interested in this topic because I have a couple of worstations that I have been considering to upgrade to NVMe storage for the boot drive: A Z820 and a Z420.

Reading various articles, almost everyone mentions the Samsung 950 Pro drive with it's Bootloader for older BIOS that don't support EFI. The Samsung SM951 AHCI is also another option (just because it works in universally-supported AHCI mode, but still on the fast PCI bus), and there has been mention of the Intel 750, which was a more consumer or client targeted version of the Intel P3500, P3600 and P3700 family of enterprise drives.

Having read some reviews on the Intel 750 and P3700, it seemed that these may have contained a similar OROM facility like the samsung 950 Pro. which is why I went ahead and bought one. But re-reading those articles now, following the above reply, the wording is ambiguous. Anyway, my P3700 drive arrived in the post today and I installed it, with the instructions from HP Knowledgebase:

https://support.hp.com/gb-en/document/c03465686

What do you know? Windows 10 Installer found the drive, formatted it and now I have Win-10 booting from an Intel P3700 1.6 TB, NMVe PCIe drive on a Z420 machine. All in 10 minutes.

The only thing needed was to change the Advanced settings in the BIOS from Legacy to EFI. Addironally, all the previous SSD and HDDs in the machine continue to be recognised and work properly.

Now this support for EFI might be limited to the 'V2' versions of the HP Zx20 machines, whose BIOS can be updated to the more recent versions (3.96). In any case, I am not sure that the Sandy Bridge processors in the 'V1' machines could properly run PCI 3 making any installed NVMe drive slower than it could be.

The Crystal mark speed tests on the P3700 are obviously much faster than the previous SATA SSD, but with regards to Win 10 boot times, hard to notice any difference.

[Mr_Raven]

---

@41C wrote:

Most of the drives mentioned at the start of this post, such as the Intel P3700 and its contemporaries were Enterprise class drives (with high endurance ratings) launched at a time when NVMe was not fully established or widespread. Hence, like the Samsung 950 Pro and Intel 750, they contain Boot Option ROMs (OROMs), which provide the necessary NVMe driver for the BIOS.

It may well be that the most recent BIOS versions for the Zx20 machines now have the NVMe support built-in, but if the evidence is based only those older enterprise drives, it may be just that they work anyway because of their OROMs

---

That is incorrect. 

First of all, the Samsung 950 Pro is not an enterprise drive, it's a "prosumer" drive, i.e. a consumer grade drive for desktops/laptops aimed at "performance users", "enthusiasts" and workstation use. In fact, all Samsung "Pro" SSDs are consumer/prosumer grade drives, not enterprise drives. Also, the Samsung 950 Pro is quite unique with its CSM, and I am not aware of any other NVMe capable drive that offers a CSM.

Second, your idea that the P3700 was from a time when NVMe wasn't "fully established" has no relation to reality because when the intel Pleasantdale SSDs (P3500) came to market NVMe has already been a valid, well defined standard, and the P3700 you said was from the early days was actually already a product of intel's next NVMe generation (Fultondale), all which (unlike Samsung's "Pro" series) are true enterprise drives. And being enterprise drives, they were designed for use as storage, not as boot drives, and like other vendors' enterprise drives the use as boot drive isn't even supported (which doesn't mean it doesn't work, just that the manufacturer doesn't guarantee that it works). Which means there is no reason whatsoever why any of those drives should have a CSM or any other "boot ROM" replacing NVMe boot support in the system's BIOS, which is why they don't have one. So there is that. 

I can also say with certainty that none of the drives I tested had any CSM but in fact all were bog standard NVMe U.2 and M.2 drives. 

Clearly, there's a lot of FUD going round regarding NVMe boot support, which to weed out was the reason for starting this thread in the first place. 

[Mr_Raven]

---

@41C wrote:

and there has been mention of the Intel 750, which was a more consumer or client targeted version of the Intel P3500, P3600 and P3700 family of enterprise drives.

No, it's not. The Intel SSD 750 may look similar to a Pleasantdale (P3500) or Fultondale (P3600/P3700) drive, but these are actually completely different drives inside (the 750 is a "prosumer" drive for "enthusiast"/workstation use) than the P-Series enterprise drives. This is already hinted at the low endurance rating of some 100TBW for the SSD 750 and in excess o 1PBW for the P-Series. 

---

Having read some reviews on the Intel 750 and P3700, it seemed that these may have contained a similar OROM facility like the samsung 950 Pro. which is why I went ahead and bought one. But re-reading those articles now, following the above reply, the wording is ambiguous.

None of the P-Series drives have any CSM or "boot ROM". 

---

What do you know? Windows 10 Installer found the drive, formatted it and now I have Win-10 booting from an Intel P3700 1.6 TB, NMVe PCIe drive on a Z420 machine. All in 10 minutes.

The only thing needed was to change the Advanced settings in the BIOS from Legacy to EFI. Addironally, all the previous SSD and HDDs in the machine continue to be recognised and work properly.

Which isn't surprising, because your z420's BIOS has NVMe boot support. 

Now this support for EFI might be limited to the 'V2' versions of the HP Zx20 machines, whose BIOS can be updated to the more recent versions (3.96). In any case, I am not sure that the Sandy Bridge processors in the 'V1' machines could properly run PCI 3 making any installed NVMe drive slower than it could be.

NVMe support came with BIOS 3.84 which should be flashable on any version of zx20 machines. 

As to PCIe support, there is zero difference between Sandy Bridge (v1) and Ivy Bridge (v2) XEONs, which both fully support PCIe 3.0.

[DGroves]

i sent a query to micron support asking about the 9xxx series in regards to booting off of them and was referred to a pdf brochure. In it they state that the 9300/7300 DOES support booting from the SSD (m.2/PCI-E only, no U.2) the 9100 has no official support see the attached pdf's.  Support also stated the ssds themselves do contain the required nvme OROM boot code. This means that the ssd's themselves have all required files necessary for booting. support was unable to say if the ssd's could boot from a non UEFI based computer so perhaps someone could test this on a x58/z800 based computer

i actually own a Intel 400GB PCI-E  based SSD  and a micron p9100 ssd and they are NOT  BOOTABLE DEVICEs IN A Z820 

here's the intel spec sheet,.. note it states you must have a z97/x99 or newer chipset for the ssd to be bootable, these chipsets were the dividing line for official nvme boot support by intel and as such motherboard makers did not offer the necessary bios support for booting on earlier chipsets such as x79/C602 that the z620/820 use

https://www.intel.com/content/www/us/en/support/articles/000005797/memory-and-storage/legacy-client-...

as for the Intel p3700 i quired Intel and was told this:

the SSD DC P3700 will boot, however you must have  UEFI 2.3.1 or later the operating system must have native driver support. i have also been told that some systems/bios's also need to have the CSM disabled 

hmmmmm,......i may want to retry the intel 750 on the z820 with CSM (legacy bios) disabled perhaps it might then boot

when i get time i'll test this

so to recap, the z620/z820 bios DOES NOT CONTAIN NVME BOOT CODE, ..the mentioned 3700/9300 ssd's have the required boot code in their firmware just like the samsung 950 pro but it's not established yet if these ssd models can also boot from a non UEFI bios,  i personally suspect the answer is no and that for these systems you must still use the AHCI based ssd's

[41C]

The BIOS Advanced tab has the option of Legacy or EFI for the Boot O ROM, so it may be loading an EFI driver from the actual drive, as implied in a test-review of the P3700 drive. I do notice that when booting in EFI mode the usual 'legacy' BIOS-press F10 option is not shown and neither is the loading of the LSI SAS driver and display of the RAID, which normally comes first. There is, however, a 'Press CTRL C' to see boot options, which if executed jshows a what looks like a very slimmed-down version of the usual BIOS  screen. So, the Machine is definately booting in a different mode. As for the EFI driver for the NMVe drive, I don't really know if that is loaded from the drive or whether the BIOS contains generic versions. same must be true for the grpahics card and LSI SAS.

The slowish boot time was also noted in this review of the Intel 750 drive:

https://www.thessdreview.com/our-reviews/intel-750-pcie-ssd-review-400gb/5/

[41C]

Another thing that I tried:

I put back the original SATA C-drive, which when booting in EFI mode, comes up as J:/ , since it boots from the Nvme disk; however, if I return to the BIOS and set the Mass Storage Options ROM  back to 'Legacy'  (Press ESC on Boot  >> Enter Start-up Screen  >> choose 'change Settings' to return to conventional BIOS screen >> go to Advanced tab>> Device Options >> set Mass Storage from EFI back to Legacy), it then boots from the old SATA drive, although the new Nvme disk remains visible and usable in Win 10.

Also, even  if the Mass Storage Option ROM  is set back to EFI , but the Option ROM Download for the PCI slot containing the Nvme drive is disabled, then it will not boot from the Nvme drive, even though it will remain visible and usable in windows.

This implies that to boot from the NVMe drive, the EFI 'BIOS' must load some code from the drive, even if just an identifier to know what to do with it. However, the code is probably not the same as found in a Samsung 950 Pro, which evidently provides Legacy Option ROM style code for legacy booting. That might be why a requirement of a UEFI 2.3 -compliant 'BIOS' / Motherboard is specified for drives such as the Intel P3700 etc.

so to summarise:

Option ROM Download (for relevant PCI slot):  Enabled

Mass Storage Option ROM:  EFI

Result = boot from NVME drive

Option ROM Download (for relevant PCI slot): Enabled

Mass Storage Option ROM:  Legacy

Result = NO boot from NVME drive (boots from original SATA 'C' drive in PC)

Option ROM Download (for relevant PCI slot):  Disabled

Mass Storage Option ROM: EFI

Result = NO boot from NVME drive (boots from original SATA 'C' drive in PC)

However, to settle this for sure, I'll try to find some common M2 Nvme drive and try to install that as a boot drive and see what happens.

Also, I note there is mention by some people of disabling CSM (which I guess relates to the legacy O ROMs?) but there is no actual 'CSM' option in my 3.95 BIOS.

[Mr_Raven]

---

@DGroves wrote:

i sent a query to micron support asking about the 9xxx series in regards to booting off of them and was referred to a pdf brochure. In it they state that the 9300/7300 DOES support booting from the SSD (m.2/PCI-E only, no U.2) the 9100 has no official support see the attached pdf's. 

First of all, M.2 and U.2 are just physical formats, M.2 can be SATA or PCIe while U.2 is always PCIe. a PCIe M.2 doesn't look any different to the host than an U.2 disk. 

Also, you seem to confuse manufacturer support for a certain functionality with actual functionality. For enterprise SSDs, the M.2 format is quite new, and pretty much all manufacturers offer official boot support for them because in modern servers these drives are often used as boot drives. U.2 drives however are generally used as storage drives, not as boot drives, so there's no boot support. That doesn't mean these drives are incapable of being used as boot drives, it just means that Micron doesn't guarantee it. 

This is what Micron says about boot support in the 9200 series in the 9200 specifications :

"The 9200 is not intended to be a bootable device. Boot functionality is not validated by
Micron, and any use in this manner is done at the user's own risk. Please visit Micron.com to find other SSD products that are recommended for boot."

Micron 7300 series SSDs are nearline drives aimed at low to medium read-intensive workloads, and are available in M.2 and U.2 form factor. As you stated, only the M.2 variants have manufacturer support for use as boot drives, the U.2 variants do not. The distinction is merely from a support perspective, though, as aside from the physical format both variants are identical (M.2 and U.2 variants even share the same firmware image). 

Micron 9300 series SSDs are U.2 format only drives for which Micron does not support the use as boot drive (it still boots fine in the zx20, though). 

The Micron 9200 drive I use in my z620 and which I showed on the imgur link I posted earlier is also only available as U.2 drive, and like other U.2 drives and it, too, comes with no official boot support by its manufacturer. However, as demonstrated on the photos, the z620 boots form it just fine.

i actually own a Intel 400GB PCI-E  based SSD  and a micron p9100 ssd and they are NOT  BOOTABLE DEVICEs IN A Z820

The Micron 9100, both in U.2 and HHHL (PCIe card) format boot just fine in a zx20 with updated BIOS. Why it doesn't for you I don't know since you didn't give any details (for example, what firmware version is on the drive, how has the SSD been formatted, what is the partitioning, what OS etc; your other post suggests you're not fully understanding all the details that go into the boot process so it may well be that you just did something wrong). But it's definitely not the drive or the computer that is the problem. 

here's the intel spec sheet,.. note it states you must have a z97/x99 or newer chipset for the ssd to be bootable, these chipsets were the dividing line for official nvme boot support by intel and as such motherboard makers did not offer the necessary bios support for booting on earlier chipsets such as x79/C602 that the z620/820 use

https://www.intel.com/content/www/us/en/support/articles/000005797/memory-and-storage/legacy-client-...

Again, you mix up manufacturer support for a specific configuration and ability to work. Taking that statement literally, this SSD would have no chance to boot say in a modern AMD based system (which doesn't use intel chipsets), but it does. It's just not supported by intel. 

---

as for the Intel p3700 i quired Intel and was told this:

the SSD DC P3700 will boot, however you must have  UEFI 2.3.1 or later the operating system must have native driver support. i have also been told that some systems/bios's also need to have the CSM disabled 

This is correct, as the boot process relies on the UEFI BIOS' NVMe support, which requires that at least for storage devices legacy support (CSM support) is disabled (you can however leave CSM support for non-storage devices like video cards enabled).

---

so to recap, the z620/z820 bios DOES NOT CONTAIN NVME BOOT CODE, ..the mentioned 3700/9300 ssd's have the required boot code in their firmware just like the samsung 950 pro but it's not established yet if these ssd models can also boot from a non UEFI bios,  i personally suspect the answer is no and that for these systems you must still use the AHCI based ssd's

---

Your conclusions make no sense as is clearly based on speculation and not facts. You seem to have made up your mind and look for confirmation. 

I already demonstrated that a z620 with current BIOS boots fine from a single Micron 9200 U.2 NVMe drive, which according to the manufacturer is not supported as boot drive. 

The reality is that the z620/820 *does* have NVMe support, and while you're free to ignore it this won't change reality that most NVMe drives, no matter what format, can be used as boot drives on a z620 and z820 running the current BIOS. 

[Mr_Raven]

---

@41C wrote:

Another thing that I tried:

I put back the original SATA C-drive, which when booting in EFI mode, comes up as J:/ , since it boots from the Nvme disk; however, if I return to the BIOS and set the Mass Storage Options ROM  back to 'Legacy'  (Press ESC on Boot  >> Enter Start-up Screen  >> choose 'change Settings' to return to conventional BIOS screen >> go to Advanced tab>> Device Options >> set Mass Storage from EFI back to Legacy), it then boots from the old SATA drive, although the new Nvme disk remains visible and usable in Win 10.

Also, even  if the Mass Storage Option ROM  is set back to EFI , but the Option ROM Download for the PCI slot containing the Nvme drive is disabled, then it will not boot from the Nvme drive, even though it will remain visible and usable in windows.

This implies that to boot from the NVMe drive, the EFI 'BIOS' must load some code from the drive, even if just an identifier to know what to do with it. However, the code is probably not the same as found in a Samsung 950 Pro, which evidently provides Legacy Option ROM style code for legacy booting. That might be why a requirement of a UEFI 2.3 -compliant 'BIOS' / Motherboard is specified for drives such as the Intel P3700 etc.

This is not surprising.

First of all, if you load legacy support (CSM support) for storage devices then this prevents NVMe boot. 

Second, any changes to legacy support will very likely alter the boot sequence, so you have to make sure that the sequence is correct.

[41C]

No, maybe not surprising, but just illustrates or confirms that the P3700 drive that I installed is unlikely to contain any legacy-type OpROM code like the Samsung 950 Pro, since it will not boot in Legacy mode (but will in EFI mode with EFI OpROM upload enabled.) This supports the premise that the newest BIOS updates to the Zx20 series (including Z420) provide adequate support for booting from NMVe drives . I also understand that U.2 drives were not intended for workstations, particularly as it's not so straight forward to install them (adapter cables using m.2 interfaces are messy and the alternative of mounting directly onto PCI adapter cards, results in very crammed PCI slots).  Also, this particular P3700 drive has a max ambient temperature rating of 35C, which further limits the installation options. Nevertheless, I needed a fairly large C drive to hold  a lot of application software and this 1.6 TB U.2 drive came up with zero usage hours on its clock and at a price similar to a 1TB Samsung 970 evo. It also means that I no longer feel the need to upgrade to a Z840.

[41C]

This week I received a 960GB, Kingston KC1000 NVMe drive. I installed it in both a Z420 and Z820 with the following same results:

Installed Win10 from USB. NVMe drive was recognised OK and install completed (Installed on fresh re-format of drive)

However, on reboot, the drive was not found by the BIOS and hence NO BOOT.

This is without changing the settings that work with the Intel P3700 drive, and for that matter no other settings that I subsequently tried would get it to work.

I therefore conclude that the latest BIOS versions of the Zx20 series may be UEFI compatible, but probably require to load a UEFI OpROM from the drive itself. If you recall, disabling the loading of OpROM from the P3700 drive would cause it not to BOOT. Either that or some NVMe drives, like the KC1000 don't contain the correct UEFI identifiers. This Kingston KC1000, which I tried, is a pro-level, MLC, drive from c. 2016, and I would guess it might well work as a boot drive in more recent systems than the Zx20.

It would be interesting to learn how the EFI implementation on the Zx20 series actually works, in terms of what generic drivers are built-in and what drivers it need to load from the drive's OpROM. It could even be that it only has built-in drivers for certain drive models, like the intel 750, P3xx0 etc.  I don't know.

Hence, all a bit disappointing, but on the bright side, I see that eBay seems to have a growing number of Intel P3520, P3600 and P3700; some of them new and unused, for reasonable money, comparable to a Samsung 970.