Z620 booting from Intel and Samsung NVMe drives (with OPROMs)

[SDH]

That is an interesting Intel Data Center NVMe-controller SSD, with an x8 PCIe connector instead of x4. That card will need to be inserted into a spare x8 (or x16) PCIe slot to get benefit from its dual NVMe controllers. DGroves might know but I have not tested that in a HP ZX20 workstations myself.

I can tell you that it uses exactly the same latest firmware (8DV101H0) that Intel released 2/17 for their Data Center NVMe drives that I have posted about above. It also uses the exact same last "bootloader" revision (8B1B0133) which is included in the same updater software. That bootloader part is used to supplement the in-BIOS Z620 EFI (UEFI) boot code. The combination allows these NVMe drives to boot successfully in the ZX20 (Z420/Z620/Z820) family of workstations (v1 or v2). Thus, there is a high likelihood the techniques detailed earlier in this thread will also work for the P3608. The other Intel Data Center drives discussed above use only x4 true electrical lanes, so the x8 true electrical lanes used by your P3608 is a difference that I have not personally tested. That extra level of complexity is a risk I'd take if the price was right. However, see the first review linked below... there is an on-board PLX chip and info about potential for use of Intel RAID 0 software. 7/25 EDIT: Intel's use of a PLX chip on the P3608 PCIe card allowed the data I/O to aggregate across x8 electrical lanes in the ZX20 series of workstations. A later faster and more technically sophisticated way to use x8 or x16 electrical lanes is now used in later HP workstations, starting with the ZX40 and up. It is termed "bifucation" which is a BIOS feature allowing splitting the 8 or 16 true electrical lanes of a single PCIe slot into multiple unique sets of x4 true electrical lanes, which is all a modern NVMe drive needs.

x8 vs x4 connector

I've attached an Intel P3608 spec sheet and product brief, plus a Solidigm PDF showing that the same latest firmware/ bootloader versions are used with these, highlighted in yellow. See both the six "Data Center" and the one "Client" series  versions (the Intel 750).

Note that there are also OEM variants of these that have OEM firmware/bootloader code (which you don't want unless you have an alternative way to upgrade to the latest). I'd recommend only buying the Intel versions of these drives because you then have an easy access to firmware/bootloader upgrades via the Solidigm/Intel DC software. If you look at the history of upgrades for the firmware/bootloader versions they were modified by Intel quite a lot until they finally got things stabilized. You want the latest. You can tell if a card is an Intel or OEM type by seeing the firmware version on the top card edge label.

I've got to get around to adding another small update to this thread showing the exact commands used to upgrade if using the Intel DC software, and some added tidbits related to a P3608 I have access to, soon. I'll focus on single volume use rather than RAID 0.

EDIT: Two P3608 reviews HERE and HERE .

[DGroves]

the 3608 card is weird ware in that it is actually two equal sized  TB ssd's on the pci-e card hence the x8 requirement

I've seen these models with OEM firmware that makes the card into a raid 1 device. it was used for systems that required high availability (IE- auto failover/backup) for systems that always had to be working/available and they are quite fast

i'm guessing that there also is firmware that makes the ssd a raid 0 device, but i've never seen one myself.

i do know that they were sold with intel firmware besides the custom OEM vender specific model's i've seen

the card has the ability to boot in a windows system if the firmware allows it and i suspect the intel firmware is able to do so

BTW, the intel based sun/oracle 1.6GB dc3605 card was called the F160 by oracle

[Seascoot]

Adding my experience here with another Intel DC series NVMe PCIe3 x4 card with 1.6TB capacity.   I purchased this several months ago when early posts indicated that these cards were able to OS boot in the ZX20 series.  When I decided to install it just now I realized the fact that it was HP branded, and therefore firmware update to the latest version had no clear path.  I decided to try it anyway with the existing HPK3 version. 

 HP part # 803199-002 Intel DC P3600 card

I can report that it installed and booted into Windows 10 Pro using a procedure similar to that described by @SDH.  It was placed in slot 4 of a Z420 with a Xeon E5-2667 v2 cpu with 32GB DDR3, all partitions erased, GPT partitioned, and NTFS formatted.

The original SSD 500GB  boot disk was cloned using a Clonezilla boot DVD and expanded into the larger P3600 partition.  Upon removal of the original boot disk the DC P3600 booted with no problem.

Intel DC P3600 PCIe3 x4 card

After seeing that there is a similar series of Samsung PCIe3 x4 cards available, the PM1725 for example also with excellent speed and endurance ratings, I wonder if anyone has experience of using them as a boot drive in the ZX20s.

[SDH]

Seascoot,

EDITED 7/25:  Good news on your HP-branded HP-firmware P3600. I've never seen Intel mix and match the bootloader and firmware versions. Thus, your card probably has the latest HP firmware also ("4IFDHPK4") shown by your run of the ISDC Tool, documented in your post below. Now if we only knew how to use the HP firmware updater... I found a source for download that via the HPE website HERE.   DGroves may know how to use that.

Samsung surely knows how to add in missing "bootloader" code so that a server or workstation from the early NVMe days could harvest that from the drive very early during BIOS boot (to supplement the server/workstation's on-board BIOS boot code) to allow the combination of those two to allow successful boot from an NVMe drive. The question is whether Samsung added in that code onto the drive you mention. Samsung did do that for the 950 Pro M.2 SSD, as we know.

I'm sticking with the Intel P3700 for now and don't have access to one of your mentioned Samsung drives. These DC P3700s cards have such a high lifetime that I have not worried about buying them recycled from eBay. 

[Seascoot]

---

@SDH wrote:

Samsung knows how to add in missing "bootloader" code so that a server or workstation from the early NVMe days could harvest that off the drive very early during boot, to supplement the server/workstation's on-board BIOS boot code to allow the combination of those two sets of code to allow successful boot from the NVMe drive.

---

It was the prior discussions regarding the 950 Pro SSD that made me wonder about their Data Center drives.  I was not able to find very detailed specifications online on the PM1725 series.  Some sell sheets said they were not bootable, but that could have been just a general statement related to dependency upon a host workstation BIOS capability.

---

@SDH wrote:

Seascoot,

Good news on your HP-branded HP-firmware P3700. 

It was the DC P3600 - sibling of the P3700, but not quite as fast

PS C:\WINDOWS\system32> isdct show -intelssd

- Intel SSD DC P3600 Series CVMD642300HP1P6DGN -

Bootloader : 8B1B0133
DevicePath : \\\\.\\PHYSICALDRIVE0
DeviceStatus : Healthy
Firmware : 4IFDHPK4
FirmwareUpdateAvailable : No known update for SSD. If an update is expected, please contact your SSD Vendor representative about firmware update for this drive.
Index : 0
ModelNumber : LT1600KEXVB
ProductFamily : Intel SSD DC P3600 Series
SerialNumber : CVMD642300HP1P6DGN

This shows the bootloader to be 8B1B0133.  I believe that may have been the last version used.  Maybe why the drive does work with the Z420.

[SDH]

I'm going to add to this project to provide some added discoveries:

1. Seascoot, your HP P3600 has the HP firmware listed as "HPK3" on the top edge label in your picture and yet from your use of the Intel SSD Data Center Tool (ISDCT) the firmware is shown to actually be 4IFDHPK4 instead. I trust your use of the Intel SSD Data Center Tool with output you show above more than what is shown on the label ("HPK3"). Also, the Bootloader version shown above from your use of the tool ends with 133, which is the very last one for these drives (it was released 2/17, along with the last firmware). You can imagine HP might have updated the firmware and bootloader from what was originally printed on the top edge label. This is why I'd prefer to buy the sold-directly-by-Intel ones because the x64 SSD Data Center Tool that works best for me is still available and works well with W11 24H2... more on that below. Plus, the recycled HP drives on eBay seem to be more expensive and less available. Finally, while there is a similar HP firmware updater tool for the HP OEM NVMe drive versions but it is hard to find, comes from the HP Enterprise (HPE) company, and is firmware-engineered for HP servers.

2. Regarding Ivo_555's question earlier on use of a unique Intel DC P3608... I like DGroves' "weird ware" assessment quite a bit. My advice is to not buy one. The ISDCT x64 I use does work with it, and each separate controller for that 2-drives-in-one-card device show up when using the tool. One of those I probed here had one controller bad and one good, which whacked the card down to 200GB usable space. Normally, if the rare Intel NVMe RAID software they custom built for that card is not used the two segments show up as two different drives of equal size in W10/W11. There is a long painful thread on Intel's support site from some years back from a patient guy trying to obtain that special RAID software and some advice on how to use the card. Bottom line, no one at Intel support knew pretty much anything about these, and the old custom kludgy NVMe RAID software is long gone. Let's hope the newer Intel VROC approach to RAID with NVMe drives works better and lasts longer than that earlier try.

3. The Intel SSD Data Center Tool (ISDCT) v. 3.0.27 is still available in a pure form as a 64-bit OS type download, and it works great with W10Pro64 and W11Pro latest too. You install it as an administrator, it shows up in your Control Panel/Programs and Features app and is run via an elevated CMD prompt with several basic commands to get done what you need. There are some newer Intel and Solidigm tools that turn out to not work as well as it does. This ISDCT tool has identified firmware and bootloader updates that later Intel and Solidigm tools did not, and some of those newer tools report earlier firmware and bootloader versions as the final available when this  better tool identified and had available the true latest versions.  See Seascoot's command line interface use of the ISDCT above for examples. The commands are:

isdct show -intelssd    This probes for 1 or more attached drives and states the present bootloader and firmware versions. It will also tell you if a firmware and/or bootloader update is available. If two of these drives are present it will also give you an "Index" value for each... 0 and 1. If a drive controller is borked you'll see something like DeviceStatus : *ASSERT_403561B8 BG" and if good you'll see "DeviceStatus : Healthy". If a FW or bootloader update is available go to the next command below.

isdct load -intelssd 0   or   isdct load -intelssd 1   These will trigger a warning and ask if you want to proceed. Generally both the FW and the bootloader code will be updated in the same process and you'll end up with "Status : Firmware updated successfully. Please reboot the system." Reboot and you can reprobe to confirm the changes if you wish.

4. ISDCT 3.0.27 x64 Download: I only use this specific Softpedia download source option. Wait a little bit... it will start the download automatically. The PDF manuals from Intel are inside the zipped folder, plus the installer. After extraction you can check software's release date "details" under Digital Signatures tab... signed by Intel 11/4/2020. There is nothing newer for these drives, listed below, after this set of firmware and bootloader code (released 2/17).

https://drivers.softpedia.com/get/HDD-SSD-NAS-USB-Flash/Intel/Intel-SSD-Data-Center-Tool-3-0-27-64-b...

5. There is a whole series of Intel Data Center and Intel Client NVMe SSDs in the PCIe Add-In-Card and 2.5" U2 form factors that use the same last Intel Firmware and Bootloader versions. The bootloader part is the supplemental OPROM code that lets the ZX20 HP BIOS code work together to allow NVMe boot.  These are:

DC D3700   (U.2 2.5" form factor)

DC D3600   (same)

DC P3700   (AIC form factor)

DC P3608   (same but "weird ware" 2-in-1 AIC)

DC P3600   (AIC form factor)

DC P3500   (same)

Intel 750      (same, but Intel's "Client" version)

Each of these can use the same last Intel firmware version and same last Intel bootloader version that I have listed earlier multiple times during this project so it is fine to use the Solidigm Storage Tool like I initially did and got lucky with. However if that does not provide you the update to those latest versions then just use the ISDCT 3.0.27 x64 approach and that will get you to the correct latest ones every time in my experience (unless the hardware is borked).

6. Remember... It is key to first GPT partition these drives for all the knowledge above to work properly.

[SDH]

(ignore)

[SDH]

A small but valuable tip:

It is virtually impossible to find original full height Intel backplane brackets for the Intel DC P3700 type PCIe add in cards unless your card came with one. The shipping boxes came with the half height bracket attached to the card and the full height one was tucked into the box also but because these were usually used in half height servers the boxes and full height bracket were usually lost to time.

The new ones you find now on eBay and Amazon are made in China, are adequate, but often come with 2 screws that are too short and too large a metric size to use with the card. Below are pics of what won't and what will work. EDIT: The top picture shows an aftermarket backplane bracket, and the two screws are shortie M3 screws. The threads in the 2 little holes are also female M3 to fit. However the Intel DC brackets and screws are made for M2.5 x 6mm or 7mm long and those will just barely slip through the threaded holes shown. They can then screw into the M2.5 female threads built into the metal of the card's structure. So, you don't have to modify the bracket at all... you just need to get the right length and diameter metric screws.

Won't Work

Will Work

The thinner longer screws are needed... they need to freely cross through the two little threaded tabs of the bracket and if I recall correctly they are M2.5 metric. The receiving two nuts are built into the opposite side of the Intel PCIe card's aluminum frame. If you're lucky the seller will include the correct machine screws but I found them also in the metric section of our local Ace Hardware store, and McMaster.com is also a great resource for such items.

[Rhothgar]

Thank you for all your dedication on this, SDH and, of course, the others you mentioned over the years.

Going to take a dive into this with a cheap 256Gb Samsung 950 Pro M.2 SSD I have found just for starters and then maybe move up to the more expensive and seemingly hard to find DC P3700 potentially if I can drop on one at a fair price.

Like you say the lifetime is predicted to be ridiculously high so even used ones are not a concern to use.

[DGroves]

SDH's post listing the bootable intel datacenter/consumer ssd's as usual is excellent, however there was one intel model missed the P3500 it also shares the same oprom/boot code as the 3600/3700 and should also boot

HOWEVER AS I RECALL THE DC P3500 is not officially supported by intel as bootable for this series as it was replaced with newer SSD models before intel finished certifying it for this function 

https://www.intel.com/content/dam/www/public/us/en/documents/product-specifications/ssd-dc-p3500-ser...

quick update:

found a lenovo document that appears to confirm the P3500's ability to be a bootable device

page 5 specifically states the device is a bootable

https://download.lenovo.com/servers/mig/2016/07/19/5621/intc_dd_nvme_1.3.0.1007_win_x86-32-64.pdf