ProDesk 600 G3 stuck at HP Post screen when using 512GB Samsung 950 Pro nvme drive

[Cowprince]

I have  ProDesk 600 G3 that had a 500GB HDD and am attempting to upgrade to a 512GB Samsung 950 Pro nvme drive from a second computer I have that I've just upgraded to a larger 970 evo drive.

The ProDesk 600 G3 functions just fine. I attempted to install Windows on the new Samsung drive, while keeping the old HDD in there for storage. The installation seemed to happen, but the machine booted to the HDD, since I had not yet wiped the drive. In an effort to prioritize the M.2 I changed the boot order in the BIOS so that the M.2 drive would boot first. I think this is when things went bad.

Now whenever I have the M.2 drive installed, the machine hangs at the HP post screen. It'll register all boot keys like ESC, F2, F9, F10, etc... but regardless of what I select it just sits at the HP screen indefinitely (I've let it sit overnight). If I pull the M.2 drive, I can access the BIOS without issue.

For some reason the machine is now just hanging because there's an M.2 drive installed.

I've tried doing a residual power clear by holding the power button with nothing plugged in for awhile.
I've tried pulling the CMOS.
I've tried resetting the CMOS.
I've tried, using a bootable USB drive.
I've removed the M.2 drive and have done a BIOS update and then tried installing the M.2 again.

For some reason the M.2 drive is hanging the machine on boot now, I don't know exactly why, but I think it has to do with me changing the boot order. Unfortunately I can't change the boot order while the M.2 drive is installed since I can't make it into the BIOS config. But then also I can't change the boot order without the M.2 drive installed, because it doesn't appear in the boot order then.