Both Intel Lan Adapters bricked after update to BIOS 03.96 Rev.A

[Z820]

Recently I decided to upgrade my Z820 from BIOS J63 v03.65 to the latest 03.96 Rev.A

The upgrade went well, but after rebooting, Windows complained about the NICs, both 82579LM and 82574L were unable to start with Code 10.

I thought it was a bad setting or something, therefore I turned off the system, erased the BIOS settings (yellow button on MOBO, 15 seconds, without power).

But after that, the adapters complained that the NVM of the NIC was corrupt.

I checked the BIOS info, and noticed the second NIC (not the LM one) had a modified MAC ADDRESS, that was odd! The BIOS update was not supposed to do that!

I downloaded the ME update, flashed it...

No luck. The NICs try to even boot using PXE, yet, they do not link, there is no lights, nothing!

Windows with error 10...

I downloaded Intel's BOOTUTIL, created a DOS USB pen drive, booted, used BOOTUTIL -up=combo -DEFCFG -ALL to no avail, the ROM section for the nics on this machine is not writable. But resetting the configuration with BOOTUTIL changed the MAC to the original one.

Yet, the NICs run the PXE boot procedure, yet, they detect no link at all! Of course, windows still shows a happy ERROR 10...

After trying even the linux version of BOOTUTIL, the NICs are unusable.

No link, nothing at all!

And they (sometimes) show the NVM corrupt message.

And no, ethtool can't save the NICs, since I do not have the right images or write access to the ROMS.

I ended up disabling them and adding a dual port PRO 1000 PCIe card.

I am writing this as a cautionary tale. And if anyone else know how to re-flash the NICs, that would be a great detail. Maybe someone smarter than me had the same problem and fixed it.

PS I downgraded the BIOS to 03.69 and still the same problem. NICs are still dead. I upgraded again to the latest BIOS anyway, but I kinda think wasting a PCIe slot for the NICs is a waste of slot... This should never happen.

[DGroves]

I maintained over 500 z820/840 systems before i retired and never ever had a bios update modify/corrupt/kill a onboard nic during a bios update

however i have see quite a few corrupted/non functional nics on these workstations where a HP lan firmware "SP" install/update was attempted on systems that had already been updated by HP when the system/motherboard was installed in a system or sent out as a warranty replacement IE- the update being applied was a earlier revision than what was already installed

nothing in the HP bios updates, reads or writes to the onboard nics firmware 

however the Intel Mgmt Engine Firmware updater can corrupt a mac address (and other settings) , and i have seen improper ME firmware corrupt onboard nics

and also cause a system restart every 30 min due to a corrupted "ME" install

last, never update the bios from within windows, always use a usb key "DOS" method or from within the bios itself

[Z820]

Sir.

I appreciate the response. I, myself, was very surprised on the outcome.

Certainly, never update a BIOS from windows, actually, the package for doing so, should be removed from the installer. Instead, an burnable ISO would be preferable.

Now, I read your segment on the update "SP" and I don't think that was the case, but might be... Who knows at this point?. The system was working flawlessly, until the update. Actually, I updated the AMT right after I detected the NICs were unusable, with the same results.

Now, How did I notice the error? Besides the obvious?

I took a photo of the BIOS information screen before and after. I always do this.

And I noticed something, very, very strange. A few Bytes were shifted around. How? Like this.

Before I had (on the second NIC, the 82574L):

12:23:45:67:89:00

After the BIOS update:

F0:12:23:45:67:89

The first octet was added, shifting the other octets to the right.

Weird, right?

This was not supposed to happen.

I know the BIOS update should never touch the NIC area, but I fail to find a different explanation.

As I mentioned before, the MACs were "restored"to the original factory defaults using Intel's BOOTUTIL, so, we can access the NIC, yet the NICs were still useless. I then tried to restore the BIOS again, this time using the BIOS method itself. Of course.

And? Again, the octets on the second NIC were changed again!

You might agree that this is got nothing to do with a "SP" update. But I could also agree, that if the Firmware is corrupt, it might also corrupt the MAC.

Now, one extra interesting point. BOOTUTIL shows both NICs with no Firmware, but it might be that the utility cannot deal with HP's implementation.

At this point, I consider the unit as working again because I added a new PCIe NIC.

My question would be.

Is there a tool to restore the Firmwares on both NICs available from HP? I'd like to have the onboard NICs working again.

I shall also add, Intel does not support 82579LM, they support a similar one. 82579LM PHY.

And, thank you for the response.

At the end, I might have had faulty hardware and the BIOS update triggered the problem.

If it ain't broken, do not fix it. Right?

Warm regards

[DGroves]

i agree with most of what you mention, and as i was not there during the bios update and have no knowledge of your setup/configuration i will not rule out that it was the trigger for your nic issues i do however still believe that there was something else in play that caused the nic's to fail but as you state there is no way to be sure at this point

FYI, i have a z820 board that a user tried to run the HP LSI sas/sata firmware updater on, and the logs show a successful install/update but upon reboot the onboard LSI port were non functional. the z820 bios still can access the LSI bios but no attached drives will register, rerunning the updater using the orig firmware file (which the update created) gives the same results, and like you we replaced the onboard controller with a better/faster add in card

[Z820]

Good day Sir.

At this point, maybe, if I get some spare time in the future, and if I get a similar Z820 system, I might use buspirate and flashrom and I might try to recover the NICs.

Right now, investing more time on solving the issue is not cost effective anymore, since the new NICs are way faster than the original ones.

Also, I refuse to ditch the system, even with new Ryzen processors and then some, since I upgraded my Z820 to run all the Operating Systems from NVMe 4Lanes PCIe cards. I am very, very happy with my setup.

And yes, it is possible to lead windows, linux, and more from NVMe drives using dumb adapters. And since we have a lot of PCIe slots on the Z820, this is a great system.

One thing that bothers me is that a lot of people talk about not being able to successfully update the BIOS when you use nVidia cards. Weird, I have dual nVidia cards and recently I updated (yet again) the BIOS with no issues. 1x Quadro P4000 and 1x Quadro K4000. Then again, maybe that is what caused the NICs to fail... Who know?

Thanks for the remarks.

Warm regards.