Z420 CPU fan not increasing under load

[Kamzeride]

Hello,

I have a HP Z420 Workstation (v2 with the Ivy Bridge boot block) with a Xeon 1650 v2 CPU, 48 GB RAM, GTX 1070 Ti, BIOS 03.96 running Windows 10 Pro build 19041.450 in UEFI mode and have noticed that when the processor is under any sort of load, the CPU fans are not ramping up to compensate whatsoever which causes uncomfortably/dangerously hot temperatures at the CPU's cores or overall package (89*C+, where the CPU has a Tcase rating of 70*C and Tjmax cut off at 95*C). I've ensured that the heatsink and fans are all free of dust and debris, and recently installed the missing front fan into the bottom of the chassis as well to improve airflow throughout the chassis.

This behavior can be observed by running HP Performance Advisor and refreshing the sensors while running a stress test such as Prime95, AIDA64 Stability Test, etc. You would expect to see the fan speeds increase, however they just remain where they are -- the fan speeds can be increased only across the entire system by cranking up the Idle Fan Speed under Thermal in the BIOS, however after the half way mark this becomes unbearably loud. I've ensured that the ambient temperature sensor in the front of the case is functional and it registers correctly.

Does anyone have any suggestions as to what may be causing this? I have seen some previous reports that this was related to Intel's MEI and AMT which I re-enabled to no avail, and wanted to check with the community before I pull the machine out to do a CMOS reset, flash back through BIOS revisions, etc.

Thanks! 🙂

[Kamzeride]

After some further troubleshooting, I believe I've narrowed down the cause of this with some interesting and concerning results. Can anyone else confirm these findings? 

BIOS versions 03.91 through 03.94: when running the CPU at 100% load with AIDA64 stability test or another stress test suite, the clock speed/voltage shifts down when the processor hits the 130W max rated TDP, and the CPU fan spins up when the processor hits 80*C, peaking briefly around 82-83*C but never exceeding that (with Idle Fan Speed set to its lowest setting in order to observe fan speed increase in HP Performance Advisor).

BIOS versions 03.95 and 03.96: Performing the same stress test shows that the processor's voltage and clock speed DO NOT shift down when the processor hits 130W TDP, quickly exceeding the thermal threshold without the fan properly ramping at 80*C to cool it. This issue becomes much more apparent if you choose to overclock the processor using Intel's Extreme Tuning Utility due to the increased voltage that is being applied, however I can confirm that this behavior is also present when the system is not overclocked, however it takes a little while longer for it to become apparent.

Example:

03.95/03.96: CPU set to 42x multiplier in XTU for all 6 cores registers 1.336V in HWMonitor with a clock speed of 4190 MHz. When running AIDA64 and the CPU hits 100% load, the clock speed will remain at 4190 MHz on all 6 cores up to 1.356V, and the temperatures quickly exceed 80*C until the system shuts itself off.

03.94 and below: CPU set to 42x multiplier in XTU for all 6 cores registers 1.336V in HWMonitor with a clock speed of 4190 MHz. When running AIDA64 and the CPU hits 100% load, the clock speed almost immediately drops to 3790 MHz across all 6 cores with the voltage at 1.211V and the fans ramp up at 80*C, holding the temperature steady within 2-3*C of that mark.

As per the release notes for the last two BIOS revisions (listed below), did Intel's microcode update or something in HP's BIOS change how the motherboard handles it's voltage regulation / power control? For those who are pushing their systems with heavy workloads and for those who have overclocked their machines (as many have begun to do), this new behavior is especially concerning as this will lead to a shortened lifespan of an already aging system.

Release notes for affected BIOS revisions:

03.96:
Fix and enhancements:

-Includes enhancements to mitigate security vulnerabilities CVE-2019-0124, CVE-2019-0151, and CVE-2019-0184.
-HP strongly recommends promptly transitioning to this updated BIOS version.
-Updates Intel TXT BIOS ACM to v2.4.
-Updates Intel TXT SINIT ACM to v2.5.

03.95:
Fix and enhancements:

- Updates Intel processor microcode to 0x42E for Ivy Bridge-EP processors, 0x718 for Sandy Bridge-EP C2/M1 processors and 0x61F for Sandy Bridge-EP C1/M0 processors.
- Includes enhancement to mitigate CVE-2018-12126, CVE-2018-12127, CVE-2018-12130 and CVE-2019-11091.

[Brian1965]

Quite an  interesting and informative post you've put together. I do overclock my E5-1680 v2 up to 4.7GHz in my Z620 but I'm still running BIOS version 3.91. After other forum members had reported a slight slow down in system performance as a result of upgrading the BIOS to v3.92, I decided to stick with v3.91.

[Kamzeride]

Hi Brian1965, thanks for replying. I believe the performance penalty is likely caused by the Spectre/Meltdown patches which were bundled with BIOS revisions after 03.91. I'm currently running 03.94 as there was a patch mentioned to prevent boot issues with Windows 10 which I am running on this system instead of Windows 7.

On the topic of the Z420 and cooling/overclocking: I have added the front intake fan which my unit did not come with, and also have both the front memory airflow shroud and a Z440 CPU heatsink on order as with the system OC'd it does tend to run a bit hotter than I'd like. I did apply some new Noctua NT-H2 paste today which dropped the temperature under load by almost 8-10*C so that's an improvement for the meantime -- however I'm wondering now if the airflow shroud and the larger heatsink will still fit together or if there won't be enough clearance? I didn't want to go with Z420's liquid cooler as I tend to stick with air cooling and the age/cost of available parts.

[SDH]

Kamzeride,

The pic below shows that you will be fine with your proposed approach.  This is a Z420 v2 with a E5-1660 v2 under the larger than stock unmodified HP Z440 "Performance" heatsink/fan.  These currently go for roughly $21.00 USD with shipping from eBay, and search for749554-001 there.  For others:  this also has the optional Z420 airflow baffle (SP P/N 663346-001 = AS P/N 663070-001), and the stock rear case fan, and the stock rear memory bank fan.  All fit in nicely together, as does the front bottom optional case fan can be seen.  Its optional front black plastic fan holder/PCI card guide is the same as used on the Z400, and thus one may find these parts under more numbers than usual:  

Z400:

619579-001 SP P/N (HP called this "Cooling fan kit with front guard & fan")
604781-001 AS P/N for the black plastic part
Z420:

684024-001 SP P/N (HP called the whole kit "Plastic front card Guide w/fan holder")
604781-003 AS P/N for the black plastic part
647113-001 = fan (this Nidec fan in the Z420 runs at 629 RPM.... excellent

The red cable you see is to add eSATA II out onto the backplane.  There are only 2 SATA III ports in the Z420/Z620... I reserve one for the boot/apps SSD and the second can be used for a documents SSD (not yet installed), or the CD (waste of speed potential in my mind).  I could get eSATA III via the second one if I needed.  The HP "2x2" Texas Instruments chipset USB3 PCIe card there beneath the video card is present to feed up to a front HP USB3 SD-card reader/writer via the USB3 "motherboard-type" header built in to that card.  It also adds 2 more full-power USB3 ports on the backplane and uses the same chipset as the Z420 motherboard so no added drivers needed.  This was a build for my better half... but hers has since morphed into a Z620 v2 project using all the same parts plus one of the nice Z620 memory dual memory fan "saddles" that perfectly fits perfectly over that big HP Z440 heatsink in a Z620 single-processor build.

I used to think I needed to swap into the Z440 heatsink a Performance fan from a Z420/Z620 heatsink, but then realized that you don't need to do that.  The Z440 heatsink's fan plug has 6 holes instead of 5, but the last two are just a ground-ground jumper (the Z440 motherboard wants to see ground on pin 6).  The Z420/Z620 do not need to use that extra hole so just hang it off the side of the Z420/Z620 motherboard's 5-pin header, out in space.  Works fine.

The pinout for a v3 or v4 processor does not match the socket for a v1 or v2 processor, but the heatsink mounting holes and spacings are all cross compatible between the ZX20 and ZX40 sockets.  The larger Performance Z440 heatsinks have almost exactly 2x the cooling fin area and 3 instead of 2 heat tubes when compared to the Performance heatsink for a Z420.

I've been dinking with Throttlestop a bit, but need to create some time to use Brian's great guidance on getting XTU to run properly, but under W10.  I am forced to use W10 over W7 now for security issues at work.  You may have seen my post here on turning off "Fast Startup" in W10 to help XTU work properly... in another post.  That has helped us save use of literally several $100k USD of medical monitors that need brightness/DICOM control software to work under W10, using software built for W7.  I got a free latte for that discovery.  By turning off Fast Startup under W10 all work perfectly now under their W7-written control software.  And, XTU does too per the post you can find here from the recent past.

Here is your pic with the big Z440 heatsink/fan in place.  Visible when a moderator releases it:

[Kamzeride]

Hi SDH, thanks for replying and confirming that the larger heatsink and airflow baffle will fit together, hopefully this will help keep the system a bit cooler overall. I'm hoping to squeeze a few more years out of this machine before having to drop the cash on a new one (which at this point looks like it might be a used Z440 but time will tell) -- these Zx20 series machines are still solid performers especially for their age, and at this point the power supply is really what's holding me back in upgradability on the graphics side.

I've been playing around with ThrottleStop as well in conjunction with XTU, and so far have managed to dial in a configuration that I'm happy with to achieve a "power efficient" overclock which also keeps the temperatures low when idle.

From my experience so far, XTU 6.5.1.321 seems to be the most solid version to use with Windows 10 and BIOS rev. 03.94 with Fast Startup disabled as you've mentioned, otherwise XTU will reset the multiplier values at random on startup. I've also enabled the Ultimate Performance power plan which keeps Windows from parking CPU cores to achieve lower latency and then have ThrottleStop doing the rest of the work.

For what it's worth, here's a quick run-down of how I did this on my system:

1. Set the power profile to either High Performance or Ultimate Performance so that Windows does not control the processor's clock speed / SpeedStep functionality.
2. Install XTU, reboot, and apply the maximum stable multiplier for all cores. Reboot again and ensure that the multiplier settings stuck correctly (CPU-Z or HWMonitor works fine here).
3. Install ThrottleStop and configure your profiles. For my every-day usage profile I have used these settings, adjust accordingly:
   1. Check Set Multiplier and set it to the maximum turbo ratio value.
   2. Check Power Saver to enable the CPU to throttle back down to minimum clock speed based on load.
   3. Click TRL and set the multiplier per active cores as desired. On this profile I was a bit more conservative, with a multiplier value of 43 for up to 3 cores and 42, 41, 40 for cores 4-6 respectively).
   4. Click Options and set PowerSaver C0% to 40 (default is 35). Note: This is completely subjective based on your workloads and how fast you want the processor to ramp it's clock speed up/down.
   5. Also check Start Minimized and Minimize on Close so that ThrottleStop's UI is silent on boot and that it remains running in the background, then hit OK to close the Options dialog.
   6. Don't forget to click Save in the main ThrottleStop window so that the settings are applied when it runs.
4. Open the Windows Task Scheduler and create a new entry for ThrottleStop to run on startup:
   1. Name: ThrottleStop
   2. Actions: Start a program > Path to ThrottleStop.exe.
   3. Trigger: At log on > At log on of any user.
   4. Ensure that Run with highest privileges option is checked.
5. Reboot again and ensure that your settings are correctly applied on boot.

[SDH]

That's interesting.... you seem to be using a hybrid approach.  Brian posted his HowTo a while back.  Do you first install that XTU and set it with his recommendations, and thereafter install the latest stable Throttlestop and apply your recommendations for it, running both?  Brian is still using W7 as is Bambi.

I'm very interested in your results with that Z440 heatsink/fan because I think it might get us close to water cooling.  Brian built a custom water cooling tower and so far he seems to be the champ.

[Kamzeride]

The latest compatible versions of XTU with Windows 10 (which do not cause immediate BSOD's) on the Z420 do not fully work as intended from what I can tell; for example the Additional Turbo Voltage slider is set to V (volts) instead of mV (millivolts) as it was in Windows 7 based on what I've seen from Brian's guides, limiting how much you're going to be able to push the system while maintaining stability.

In my approach, XTU is only used to unlock the maximum multiplier value so that ThrottleStop can do the rest of the work. Without XTU setting a higher turbo multiplier beyond the stock value in the TRL dialog in ThrottleStop has absolutely no effect, and even though "Set Multiplier" is checked and set to let's say 43, the processor will still max out at 39. I presume that XTU must be capable of writing to the MSR register for the highest allowed multiplier whereas ThrottleStop can not due to security limitations in Windows and the low level system driver which XTU installs (iocbios2.sys).

For documentations sake on the original post: I've noticed one change between the microcode versions from BIOS 03.94 and 03.95 which may explain some of the temperature differences -- the Tjmax value for the CPU is reported as 90*C on the older versions vs 95*C on the newer versions, I assume this is likely also having an effect on when the CPU is supposed to throttle due to TDP and when the fan is supposed to be ramping up (which was 80*C on older versions at least based on what I've seen here on the forums, and have noted a 5*C difference between HP Performance Advisor's CPU temperature reading and HWMonitor, Open Hardware Monitor, and Core Temp) as well across all versions tested.

[SDH]

For those who have not heard of the Ultimate Performance Power Plan HERE  is a nice quick HowTo article:

https://betanews.com/2019/04/01/windows-10-ultimate-performance-power-plan/

Maybe even better,  HERE .

https://www.howtogeek.com/368781/how-to-enable-ultimate-performance-power-plan-in-windows-10/

[Kamzeride]

To add to notes for the original post on different behavior between BIOS 03.94 and 03.96 with the Z420, I tried the latest version once more with another CMOS reset to see what would happen out of curiosity and ensured that Fan Idle Speed was set to the lowest value to observe fan behavior.

In addition to the Tjmax value having been adjusted from 90*C to 95*C for the Xeon 1650 v2 in 03.96, the CPU fan now waits until the reading hits 85*C before ramping up to cool the processor off instead of the previous value of 80*C. Interestingly enough the reading in HP Performance Advisor now also matches what other software reports for the CPU Package temperature instead of being approximately 5*C higher which was the behavior in 03.94 and below.

Something else to note is that pushing the CPU with a stress test such as MSI Kombustor's CPU burner or Prime95 will only cause the clock speeds and voltage to throttle slightly after hitting the 85*C mark (4290 MHz to 4080 MHz from my brief testing), whereas previously mentioned 03.94 will throttle once it hits 80*C and pushes clock speeds much further down to 3790 MHz (with a 43x multiplier set, with this CPU at stock clocks this drops to 3490 Mhz).

Overall for overclocking, at this time 03.94 appears to be the best choice for stability and my system is currently rock solid with that revision with a 43x multiplier overclock using the methods I posted earlier. However this situation is a bit conflicting, as 03.96 should in theory allow the system to perform better with higher clock speeds under load with less throttling involved as observed, but the system is less stable under 03.96 than 03.94 with the same values likely due to a brief peak to Tjmax which causes the system to power down and back on in order to protect the processor from heat damage. I'll test this theory again once I have the larger heatsink from the Z440 installed to see if the shutdowns are indeed caused by the thermals or if it might be something else.