Successful Low Cost Liquid Cooling in HP z620

[BambiBoomZ]

Hello,

Some time ago, I posted a question: Are the z420 and z620 liquid coolers the same part?

As I was assembling a new z620 to replace the office z420 (Xeon E5-1660 v2) and the idea was to use the E5-1680 v2 (8-core @ 3.0 /3.8Ghz) at the Intel rated 4.3GHz overclock,  it was apparent that liquid cooling would be necessary.

Custom liquid cooling solutions can be difficult to specify, expensive, and refined proprietary workstation designs such as the HP z-series are resistent to subversion. This is logical as overclocking and changing the cooling could affect reliability and longevity as well as potentially make the system noisy and use more energy.  Certainly, be especially careful of any upgrades of a system under warranty.

As the z620 motherboard and z420 appear to be the same board except for the 2nd CPU riser sockets on the z620 version, it occurred to me that a z420 liquid cooler might work in a z620.

I superimposed photos of the z620 standard fan/heatsink and the z420 liquid cooler:

And, from this I could see that the two coolers occupy more or less the same general envelope / volume.

I also tried to work out possible ways to use a third-party liquid cooler:

> which employs an extrnally- mounted radiator.

But, a dual 92mm fan version would cover the top PCIe slot cover.  I could see that a single-fan radiator would work, but I could not find a closed loop liquid cooler with a single 92mm fan. These are all 120mm = too wide.  

Here's a more feasible design using an external, extended plexi mounting for a single 120mm fan unit that doesn't obstruct the PCIe rear slot covers:

 

This is feasible as it would effecively extract from the entire rear panel openings in addition to the two 92mm case fans. Fabrication from 1/16" plexi should not be terribly difficult. The mounting panel is not shown in this model, which uses the same holes as the case mounting.

With the z420 liquid cooler, the question was as to whether the much wider radiator could project, in effect, downwards beyond the CPU shroud.

As the envelope /volume of the two coolers did appear to be the same re: the superimposed photos, I decided to give it a go:

The standard z620 fan/heatsink with the shroud removed:

The z420 liquid cooler installed:

Notice that the z420 cooler has a special small diameter fan - it's just visible to the upper right, below the radiator. This directs downward air flow to the chipset- which is the same location on the very similar z420 and z620 boards. A nice little enhancement.

With the shroud on:

The projection out the bottom of the shroud shows the reason that this will only work in a single processor system. The 2nd CPU riser can not be fitted.

Total installation time: removing the original fan/heatsink and installing the liquid cooler was about 30 minutes- including taking the photos.

So, the cooler does project though the shroud downward side. There is a cubistic black plastic guard that protects the 2nd CPU riser visible to the lower left of the radiator, but there is plenty of clearance.

I was concerned that the air intake side vertical support for the shroud might somewhat impede the air flow into the fan, but the CPU temperatures with all eight cores at 4.1GHz at idle were very encouraging:

I am still looking at ways to add a 92mm pull fan on the output side of the radiator, perhaps powered off the Molex 4-pin but until I start some heavy CPU rendering,  I won't know if it's necessary.

After some experimentation, using intel Extreme Tuning Utility, which had been very successful in April in the z420, I settled on running all cores of the E5-1680 v2 at 4.3GHz with the goal of having a Passmark Single Thread Mark of 2300:

And the magic single thread mark of 2339 is very good.  The average Passmark single Thread Mark for the i7-6700K- which is a 4-core, is 2349. The temperatures at the 4.3Ghz clock speed and with more voltage added (+250mV) for stability are still quite acceptable.  These temperatures were recorded as soon as possible after the above test:

This shows that during the Passmark test results posted above- by the way, the very long version, that the maximum Package temperature rose to a maximum of 63C and the maximum individual core temerature was 58C. This is quite comfortable for the E5-1680 v2, which is rated to 85C.  A similar rendering run on z420_2 all cores at 4.1GHz using the standard fan /heatsink would see a Package maximum of 74-78C with individual core maximums at 65-71C- uncomfortable for the E5-1660 v2 rated to 72C.

It may be my imagination, but it seems that the z420 liquid cooler is at least as quiet as the z620 fan/heatsink.

These clock speeds and temoerature results are not in the league of our Forum friend Brian1965's z620 whose Tower of Cooling is so very-well, -cool,  but in my variegated uses: 3D modeling, (architecture and industrial design), graphic design, simulation, and CPU rendering I think it's going to have very good performance and reliability.  This z620 in effect replaces two systems: the z420_2 and z620_1 which has 2X Xeon E5-2690 8C@ 2.9/3.8GHz for CPU rendering.  However, I discovered that the CPU rendering I use- VRay, is not epsecially good at running on two processors- the z420 E5-1660 v2 six core could be faster.

The final specification of z620_2:

HP z620_2 (2017) (Rev 1) > Xeon E5-1680 v2 (8-core@ 4.3GHz)  / z420 Liquid Cooling / 64GB DDR3-1866 ECC Reg / Quadro P2000 5GB / HP Z Turbo Drive M.2 256GB + Intel 730 480GB + Seagate Constellation ES.3 1TB / ASUS Essence STX PCIe sound card / 825W PSU /> Windows 7 Prof.’l 64-bit  > 2X Dell Ultrasharp U2715H  (2560 X 1440) / Logitech z2300 2.1 Sound
[ Passmark Rating = 6322 / CPU rating = 17178 / 2D = 852 / 3D= 9012 / Mem = 3032 / Disk = 14227 /  Single Thread Mark = 2339  [7.3.17]
[ Cinebench R15 = cb1214 (CPU) / 153 (Single Threaded) / 150.77 (OpenGL) MP Ratio 7.92x / Accuracy 99.6% ] 7.21.17

The z420 liquid cooler for $50 represents by far the best cost/ performance upgrade I've ever done.

This is also a request that HP consider a liquid cooling option for all the Xeon E5 z-series.  As the high core count + simultaneously high single-thread (= high clock speeds) performance race is on, there are some hot E5's in our future.  

Cheers,

BambiBoomZ

[MtothaJ]

---

@BambiBoomZ wrote:

Hello,

Some time ago, I posted a question: Are the z420 and z620 liquid coolers the same part?

@as I was assembling a new z620 to replace the office z420 (Xeon E5-1660 v2) and the idea was to use the E5-1680 v2 (8-core @ 3.0 /3.8Ghz) at the Intel rated 4.3GHz overclock,  it was apparent that liquid cooling would be necessary.

 <cut>

---

Congrats on the results and thanks for the write up.

For anyone with the Z620 the fact that you verified that the Z420 liquid cooler fits without any modifications will no doubt be very helpful. This is a very good and clean no-fuss solution. The small fan cooling the VRM is definietly a welcome addition, since the Z420 / Z620 boards have no heatsinks on these components and when overclocking they get very hot.

The only potential thing that I would be concerned about is liquid evaporation over time as is the case in all AIO type units - but should that occur there are probably some methods of refilling / bleeding the unit and in any case at that price you could equally well just get another unit.

As for the push pull, for the second fan, I am thinking the easiest thing to do would be to just get a second HP CPU fan, or use the one from your air cooler and rig it up to the CPU PWM connector, where the first fan is located. At the ghetto level you could cut the connector of the second fan, strip the wires from under the cable insulation and insert these into the cpu plug connector of the first fan - attaching the connector to the motherboard would keep the cables in place. Obviously for the second fan you would just use the GND, 12V and PWM cables, the Tach signal would come from the first fan and since they are the same this would ensure proper PWM operation.

The more pressing problem is mounting the second fan, since from what I recall the HP liquid cooler does not have any fan mount holes on the other side of the rad.

[BambiBoomZ]

MtothaJ,

I appreciate the comments and suggestions. 

My idea of adding a second fan has been along the lines you mentioned, to make a splitter/ adapter for the 5-pin pump / fan  motherboard connection out of the connector of the original fan /heatsink and mount the original CPU fan on a fabriated sheet metal,  U-shaped bracket. The bracket makes a kind of clip  under and over the radiator and has mounting holes for a CPU or case fan.  The problems with this scheme is in making the brackets accurately enough in a metal gauge heavy enough for theaded mounting points. plus it needs to work on a correct spring tension- I can't drill into the radiator and don't want to solder to it.  This kind of fabrication is difficult to achieve at home. I am also concerned at the power limits of the motherboard connector and whether splitting a PWM connection voltage will affect that control. I tried raising the cooler speed with the BIOS Thermal setting, but the liquid cooler went into RPM hunting- noisily speeding and slowing. This is the problem with modifications, each change upsets a hidden layer of design parameters.  

I'm not aware of the possibility of coolant evaporation in a closed loop system, but that is certainly worth looking into.

I am still interested in that improving the thermal effieciency  as the cooling is the limitation on clock speed.  Our friend Brian1965 had spectacular results of a stable 4.7GHz - the highest clock speed of any HP z-series I've read, but required a cooling tower taller than the z620, drawn in Solidworks, and fabricated with a 2000W laser.  Fantastic work, and the cooling capacity allowed a flirtation with 4.8GHz. 

With my project, using the z420 liquid cooIer did allow a usable 4.5GHz but when running large test VRay renderings it was pushing a sustained 80C on several cores. Eventually, the 4.5GHz proved unstable when running multiple 3D CAD and photo editing programs simulataneously. Running at 80C+ is not impossible for the E5-1680 v2 which is rated to 85C , but I'm a believer in testing the limits and taking a step back - 4.3GHz is enough, epecially with a CPU that cost more than $1,700 new.  Anyway, in a number of applications, the E5-1680 v2 runs at it's standard all-core speed of 3.4Ghz.  The all-core speed limitation is the reason that the z420 E5-1660 v2 could run smaller renderings faster as it's all-core clock speed is 3.9GHz. the all-core speed is the reason that I considered moving the z420 E5-1660 v2 into the z620 as I ran it for a good while at 4.2 GHz on the standard fan/heatsink. With the liquid cooler, it may be possible to run it 4.4 or 4.5GHz and have a single thread rating over 2400.

Again, thank you for you interest in the project. This would not have happened at all if you hadn't mentioned in April the application of the Extreme Tuning Utility to some Xeon E5-1600 v2 series CPU's on proprietray systems.

BambiBoomZ

[MtothaJ]

---

@BambiBoomZ wrote:

MtothaJ,

I appreciate the comments and suggestions. 

My idea of adding a second fan has been along the lines you mentioned, to make a splitter/ adapter for the 5-pin pump / fan  motherboard connection out of the connector of the original fan /heatsink and mount the original CPU fan on a fabriated sheet metal,  U-shaped bracket. The bracket makes a kind of clip  under and over the radiator and has mounting holes for a CPU or case fan.  The problems with this scheme is in making the brackets accurately enough in a metal gauge heavy enough for theaded mounting points. plus it needs to work on a correct spring tension- I can't drill into the radiator and don't want to solder to it.  This kind of fabrication is difficult to achieve at home. I am also concerned at the power limits of the motherboard connector and whether splitting a PWM connection voltage will affect that control. I tried raising the cooler speed with the BIOS Thermal setting, but the liquid cooler went into RPM hunting- noisily speeding and slowing. This is the problem with modifications, each change upsets a hidden layer of design parameters.  

I'm not aware of the possibility of coolant evaporation in a closed loop system, but that is certainly worth looking into.

I am still interested in that improving the thermal effieciency  as the cooling is the limitation on clock speed.  Our friend Brian1965 had spectacular results of a stable 4.7GHz - the highest clock speed of any HP z-series I've read, but required a cooling tower taller than the z620, drawn in Solidworks, and fabricated with a 2000W laser.  Fantastic work, and the cooling capacity allowed a flirtation with 4.8GHz. 

With my project, using the z420 liquid cooIer did allow a usable 4.5GHz but when running large test VRay renderings it was pushing a sustained 80C on several cores. Eventually, the 4.5GHz proved unstable when running multiple 3D CAD and photo editing programs simulataneously. Running at 80C+ is not impossible for the E5-1680 v2 which is rated to 85C , but I'm a believer in testing the limits and taking a step back - 4.3GHz is enough, epecially with a CPU that cost more than $1,700 new.  Anyway, in a number of applications, the E5-1680 v2 runs at it's standard all-core speed of 3.4Ghz.  The all-core speed limitation is the reason that the z420 E5-1660 v2 could run smaller renderings faster as it's all-core clock speed is 3.9GHz. the all-core speed is the reason that I considered moving the z420 E5-1660 v2 into the z620 as I ran it for a good while at 4.2 GHz on the standard fan/heatsink. With the liquid cooler, it may be possible to run it 4.4 or 4.5GHz and have a single thread rating over 2400.

Again, thank you for you interest in the project. This would not have happened at all if you hadn't mentioned in April the application of the Extreme Tuning Utility to some Xeon E5-1600 v2 series CPU's on proprietray systems.

BambiBoomZ

---

For the fan bracket I think you are right that it will be difficult to devise a solution at home which will ensure a good fit (tight enough that it doesn't fall off, but not too tight so as not to cause vibrations) and which looks presentable / OEM like. The fan bracket idea, perhaps also adapting an exisitng one and bending / shaping it into place seems like a good one though.

For the PWM splitter, you could go for something like this: https://www.ekwb.com/shop/EK-PSS/EK-PSS-3831109867808.pdf

Obviously you do not need to split into 4 fan connectors, but the general idea here is that the PWM and Tach signals go from the motherboard, but the power is supplied by way of a molex connector. But I think this is more a peace of mind type thing - can't really imagine that hooking one additional small fan would cause much problems.

Regarding the HP liquid solution unit - do you know if the pump itself is PWM controlled (i.e. pumps faster / slower dependig on load) or is constant 12V DC with the PWM just controlling the speed of the fan?

@with the Z420 / 620 the limiting factor when overclocking is the 130W TDP limit imposed by the bios at startup. It still gives ample room for performance improvemnts especially when running applications which only challange 1-2 cores, but the moment one tries to run all cores at this speed in e.g. Prime95 the system will throttle down to stay within the above thermal envelope. Even my 1650v2 @4.4 on all cores in Prime95 under full load puts out c.a. 190W of heat, and in the case of the E5-1680v2 this would be even higher due to the extra 2 cores.

[BambiBoomZ]

MtothaJ,

Again, thank you for the comments.

I believe that the audible RPM hunting of the liquid cooler is that the PWM fan control is sending a varying voltage to, in effect, whatever unit is connected.  My concern was that if I split the PMW control, the current to both units would be halved and would never increase the current sufficiently to increase fan speed. That might even present an error message that the CPU fan was not running at nominal RPM. 

When it is only a fan, the RPM change is not audible except under extreme circumstances- and CPU rendering can produce that condition. When the liquid cooler is connected, the  pump is speeding and slowing audibly. It is doing that constantly  in the present conditions as I can see in HWMonitor, but unless the BIOS Thermal  setting is increased by an asterisk or two, it's not audible in the very quiet z620. 

This is the reason I had thought of an independent fan off the Molex that would run at a constant RPM.  I have a Thermaltake external USB-powered fan with a speed control which was very useful in evaluating a Tesla M2090 GPU compute unit. I'm trying to  set it up temporarily next to the radiator and exhuast through the rear panel.  One of the unexpected benifts of the Thermantake fan was that while cooling the passively cooled M2090, it also took quite a few degrees off the HP Z Turbo Drive as that has a very good thermal solution employing an exposed finned alloy heatsink.  Anyone have Samsung 950 Pros that are thermally throttling?  If the temporary fan improves the situation enough, I will pursue the bracket idea and run the fan with independent power supply and speed control.

BambiBoomZ

[bivoy]

Guys, excellent posts

Do I get it right - you can overclock z620 with Intel XTU?

[MtothaJ]

---

@BambiBoomZ wrote:

MtothaJ,

Again, thank you for the comments.

I believe that the audible RPM hunting of the liquid cooler is that the PWM fan control is sending a varying voltage to, in effect, whatever unit is connected.  My concern was that if I split the PMW control, the current to both units would be halved and would never increase the current sufficiently to increase fan speed. That might even present an error message that the CPU fan was not running at nominal RPM. 

When it is only a fan, the RPM change is not audible except under extreme circumstances- and CPU rendering can produce that condition. When the liquid cooler is connected, the  pump is speeding and slowing audibly. It is doing that constantly  in the present conditions as I can see in HWMonitor, but unless the BIOS Thermal  setting is increased by an asterisk or two, it's not audible in the very quiet z620. 

This is the reason I had thought of an independent fan off the Molex that would run at a constant RPM.  I have a Thermaltake external USB-powered fan with a speed control which was very useful in evaluating a Tesla M2090 GPU compute unit. I'm trying to  set it up temporarily next to the radiator and exhuast through the rear panel.  One of the unexpected benifts of the Thermantake fan was that while cooling the passively cooled M2090, it also took quite a few degrees off the HP Z Turbo Drive as that has a very good thermal solution employing an exposed finned alloy heatsink.  Anyone have Samsung 950 Pros that are thermally throttling?  If the temporary fan improves the situation enough, I will pursue the bracket idea and run the fan with independent power supply and speed control.

BambiBoomZ

---

BambiBoomZ,

Usually there are no problems in splitting PWM fans so long as the amps rating of the motherboard connector is not exceeded (I do not recall I have seen these values published for HP Z420/620i). But the fact is that at the moment you do have quite a few things hooked up to that connector - water pump, integrated VRM fan and the radiator fan so it may make sense to just use a DC fan connected somewhere else, especially since you already have the fan.

WIth regard to more precise fan control I have read that SpeedFan is pretty good in this regard but have not tried it myself so far.

I have the Samsung 950 Pro, but as I am not an intensive drive user I can't really comment on the thermal throttling. But I do recall that the matter has been brought up on various occassions. At one point I did have m.2 to PCIe adapter for the drive with a small fan, but I have since got rid of it, since I didn't notice any improvements in performance and the small fan was pretty noisy.

Bivoy,

That is correct, but this is limited to a small number of E5 v2 CPU's with unlocked multipliers - so far the following are confirmed to work: E5-1650v2, E5-1660v2, E5-1680v2.

Since they are all v2 CPU's you need a 2013 boot block machine to run them.

[Javato]

Hi BambiBoomZ, saw this post after I did something "similar" yet not as appealing as the drawings you posted up here...

  Have a look if interested! https://h30434.www3.hp.com/t5/Business-PCs-Workstations-and-Point-of-Sale-Systems/3rd-party-liquid-c...

  It is definitely not as elegant as what you had in mind, but somewhat similar... Thought you would like to have a look...

Cheers,

Javier.

[Daaavooo]

Hi !

how did you manage to OC that CPU ? as far as i know, e5 xeons are not OC and also OC on HP board ?! Sounds amazing tho...

I was trying Intel XTU also but no luck on mine E5-2697v2 ...