HP Z620 + Quadro K4200 + Tesla M2090

[Brian1965]

I have had some interest from a couple of forum members with regards to an upgrade project I have been working on which involves installing a Tesla M2090 GPU card in my HP Z620. I thought it may help to provide a little bit of background to give some context to this post;

Originally purchased my HP Z620 in March 2016 and it arrived with the following spec;

E5-2660 x1, 32GB (8GB x4) PC3-12800R ECC REG RAM, Quadro K4000 and 1TB 3.5” HDD

Since then I have added/upgraded the following;

2nd CPU riser and 2x Xeon E5-2670’s, replaced the Quadro K4000 with a K4200, added a Tesla M2090 GPU card, upgraded the RAM to 64GB (8x 8GB) PC3-14900R ECC REG, added a 2.5” + 3.5” Swappable ICY Dock (connected via an additional 2-Port SATA III PCIe card), and added a 256GB SM951 as a boot drive (via an ASUS PCIe card), and filled the drive bays with a 480GB SanDisk Extreme II SSD, 4TB and 2TB HDD’s.

Basically looking to max out the machines performance but within a sensible budget. I mainly use the Z620 at home for SolidWorks modelling and music production but my son started at University last year and is studying for a degree in 3D modelling and computer animation. Consequently, I was looking at ways of improving rendering speeds but shuddered at the price of second hand high end Quadro cards.

I've been working on a mod project to add a relatively cheap Tesla M2090 GPU card to my current HP Z620 workstation and thought it may be of interest to some forum members. Currently there are several Tesla M2090 on eBay selling for around the £70 – £80 mark. The main issue with the Tesla M2090 card is that it was never designed to be installed in a standard PC enclosure since it relies purely on forced air cooling to stop the GPU from overheating and shutting down. It’s a 225W card which generates a lot of heat. My plan was to use liquid cooling for maximum cooling efficiency and hence maximum performance.

Before I get to the build details, I thought some benchmarks may be of interest to help anyone decide if the hassle outweighs the benefits. First up is a render of a sample scene in SolidWorks Visualize;

Image size 1920 x 1080 pixels, 2500 passes

https://www.dropbox.com/s/fvbxuqwnymlbape/1969Camaro%2010.jpg?dl=0

The results;

Octanebench is another great benchmarking tool for measuring GPU performance. 

Benchmark scores;

From L-R - K4200 only (score=39.94), M2090 only (score=52.20), K4200 + M2090 (score=91.85)

For comparison, here’s a link to the results page; https://render.otoy.com/octanebench/results.php

So in summary, faster than a pair of Quadro K4200’s (score: 79), and the Quadro K6000 (score: 87). Checking on eBay tonight, the cheapest Quadro K6000 listed is £1250 + shipping!!!

So now the build project. This being my first ‘proper’ PC mod, (and my first hands-on encounter with liquid cooling), there was a certain degree of discovery involved during the project, (or in other words, I made it up as I went along but with a rough plan). I suppose in hindsight, there are a couple of minor tweaks I would make if I were to start again since there was a bit of confusion and miss-match regarding the tubing barb sizes. I should also caution that this mod did require some minor modification to the back of the Z620 chassis. Also, HP does not provide any support for the M2090 when installed in a standard desktop workstation and any modding is likely to invalidate your HP warranty.

First step was to remove the existing passive cooler on the Tesla M2090 and fit the custom Tesla active water cooling block and the 10mm barbs.

The next step was to mount the PWM fan temperature controller PCB board to the back of the Tesla card. Due to the short length of the thermocouple sensor, it was not possible to mount the PCB anywhere else. The controller board is secured in place using 4 layers of 3M car number plate tape attached to the back of the PCB. Another piece of tape was used to secure the thermocouple directly behind the GPU, (i.e. dead centre of old cooler mounting holes).

Now the brave part. Using a ‘Dremel’ like multi-tool, I cut a small opening in the rear of the chassis sufficiently large enough to accommodate the cooler tubing and wiring.

The 4-pin Molex connector tails and PWM extension cables were then attached to the controller board and the card fitted into the HP Z620.

I mounted the radiator, fan and pump/reservoir onto a metal plate I fabricated at work and started with the plumbing. My online search indicated the best arrangement is for the pump to feed directly into the GPU card cooler, the GPU card outlet is then connected to the bottom radiator connection, the top radiator connection then feeds back to the reservoir which sits above the pump. Keeping the reservoir located above the Tesla card, pump and radiator eliminates the risk of air pockets in the loop.

Next was connecting the Tesla cooler to the external radiator set-up.

Then finally connecting the fan controller board to a spare 4-pn Molex connector located behind the DVD drive;

To make the build look a bit less DIY, I also made a simple cover to tidy the appearance of the wiring and tubing. Below are images of the final build.

Anyway, I hope this post is of interest to some and perhaps add a little insight should you be considering adding liquid cooling.

By fitting the temperature controller board the Z620 is virtually silent when not rendering or performing GPU intensive work since the fan will only spin at 40% speed. As the M2090 starts to heat up the fan speed is controlled proportionally to the M2090 temperature.

Finally, here’s the complete parts list;

If you have any questions or are considering a similar type of project and would like some further details then please feel free to pm me.

[Calaudude]

NIce modding!

How much you payed now altogether for your rig?

[Brian1965]

Hi Calaudude,

I paid £700 for the original machine and it was spec'd at 2x E5-2640, 64GB RAM, K4000 and 2TB HDD. What I received was 1x E5-2660 (and no 2nd CPU riser), 32GB RAM (PC3-12800R), K4000 and 1TB HDD. However the seller supplied the 2nd CPU riser, additional E5-2660, and 32GB RAM (PC3-14900R) and 2TB HDD.

I got my K4200 for £275 on ebay but sold the K4000 for £225! So £50

Sold the 32GB PC3-12800R for £80 and bought another 32GB PC3-14900R for £100. So £20

Sold the 2x E5-2660 for £85 and bought 2x E5-2670 for £125. So £40

I think you get the general idea. The 2-port SATA III PCIe card (also has 2 ext. USB3 ports) cost under £15, ICY dock £35, 4TB drive £100, 256GB SM951 £60, ASUS Hyper Mini PCIe card £16,  SanDisk 480GB SSD £85.

The Tesla project was by far my biggest budget item to date coming in at £230 but still better than paying £1200+ for a K6000.

Total spend so far I suppose is at the £1350 mark. I generally get good deals by searching out the bargains on ebay or gumtree.

[Calaudude]

I got s similar rig for 890 quid , free shipping

Z620,2xe5-2670,96g,nvs 315,1tb sata,win 7 .  I did buy a more complete set because I dint want to work much to use it

so I removed the nvs and sata and added a gf970 and a ssd 256, 3tb sata (all from my old rig) , the free update from win 7 to win 10 was fine too

I dont do anything with cad or modelling I just got this rig because I liked the case and hp engerneering

I like the idea to have 32 logical cores despite I dont need them

[FoxyPC]

Doing the Same here in Switzerland, second hand Z620 with Quadro + Tesla M2090,  I m waiting now for the hardware to arrive by the mail to start on the Modding,

But I was hoping that a large Noctua Fan with a PWM temp controler module insde the box ( or outside )

would have done the Job for less ( efforts, Style and Money) ?

Maybe I guess you've tried before with no success ? or did you just went straight for the "heavy gun" with water cooling ?

Nice Job anyway, and I guess very effective . 

Regarding Software 

I m planning to use Nvidia Maximus (to bind the GPU without SLI ) , Cuda Toolkit for Cuda C, and Open ACC.

I was also contemplaing the idea to do a Lab cluster With two machines with Voltaire-Infiniband PCIe Cards 20GB/s to link them , and have another cheap TESLA M2090 6GB hooked up on another machine, as the Z620 is lacking space and spare PCIe slots for a second one. Any thoughts on any of the above welcome ! 

Best,

Fabs. 

[FoxyPC]

Also I wondering is you RAM 32GB PC3-14900R running at full speed 1866MHz ? or is it maxing out at 1600MHz 

i am still hesitating on upgrading but i think PC3-12800/1600  would bring the same benefit ?  

Thanks

[Brian1965]

HI FoxyPC,

Yes the RAM is curently only running at 1600MHz. The plan is when the V2 processors drop in price I won't need to upgrade the RAM when I install the V2 processors.

In case it's of interest;

Please note that Passmark doesn't measure the additional GPU (M2090) performance. For that I used Octanebench;

The M2090 only kicks into gear when I use SolidWorks Visualize which is a 3D CAD render engine but the speed increase, (compared to the K4200 by itself), more than doubles. Better performance than a single K6000 card!

[Brian1965]

Regarding the choice of air cooling versus water cooling the M2090, I was more concerned with fan noise. (I also wanted to ensure maximum cooling or 'heavy guns' as you eliquently put it).

Having previously owned some high end Nvidia gaming cards, I know just how loud these things can get. I also wanted to make sure that the M2090 was almost transparent when browsing or using other software, i.e. the fan on the M2090 cooler only runs at idle when I'm not rendering.

There is another forum member, (BambiBoomz), who has successfully ran a M2090 in a near identical system using air cooling;

This is still a work in progress but basically he has a fan located between the M2090 and the bottom chassis fans as shown above. The fan is currently powered from a front USB port so the fan is only ever on or off and is currenlty controlled manually.He has stated that his set-up is a little noisely when on.

The fan set-up on my system uses proportional control so fan speed is automatically controlled.

[Brian1965]

Hi FoxyPC,

I'm not sure why but for some strange reason my posts are not being uploaded, (even though the forum says 'success, your post has been listed') but here goes again;

With regards to the choice of using water cooling, I was more concerned with noise than anything else. I have owned a couple of Nvidia high end gaming cards so I know just how loud these things can get. My son uses my computer to render 3D animations in Maya, (using the FurryBall RT plugin), and some of those run for hours at a time. You are also correct in that I went for the 'Heavy Guns' as you eloquently put it. I know liquid cooling is far more effective than air cooling. I also wanted a method to monitor the M2090 GPU temperature as this is not possible in Windows 7 or 10? The fan on the M2090 cooler is proportionally controlled, (via the temperature sensor mounted to the back of the M2090), so when I'm not rendering the fan is at idle speed and virtually silent. There is another forum member, (BambiBoomz), who has successfully used air cooling;

As you can see, he has placed a fan on the inside of the Z620 chassis which is powered from a front USB port. With this arrangement the fan is either on or off though and noise has increased quite a bit. But it still works and appears to do the job.

With regards to the RAM, yes, this currently only runs at 1600MHz presently. The plan is to buy a couple of V2 CPU's when the prices start to drop without the need to upgrade the RAM in future.

[Brian1965]

Forgot to mention this but if you check out my first post on the forum I discussed a few options and did some initial testing using a stock, (unmodified), M2090 GPU. It kind of explains how I came to the decision to go with liquid cooling.

http://h30434.www3.hp.com/t5/Business-PCs-Workstations-and-Point-of-Sale-Systems/SM951-M-2-SSD-in-a-...

Pages 2 and 3 discuss the Tesla M2090.