Create an account on the HP Community to personalize your profile and ask a question
11-07-2021 11:18 PM
I have a Z840 with 2 x Xeon E5-2680 v4 @ 2.40GHz (28 Cores) and 224GB RAM used to run compiled Fortran codes for numerical simulations (mostly using OpenMPI).
I want to add more (2 or 3) Z840s (found some refurbished ones) to scale the computational power up by connecting all machines together using InfiniBand. The motherboard supports PCIe v3.0 so I assume my options are limited and I do not have much knowledge on how to best achieve what I want.
From what I could gather a QDR (40Gb) InfiniBand connection should be adequate. If someone has experience on how to best setup 3-4 Z840s connected together via InfiniBand, it would be very much appreciated. e.g., what's the best network card(s) to buy?, do I need a switch? or can machines be connected to each other directly?
11-08-2021 06:02 AM
does your software support running multiple instances of itself on separate systems?
if not then a 3rd party application must be used to tie your application together
simply networking computers together will not make the software you run on them able to mesh/sync their data the software you are running must support this function
last, yes you will need a switch
11-08-2021 07:11 AM
thanks for the reply.
The open-source code I am using is compiled with gfortran+openmpi and can run on multiple nodes on a proper HPC cluster (e.g, 100s of cpu cores distributed over many nodes).
Locally though the code (software) is developed and tested on one Z840 workstation. I am looking into increasing the “local” computational power by adding homogenous workstations to reduce the need to go to the HPC cluster.
Very simply put, I want to increase 28 cores to 112 cores by adding 3 more Z840s. So it’s not about running multiple instances of the software on multiple nodes. I want to be able to run one instance of the software using the combined cpu/ram of 4 workstations.
From what I’ve read so far a qdr infiniband card on each machine with a switch to connect them all will do this but I cannot be sure if I am missing something.
I am looking for cost effective hardware advice (infiniband cards, switch, cables) that are compatible with the Z840s (supports PCIe v3.0).
any further advice is appreciated. Cheers!
11-08-2021 05:33 PM
what you are asking/wanting to do is not a turnkey solution you will have to spend time to figure out how to configure the software to work in a multisystem configuration and at the same time hope your selection of hardware will be flexible/configurable for the required task.
i recommend you start doing quite a bit reading on this subject by reading any white papers the software vender has written on this subject and then also post the the venders online forum where other users of this software can reply
Didn't find what you were looking for? Ask the community