Modify HP Z8 G4 UEFI (BIOS) to enable Windows O/S & HP Perf. Advisor to see all 32GB GPU memory

[joselso]

At CommLab - my place of work, I have started using a brand-new HP Z8 G4 workstation, model Z3Z16AV CTO, preinstalled with Nvidia high-end Quadro GV100 32GB HBM2 GPU (HP model 3XK94AV), as well as 192 GB RAM (HP model 6CN34AV). Operating system - Windows 11 Pro for Workstations.

I started HP Performance Advisor, and was surprised to read in Configuration Report (version: 3.1.9319.0, Database Timestamp: 29/09/2023 16:55:11):

Operating System: Windows 10 (64-bit) – Microsoft Windows 11 Pro for Workstations

Graphics Card Name: Quadro GV100

Memory Size: 4 GB

Graphics Card Provider: HP (103C)

Well, 4 GB instead of 32 GB! So, I checked System Information:

In Windows 11 - System Information – Components - Display:

Adapter Description NVIDIA Quadro GV100
Adapter RAM (1,048,576) bytes
================================
Memory Address 0xB9000000-0xBA0FFFFF [3,103,784,960 - 3,121,610,751]
Memory Address 0xE0000000-0xF1FFFFFF [3,758,096,384 - 4,060,086,271]
Memory Address 0xF0000000-0xF1FFFFFF [4,026,531,840 - 4,060,086,271]

So, Windows operating system recognizes only a small fraction of the 32GB GPU memory, and apparently dedicates RAM accordingly.

On the other hand, in Nvidia control panel on the same workstation, different information is presented:

NVIDIA System Information report:

Operating System: Windows 10 Pro for Workstations, 64-bit

Total available graphics memory: 130411 MB
Dedicated video memory: 32768 MB HBM2
System video memory: 0 MB
Shared system memory: 97643 MB

Now, vendors of high-end software applications (Cisco, Dassault - Simulia CST) state that all supported GPU cards require enablement of the BIOS setting that allows greater than 4 GB of memory-mapped I/O (MMIO). I could not find such explicit setting in HP Z8 G4 UEFI BIOS version 2.91.

While putting aside the strange mixture between Windows 10 and 11 exhibited in the a/m sources of information located on the same workstation, the important question of GPU memory size remains:

Please advise how to modify HP Z8 G4 UEFI (BIOS) settings to enable Windows operating system and HP Performance Advisor to recognize all 32GB of GV100 GPU memory.

[DGroves · Accepted solution]

you do not have a bios issue, you have a issue with the Advisor software not properly detecting the correct video card capabilities. the GV100 is indeed a 32GB video card

https://www.nvidia.com/content/dam/en-zz/Solutions/design-visualization/productspage/quadro/quadro-d...

solution.check for a program update if you have a older version of the advisor software and if none wait for a fix/update from HP, your system is working fine hardware/software wise

some  bios's have a option to toggle the decode above 4GB on/off,  to mostly improve gaming performance  as this setting when enabled on windows operating systems with 64gb or more of ram can safely access larger chunks of the video cards memory which can improve performance (along with resizable bar bios setting) in some specific cases

[joselso]

Well, I won't niggle over this minute issue of mis-reported GPU memory size, when on the whole the GV100 GPU preinstalled in HP Z8 G4 does an excellent job in accelerating simulations and cutting short run-time by a factor x7 up to x11 in some cases that I have already run this week.

There are other aspects of the use of GPU worth noting:

In our lab, the HP Z8 G4 is LAN-operated remotely without a locally attached display monitor, in TCC-mode of driver, as is strongly recommended for NVIDIA GPU's by several vendors of simulation software applications.

One disadvantage of this mode is that Windows Task-Manager cannot monitor GPU activity at all (GPU greyed-out when pressing View in Task-Manager window).

Note that earlier this week, before switching nvidia-smi.exe from its initial WDDM-driver-mode to TCC-driver-mode, GPU activity was fully monitored in Windows Task-Manager and reported up to 99% GPU-memory usage as well as GPU temperature during simulation.

However, a warning message by the simulation software stated then:

Hardware Accelerator Device warning:
- GPU 0 uses WDDM driver mode. For optimal performance, and also to prevent memory allocation problems, please activate TCC driver mode.

So I had to set nvidia-smi.exe to TCC-driver-mode.

Also worth noting: That file nvidia-smi.exe was not present in its standard folder:
C:\Program Files\NVIDIA Corporation\NVSMI
In fact, that folder does not exist in our workstation.
We managed to find the desired file nvidia-smi.exe and ran it elsewhere in C: drive, as evidenced from the following Administrator-mode commands:

C:\Windows\System32>cd "c:\Program Files\NVIDIA Corporation\NVSMI"
The system cannot find the path specified.

C:\Windows\System32>cd "\Windows\System32\lxss\lib"

C:\Windows\System32\lxss\lib>nvidia-smi -L
GPU 0: Quadro GV100 (UUID: GPU-#-#-#-#-#) (ID removed)

C:\Windows\System32\lxss\lib>nvidia-smi -i 0 -dm 1
Set driver model to TCC for GPU 00000000:2D:00.0.
All done.
Reboot required.

The NVIDIA Windows 11 driver that was preinstalled with GV100 GPU by HP in our Z8 G4 model Z3Z16AV CTO workstation is of version 527.48. There are more recent NVIDIA driver versions offered by HP for this workstation, but for now I will keep what HP decided best for preinstalling in this brand-new workstation, manufactured in October 2023, seeing that it is doing its job as expected.