I have an update on my personal situation of this issue. With less than a month of warranty left, I finally decided to take matters into my own hands and change the thermal paste for the CPU and GPU myself. Upon disassembly, I found the old thermal paste dried up and poorly applied, to nobody's surprise (see pictures below).
I used Arctic MX-4 thermal paste and now the temperatures are better, they seem to have dropped by 5-10 Celsius. I still seem to get thermal throttling when doing intensive stuff and the laptop still gets hot after a while of intensive work but the temperatures seem to be lower nevertheless.
In short, the best solution for me personally was to undervolt the CPU with ThrottleStop (-0.125V for both CPU Core and CPU Cache), replace the thermal paste with higher quality paste (I used Arctic MX-4), and, optionally, enable Comfort Mode under Performance control in the HP Omen Command Center.