Hello dyyang
I had an HP 1360t with the same symptom.
I swapped every component connected to the motherboard from the power supply to graphics, and RAM, except the CPU.
I did not have a spare CPU and was not going to buy one just to test my PC. I had other spare good replacement parts and used them.
Disconnected all non-essential peripherals. Disconnected power to HDD and optical drive. Disconnected USB.
Nothing I did changed this behavior.
My conclusion was the motherboard had failed.
Grzy