Is it the GPUs that are causing this? Even a very powerful PC need not draw much at true idle. The trick is getting to to a "true idle" which can be inhibited by lots of unexpected situations and broken hardware. One very common issue is that Realtek NICs prevent deep sleep states because either they are broken or their Linux driver is broken. Any kind of ethernet is worse than wifi in terms of energy consumption. Keeping a display attached to the box can greatly increase power consumption and might not be needed on a server. I have a fairly power Alder Lake-based server with 3 NVMe SSDs and 12 SATA devices, that draws < 15W from the wall when it's on but inactive.