I've been slowly collecting bits to build a liquid-cooled server. I have no practical reason to do so, but I've been intrigued by the concept. Finally I bit the bullet and got the following:
- AMD Ryzen 3900X (12-core, 24-thread)
- Gigabyte X570 Aorus Elite motherboard
- Rosewill "bitcoin mining" 4U case
- Thermaltake 360mm ('thicc') radiator
I then ordered a bunch of parts from EKWB:
- PowerColor Liquid Devil RX5700 XT
- CPU/motherboard full block
- X570 chipset block (the stock Aorus fan/block isn't compatible with the GPU)
Everything else was pretty normal to build a PC/pseudoserver. It took me quite a while to put this together- not only did I have to wait for two batches of parts from EKWB in Slovenia, plus deal with a customs delay, but I also needed to fabricate a bunch of parts.
The first thing was the radiator. I designed two "sliders" to hold it in, since the width of the radiator is close to the case width. It took a lot of iterations to deal with bolt holes, drain cap of the radiator, and so on.
I then designed a block to hold the pump. It only took a couple of tries on it. I was happy to put a bit of a "lean" on the side bolt holes, which leaves room for a rounded corner on the inside of the case.
I installed the motherboard and then tried adding the GPU and the water cooling. They both required disassembling the motherboard from its mounting plate. So many screws! The GPU was even more work when I realized the X570 chipset heatsink/fan were in the way. I first took it off and thought about cutting it, but it's a big chonky piece that would have been really difficult.
The next GPU problem is the Rosewill case doesn't have expansion slots in the back. So weird! Obviously it's designed for a motherboard to be placed there, but I guess they assume nobody will need to add a card to the motherboard. So I Dremeled out the back of the case. I might design and print something to secure the card in place.
To add to my list of GPU struggles, the liquid cooling makes the GPU taller than a 4U case. I took some measurements, carefully dremeled a hole, then designed a plexiglass popemobile window. I might add a second window or enlarge that one to show the cooling and RGBs, but that is optional and would come later.
I'm unhappy with a couple of things:
- the motherboard only seems to have two fan headers that are usable from Linux. One has the pump, the other has puller fans. I have a few other rows of fans, so I either need to daisy them all together or control them from off the motherboard.
- the RGB LED headers aren't controllable.
- there are no ports for off-motherboard thermistors. I put four inline thermistors on the cooling loop but I have nowhere to plug them in.
- It's hard to know if the pump is running. It sends RPM back, but there's no other real verification. I haven't solved this yet, but I will probably stick a flow indicator in the loop, then 3D print an optical sensor to look through it.
- The cabling mess. I tried heat-shrinking the fan cables, but that made things worse, partly because it's thick heatshrink that glued everything together, so I can't remove a fan from the shrink. There's a lot of cabling coming from the PSU too. I'm delaying solving this until everything is done.
I have a lead on solving the first three things- I've designed a board around the ESP32 with three fan headers and two DRGB headers. If that works I'll add at least five inputs for the thermistors and flow control. That way I can use the DRGBs to display temperature at a glance, plus control the fans and gather/transmit all the important data back to my network.
What am I using it for? I have a couple things in mind. First, it sits in my Kubernetes cluster, and the GPU will let me transcode my security videos better. I thought I'd dabble with some other GPU computing, but there aren't great drivers or support for non-Nvidia cards :/. As a final use, if I can figure out how, it'd make for a nice Steam Remote gaming system, if I can figure out how to do it with a non-Windows and container-based system.
The crowning touch was to add the 3D-printed replica of the Gavin Belson signature to the case. It looks perfect!