Power Consumption
The redundant power supplies in the server are 1.2kW units. These are 80Plus Platinum rated units. At this point, most power supplies we see in this class of server are 80Plus Platinum rated. Almost none are 80Plus Gold at this point and a few are now Titanium rated.
For this, we wanted to get some sense of how much power the system is using outside of the hard drives. Since there are 36x hard drives, plus multiple SSDs, those are going to dominate the power budgets which will have the biggest influence on power. We tested the system with just the RAID controller, NICs, CPU, and memory to get a sense of how much room there is in the PSUs.
- Idle: 0.27kW
- STH 70% CPU Load: 0.40kW
- 100% Load: 0.49kW
- Maximum Recorded: 0.56kW
That leaves around 700W for drives which is reasonable. We probably would have preferred 1.4-1.6kW PSUs to be safer power-wise. This system fully loaded with expansion cards, SSDs, and hard drives could easily go over 1kW in power consumption. We suggest one needs to run this system on 200-240V power to ensure the full 1.2kW is available.
Note these results were taken using a 208V Schneider Electric / APC PDU at 17.5C and 71% RH. Our testing window shown here had a +/- 0.3C and +/- 2% RH variance.
STH Server Spider: Gigabyte S452-Z30
In the second half of 2018, we introduced the STH Server Spider as a quick reference to where a server system’s aptitude lies. Our goal is to start giving a quick visual depiction of the types of parameters that a server is targeted at.
The S452-Z30 is not designed to be a system with high compute or memory density. Although there are NVMe drive bays, the system is not designed for that role either. Instead, this system is designed to deliver capacity hard drive storage in standard 4U racks. With that, the system has quite a bit of density without resorting to a top-loading design. This makes servicing the system easier, but it also means that Gigabyte offers a lot more expandability versus competitive Intel solutions with the large PCIe Gen4 slot array.
Final Words
Overall, the Gigabyte S452-Z30 is a solution we really like. Conceptually, it offers a great storage box without needing deep racks or top loading. Adding small features such as tool-less drive trays will be welcome by anyone who has ever had to install drives in a system like this.
There are a few areas that can be improved. This feels like a platform where Gigabyte is making a cost-saving v. serviceability trade-off as some parts are harder to get to. We wish the SAS Expander did not have a fan. There are a few small items we noted on the front rack ear and with the BMC password sticker that are relatively minor.
One must remember that this is a single-port SAS solution. As a result, it is likely to be used more in a scale-out storage scenario such as for Ceph, Lustre, Gluster, or other similar solutions. If you need a high-availability node, this is probably not the best solution.
Since we had 36-bay designs in the Xeon E5 era, the appeal of this system was immediate. We get ~4x the backhaul storage bandwidth versus our old 6Gbps units thanks to the newer Broadcom SAS expander which means we can use a single RAID card/ HBA instead of multiple cards. That leaves more room for expansion whether that is for additional disk shelves, networking, accelerators, or more NVMe storage. The AMD EPYC single-socket solution feels like a natural upgrade path from dual Xeon E5 systems and effectively cuts the CPU sockets required versus Intel Xeon Scalable solutions that usually will feature two CPUs in systems like this. As an added bonus, the topology is built around a single processor instead of being split between multiple processors where we would need to consider a UPI inter-socket hop.
Many of our users are transitioning wholeheartedly to flash storage, but there are still many scenarios where hard drives fit the right capacity per dollar metrics. Five to seven years ago we may have been more concerned with the aggregate performance of drives in a system like this, but now we see performance storage moving to flash while HDDs reign in capacity storage. For those who need capacity storage in standard racks without resorting to top-loading, the AMD EPYC-based Gigabyte S452-Z30 is an awesome solution.
Almost ordered 3 of these systems today, unfortunately it just missed the mark for us. It would have been a home run if someone took out 6 of the 3.5” drive bays from the back, moved the power supplies down to where ~ 3 drives were and put between 6 to 8 nvme where the other 3 drives were. That would free up room in the top 2 u for a motherboard with 2 sockets and 32 dims. With that you could have tired storage, or vm’s that could include nvr duties with possible room to add accelerator cards for video AI. Would be a branch office powerhouse, just add networking.
Something I think is an interesting use case is utilizing the PCIe slots for NVMe storage whether that is an internal card or an external chassis.
@Patrick: Thoughts of using this as a high-performance TrueNAS Core appliance with a Gigabyte CSA4648 (Broadcom SAS3008 in IT Mode) HBA?
We did not get to test FreeNAS. Proxmox VE or soon TrueNAS Scale are based on Linux.