Gigabyte GA-7PESH3 Review – High-End Workstation/ Server dual socket motherboard

The Gigabyte GA-7PESH3 is a high end workstation motherboard that has lots of functionality built-in making it at home both with server and workstation components. When Gigabyte contacted us about reviewing this motherboard we were more than excited. It has all of the features one would need from a workstation motherboard and covers just about every want. Oh, and it comes with Gigabyte’s excellent Avocent IPMI interface if one were to use it as a GPU-less server platform.

Test Configuration

We were able to use two Intel Xeon E5-2697 V2 chips as well as our previous generation Intel Xeon E5-2687W (v1) chips with the motherboard. We also had to employ more SSDs than normal due to the addition of the SAS controller. Overall we think this would be considered a fairly high-end workstation configuration.

1. CPUs: 2x Intel Xeon E5-2697 v2 and Xeon E5-2687W CPUs
2. Motherboard: Gigabyte GA-7PESH3
   
3. Memory: 8x Samsung 16GB 1600MHz DDR3 ECC RDIMMs
   
4. SSDs: 2x Samsung 840 Pro 256GB 8x Crucial M500 240GB, 4x SanDisk Extreme II 240G
5. GPU: NVIDIA Quadro K4000
6. Operating System: Ubuntu 12.04 LTS, Windows Server 2012 R2, VMware ESXi 5.5

We also ran the Gigabyte GA-7PESH3 through our standard expansion card compatibility suite with SAS HBAs, RAID controllers, Infiniband cards and 10GbE/ 40GbE networking cards.

Gigabyte GA-7PESH3 Overview

The Gigabyte GA-7PESH3 is a fairly large motherboard utilizing an E-ATX or SSI EEB form factor (12″ x 13″.) Even given its size, the motherboard manages to pack a lot of functionality into the space. The dual Intel LGA2011 sockets and their four DIMMs per socket are oriented in a top to bottom fashion. This orientation is popular for workstation chassis with top airflow venting.

Gigabyte utilizes the LGA2011 square ILM socket which is also common on single socket LGA2011 platforms. This makes cooling options plentiful. The area around the dual LGA2011 sockets is generally clear. Intel Xeon E5-2600 and E5-2600 V2 processors can have a TDP of 150w making cooling a key consideration at the top end.

Each Intel Xeon E5 (or Core i7 for single socket considerations) gets four DDR3 DIMM slots. The DIMM slots can support UDIMMs for the Intel Core i7 processors and both UDIMMs and RDIMMs up to 1866MHz for Intel Xeon E5 processors. The Intel Xeon E5-2600 v1 and v2 families can support more DIMMs but it is uncommon to see eight DIMM per processor configurations in EATX form factors that also have many PCIe slots.

As one can see, the sockets and DIMMs take up a large amount of physical space almost extending to the edge of the board.

Sitting above the CPU sockets are the two 8-pin CPU power connectors along with the standard ATX power connector. One other feature  along the top edge are the 4-pin PWM CPU fan power connectors. Overall there was plenty of room to install power connectors on the Gigabyte GA-7PESH3.

Perhaps one of the biggest features of the Gigabyte GA-7PESH3 is the array of seven PCIe x16 physical slots. For those with large cards such as GPUs (e.g. NVIDIA Quadro K4000 cards) and PCIe storage and network cards, the massive array of PCIe lanes is made possible largely because each Intel Xeon E5-2600 series processor gets 40x PCIe 3.0 lanes for a total (possible) on a dual socket system of 80 lanes.

With all of that PCIe bandwidth from the Intel Xeon E5-2600 series one can run up to four PCIe 3.0 x16 electrical cards or even a PCIe 3.0 x16 electrical with six PCIe x8 electrical slots. This is a great configuration for adding multiple GPUs, storage controllers, high speed network controllers and etc. The Gigabyte GA-7PESH3 supports NVIDIA 4-way SLI and AMD CrossFireX delivering high-end workstation graphics capabilities.

Gigabyte provides two internal USB 3.0 headers which is an interesting feature while many competitor motherboards feature only a single internal slot. These internal USB ports are especially useful for applications such as installing a base hypervisor such as VMware ESXi, installing recovery media or utilizing them for USB software license authentication devices.

Beyond the two internal USB 3.0 ports, the Intel C602 chipset provides 2x SATA III 6.0gbps ports (red) and 4x SATA II 3.0gbps ports (black.) These six ports are only part of the story.

One can see a heatsink specifically covering a LSI SAS2008 controller. The LSI SAS2008 is the same SAS controller found in cards such as the LSI 9211-8i and IBM ServeRAID M1015. The SAS controller provides eight 7-pin SAS/ SATA connectors that work at up to 6.0gbps per port. The LSI SAS2008 also has fairly good SAS expander support so it can support chassis that hold many spinning disks or SSDs.

Between the LSI SAS controller and the Intel C602 ports there are a native 14 storage ports with ten running at 6gbps and four at 3.0gbps.

Looking at the rear I/O panel we can see a somewhat standard workstation configuration. There is the “legacy” stack of a combo PS/2 keyboard/ mouse port just below two USB 2.0 ports. Next to this there is a legacy DB-9 serial connector and a VGA port. The VGA port is unusual but comes from the Gigabyte GA-7PESH3 using the ASpeed AST2300 controller. It is certainly not something to use for gaming but is valuable as standard server-type functionality.

Four USB 3.0 ports and two USB 2.0 ports round out the USB functionality. This is a solid overall set given the fact that the Intel C602 platform is lacking in native USB 3.0 ports. There is one management network port for the IPMI interface along two Intel 82574L gigabit Ethernet controllers. The Intel 82574L was a very popular server network controller for many years so it is likely one of the most well supported NICs around. One can see one of the Intel Ethernet controller chips between the two PCIe slots in the picture below.

Along with all of this network functionality is 7.1 + 2 HD audio. Unlike server oriented motherboards, the Gigabyte GA-7PESH3 has all of the standard workstation features such as onboard audio.

One very important note on the platform. Although it does have all of the almost mandatory workstation features such as 4-way SLI, HD audio and USB 3.0 ports, it also has many server features. The LSI SAS2008 controller is a very high quality part as are the Intel 82574L gigabit NICs. Beyond these the ASpeed AST2300 based BMC provides standard IPMI functionality allowing for remote management. In fact, before we installed the NVIDIA Quadro K4000 GPU we were able to setup the machine completely remotely using the iKVM feature. The bottom line is this: the Gigabyte GA-7PESH3 has all of the major features on would want from a dual socket LGA2011 workstation and server motherboard. As a result, a workstation built today could easily transition to become a server when newer platforms eventually become available.

Thermal Results

For the thermal imaging we used the Intel Xeon E5-2697 V2 processors. Since we were utilizing our standard 4U chassis test bed we added the heatsinks/ fans in front to rear configuration as if it were in a server-style cooling enclosure. We did this for a simple reason. Our test bed was setup for front to rear airflow and the additional chassis fans we needed to blow in the correct orientation.  Standard passive heatsinks work less well with top to bottom memory configurations. Chips like LSI SAS controllers generally have passive heatsinks (as with the Gigabyte GA-7PESH3) and therefore require airflow from chassis fans. The motherboard’s memory configuration is setup for top cooling fans to exhaust CPU heat which is common in a workstation environment.

To capture these images we utilize our FLIR Ex series professional thermal imaging camera and turn on FLIR’s MSX enhancements so we can see components outlined clearly. We put the system under 100% CPU load for a period of 24 hours to let “heat soak” set in prior to taking the images. We also used a RAID 0 set and a sustained 2GB/s read over the LSI controller to ensure it was reasonably loaded.

Overall, the Intel Xeon E5-2697 V2 is a very cool chip the motherboard overall stays well within operating tolerances. This is important since components such as NICs and SAS controllers can overheat if the setup is designed poorly and cause stability concerns. One can clearly see the impact of changing the fan orientation as the second CPU heatsink is slightly warmer than the first.

One can see the key heat sources aside from the processors are the AST2300 BMC, the Intel Intel C602, the dual Intel 82574L controllers, the LSI SAS2008 controller. There is certainly some heat generated by the power delivery circuitry.

All temperatures are well within operating thresholds and stay relatively cool, even when installed in a sever oriented airflow environment.

Gigabyte – Avocent IPMI Interface

Using this motherboard and testing it through VMware ESXi 5.5, Windows Server 2012 R2, Hyper-V Server 2012 R2, Ubuntu 13.10 and CentOS 6.5 required many operating system installations. We also needed to reboot the system many times during our card compatibility testing where we cycle through various cards such as Mellanox ConnectX-3 VPI 56gbps Infiniband/ 40GbE and LSI SAS 3008 controllers. The Avocent IPMI solution is simply great. After a few weeks of testing it was very stable and more responsive than a standard AMI MegaRAC solution common in the industry.

For those wondering, we did a full two part Gigabyte IPMI walk through (see Part 1 and Part 2). Key features include the ability to remotely mount multiple images over the network and iKVM features.

After spending some time with the Gigabyte-Avocent MergePoint solution it is certainly one of the better web interfaces out there remaining responsive during our testing.

Conclusion

Overall, this is an absolutely awesome workstation motherboard. The Gigabyte GA-7PESH3 combines all of the necessary workstation features with features usually reserved for server motherboards. The combined feature set makes for a strong motherboard that can be a workstation today while re-assigned to a server role in the future, thereby protecting the initial investment.