ASUS Z9PE-D8 WS Dual Xeon E5-2600 Workstation Motherboard Review

The ASUS Z9PE-D8 WS motherboard is a workstation board meant to house dual Xeon E5-2600 series CPUs as well as several PCIe 3.0 cards. Recently Pieter previewed the Z9PE-D8 WS and noted that it is a competitor to offerings such as the Supermicro X9DA7 and EVGA’s SR-X Classified. The ASUS Z9PE-D8 runs around $600 right now at retail, which is fairly expensive until one realizes that the chips that go in the motherboard are similarly very expensive with chips such as the Intel Xeon E5-2687W running in the $2,000 range. With dual LGA 2011 sockets, this is clearly not a mainstream motherboard and the higher price tag does reflect the fact that this is a high-complexity and lower volume platform. With that said, the current consumer/ prosumer platform of choice the Z77 – LGA 1155 or C216 – LGA 1155 has nowhere near the memory capacity, compute potential, nor the PCIe bandwidth the LGA 2011 has to offer. Let’s take a look at the package from ASUS.

Test Setup

For this review I am using dual Intel Xeon E5-2690 CPUs which are 135w TDP parts and sit with the E5-2687W as Intel’s current top of the line Xeon E5-2600 series CPUs.

1. CPUs: 2x Intel Xeon E5-2690 CPUs
2. Motherboard: ASUS Z9PE-D8 WS
3. Memory: 8x 4GB Kingston unbuffered ECC 1333MHz DIMMs
4. SSD: Corsair Force3 120GB, OCZ Vertex 3 120GB 2x OCZ Agility 3 120GB
5. Power Supply: Corsair AX850 850w 80 Plus Gold
6. Chassis: Norco RPC-4220
7. Cooling: 2x CoolerMaster Hyper 212 EVOs
8. Operating System: Ubuntu 10.10 Server and Windows Server 2008 R2

This is the typical test setup I have been using for my dual Xeon E5 reviews. My 8GB 1600MHz CL11 DIMMs are still on order but I do hope to update soon. Interesting note, the setup runs virtually silent at idle.

Features

The board itself is large measuring a hefty SSI EEB size 12″ x 13″ which is considerably larger than a standard ATX motherboard and the same size as Extended ATX (EATX) motherboards. I will note that the vast majority of dual Xeon E5-2600 motherboards do come in larger-than-ATX form factors because designers need to fit two large sockets and heatsinks, a minimum of eight DIMM slots and a variety of drive connectivity plus PCIe slot options. Just to give one an idea, the EVGA SR-X classified is a custom HPTX form factor measuring 15″ x 13.6″) which is about 33% larger PCB area than the ASUS Z9PE-D8 WS (156 sq. in versus 204 sq. in.)

First focusing on the CPU area one can see that there are two LGA 2011 sockets atop the board. These two sockets are oriented to pull cool air to the top of the chassis from the bottom. Consumer-style workstation chassis tend to have large exhaust fans atop the chassis so this orientation is for that enclosure layout rather than a typical rackmount enclosure where airflow is straight front to back. One can use large CPU coolers but as overclocking is limited to say the least on the Intel Xeon E5-2600 series platform, one can optimize on cost effective, quiet cooling and make a very quiet and powerful workstation. Along these lines the eight DDR3 DIMM slots (four per CPU) provide enough DIMM slots to feed the dual CPU’s quad channel memory interfaces. These eight DIMM slots are also oriented from top to bottom which again points to this board being designed for workstation chassis with large fans atop the case. A user can add a maximum of 64GB of unbuffered ECC or non-ECC DDR3 memory or 256GB of registered ECC memory to the Z9PE-D8. The ability to use registered ECC memory gives the board a decent amount of headroom in terms of memory capacity even with only eight DIMM slots. Some high-end server configurations have twenty four DIMM slots, but this is a fairly reasonable configuration for a workstation product. One very important point in this screenshot is that there are two eight-pin CPU power connectors alongside the top of the board and the 24-pin ATX power connector. Like the CPU sockets and the DIMM slots the MOSFET heatsinks are in a north-south orientation.  I will say that one can see from this picture that the ATX power connector and adjoining 8-pin power connector are oriented such that the latches are on the side next to the heatsink. For those with large hands, this may make disassembly a bit more difficult.

One of the areas that we see ASUS really do a great job of here is with the PCIe connectors. There are a total of seven PCIe 3.0 x16 physical connectors. Since each Intel Xeon E5-2600 series CPU has 40 PCIe 3.0 lanes total ASUS has a maximum of 80 lanes thus less than needed to drive all physical x16 slots at x16 electrical (afterall 7 slots * 16 PCIe 3.0 lanes each would equal 112 lanes.) ASUS instead provides the four blue slots to be PCIe 3.0 x16 electrical and the black slots to be PCIe 3.0 x8 electrical. For those looking at building massive workstations, this is about as good as it gets. I do want to point out that I place Intel and Supermicro 10GbE NICs, LSI RAID controllers and Mellanox Infiniband controllers in every server and workstation motherboard I review just to check compatibility. ASUS does do extra compatibility testing on their WS series lineup to ensure that users have better experiences with adding non-video expansion cards to the WS lineup than traditional consumer motherboards.

ASUS of course supports Quad-SLI and Crossfire on this motherboard but one could just as easily utilize Infiniband or 10GbE NICs, RAID controllers, PCIe Solid State Drive storage and etc. ASUS does provide a bevy of SLI and crossfire connectors in the box to take advantage of these features as well as some optional expansion port USB, Firewire and serial port interfaces if one needs those and has an extra expansion slot to spare.

Moving to the storage area, ASUS has a lot going on. On the bottom left of the below picture one can see two USB 2.0 internal headers which are very important in the workstation market as oftentimes software licenses are tied to USB keys. Having these USB keys loaded internally secures them versus having a potentially valuable USB key sitting outside of the chassis. ASUS provides two SATA III 6.0gbps ports and eight SATA II 3.0gbps ports off of the Intel C602 controller. Further, ASUS adds a Marvell 9230 controller onboard providing four additional SATA III 6.0gbps ports. ASUS also has SSD caching software that allows one to use a mix of hard drives and solid state drives to both utilize large storage capacities provided by traditional spinning media while caching small frequently used files on solid state drives. With the ability to connect up to fourteen SATA devices and the huge amount of PCIe storage available, one cannot underestimate the sheer amount of storage and network bandwidth this platform can handle.

I also wanted to draw one to the controllers lined up at the edge of the board. One can see the two small Intel 82574L controllers which are very popular and probably the most well supported Gigabit network controllers from an operating system, hypervisor and software perspective. They are also known to perform very well even under heavy load and are basically a staple of server and workstation boards today. Recently the Intel I350 gigabit network controllers have become popular, but those are still newer and not as well supported. Also one can see the ASmedia USB 3.0 controller and an ASPEED AST2300 controller. To me, this is one of the most intriguing features of the board. For those that do not know, the ASPEED AST2300 offers an ARM management processor and PCIe x1 2D graphics. Typically we see the ASPEED AST2300 on high-end server motherboards that have IPMI 2.0 and KVM-over-IP. I did not have one to test this functionality with, but there is a ASMB6-iKVM remote management module header so presumably one could use iKVM with this motherboard.

Looking at the rear I/O panel one can see six USB 2.0 ports, a combo PS/2 keyboard and mouse port or basic perhiperal connectivity. The ASmedia controller USB 3.0 ports on the rear I/O panel with the other two ASmedia USB 3.0 ports hanging off of an internal header for front panel connectivity. The two gigabit NIC ports from the Intel 82574L controllers are also found on the rear panel. For audio ASUS has six audio jacks for eight channel 7.1 audio controlled by a Realtek ALC898 codec along with a S/PDIF optical port. ASUS also provides DTS Ultra PC II with the Z9PE-D8 so this has more than just basic audio capabilities.

So here is the mystery of this motherboard, why does ASUS provide an ASPEED AST2300 with onboard video, but no VGA port on the rear panel? I still have not figured this out but I suspect it is because the ASPEED AST2300’s video capabilities are limited at best and most workstation users of this board will have multiple GPUs anyway. Still, adding that VGA port would basically make it so that one could use this motherboard as a huge I/O server board at some point in the future since it supports ECC memory and has remote mangement. The remote management features can lock up in very rare circumstances so having a VGA port onboard is still somewhat necessary in server scenarios. Still, having the iKVM option is absolutely great, I just wish ASUS would have exposed this feature to its users.

Conclusion

The ASUS Z9PE-D8 WS shows a lot of promise. Dual processors in a big workstation motherboard is certainly something that will create a huge system. For those looking for a powerful workstation today, with the possibility to use the board as a remotely managed server later (with the iKVM option and the onboard 14 SATA ports) the ASUS Z9PE-D8 makes a compelling case since whoever is purchasing a system based on this board will want to retain value from the system for some time to come. The ASUS Z9PE-D8 is a great board that worked flawlessly with the various 10GbE network cards, Infiniband cards, and LSI RAID controllers I used in it as well as the NVIDIA and ATI GPUs I was able to try. This is important because ASUS basically expects that anyone purchasing this motherboard has high-end PCIe cards they are looking to house. Price wise the Z9PE-D8 it sits between the EVGA SR-X classified and the Supermicro X9DA7 so given its feature set, the pricing is probably spot on. If you are in the market for a high-end workstation motherboard with lots of features, this is certainly a board to consider.