In our HPE ProLiant DL325 Gen10 review, we discussed how we now run many of the servers in the STH infrastructure. We generally buy systems with 24 core AMD EPYC chips. Paired with 8x 32GB DIMMs each, that gives us just about 10GB of RAM per core plus a bit for system overhead. We finally received our first AMD EPYC 7002 generation HPE ProLiant DL325 Gen10 and there was a major change to the networking side that we do not think is for the better. The new HPE ProLiant DL325 Gen10 that just arrived lost it’s signature basic networking which impacts the entire system.
First Generation HPE ProLiant DL325 Gen10 BCM5719 Networking
Here is the test HPE ProLiant DL325 Gen10 we used for our AMD EPYC 7002 generation review. You can see the four network ports that sit between the VGA and iLO port.
Here is one of the first systems we purchased looking down on that section. Between the HPE SAS mezzanine on the left, and the PCIe riser on the right you can see the Broadcom BCM5719. This is a quad-port 1GbE networking option that is integrated into the motherboard. HPE calls this the HPE 1 Gb 331i Ethernet adapter solution.
For some perspective here, this Broadcom chip was populated not just on every ProLiant DL325 Gen10 we installed all the way down to the ~$1100 low-end EPYC 7351P system we bought. It is also present in the HPE ProLiant DL385 Gen10 we have in the lab. To us, this seemed like a standard feature.
This “standard” feature of 4x 1GbE has been changed with the latest server that arrived in the lab.
Enter the HPE Ethernet 1Gb 4-port 366FLR Adapter
On the newest system, we saw the spec that said 4-port 1GbE and thought that it would be the same Broadcom configuration. This server was ordered just before the DL325 Gen10 review went live on STH. When it arrived a few weeks later, this is what we saw:
Apologies for the photo. Datacenter lighting. What you will notice in the upper middle of the photo, is that the Broadcom BCM5719 was de-populated as were the 4x 1GbE ports. Instead, a quad 1GbE NIC is installed in the FlexLOM slot.
From HPE’s current Quick Specs: “NOTE: Embedded 4x 1GbE LOM is not equipped on new boards for 7xx2 series processors.”
The adapter used instead is a HPE Ethernet 1Gb 4-port 366FLR adapter. On this FlexLOM NIC is an Intel i350 controller which is a perfectly well-supported NIC. Still, it is different.
Final Words
The change has two impacts. First, if you are an existing HPE ProLiant DL325 Gen10 or DL385 Gen10 customer, you need to be extra careful when ordering your HPE ProLiant DL325 Gen10 systems now, especially pre-configured models. Older systems (e.g. the EPYC 7001 series HPE P04648-B21) will most likely utilize HPE 1 Gb 331i Ethernet adapter. The refreshed versions (e.g. the EPYC 7002 series P16696-B21), especially the Smart Buy configs are likely to have the Intel NIC.
Perhaps more profound is that this now utilizes the FlexLOM slot for lower-value 1GbE networking. We have been using 40GbE HPE FlexLOM adapters in this slot. With the new systems, replacing the HPE 366FLR 1GbE card means that the server no longer has 1GbE networking that we use for hypervisor management and provisioning networks. To get higher-value networking the option is now to add a PCIe network adapter instead of purchasing a FlexLOM upgrade. Effectively that means if you want higher-speed networking, the newer HPE ProLiant DL325 Gen10’s with the Broadcom NIC de-populated has a single available PCIe slot instead of two.
For those that rely solely on 1GbE networking, and have no Broadcom v. Intel preference, this is a complete non-issue. On the other hand, for our lab use, this change makes the systems no longer suitable. As we were installing this in the lab and opened the box to find this configuration, we had a few folks in the data center look. Unanimously this design was not the favored solution instead of having consistent onboard networking since very few shops are running 1GbE for anything other than management networks.
We are, as a result, going to update our HPE ProLiant DL325 Gen10 review to link to this article. HPE discloses this in their spec sheets so it is not an underhanded change by any means. The Intel i350 is a very popular NIC. On the other hand, losing continuity with the HPE 331i / Broadcom adapter from previous systems as well as unnecessarily filling the FlexLOM slot to achieve the same purpose is something we want our readers to be aware of as it has a drastic impact on expandability.
Update 2019-11-04
We have an official response from HPE on this and wanted to share with our readers:
HPE is committed to delivering latest technologies and the best experiences to customers. We introduced new levels of choice and flexibility in network adapters for the HPE ProLiant servers based on customer feedback.
The 4 x 1GbE network interface (LOM), standard on the HPE ProLiant DL325 and HPE ProLiant DL385 Gen 10 servers with the 1st Gen AMD EPYC™ (Naples), was removed from the subsequent variants equipped with the 2nd Gen AMD EPYC™ processors (Rome) that we introduced in August. This allows customers to choose from 1GbE, 10GbE and 25GbE interconnect options without worrying about isolated and unutilized resources.
If a 4 x 1GbE NIC experience is required, two choices are available: an HPE Ethernet 1Gb 4-port FLR Adapter in a FLexibleLOM form factor or an HPE Ethernet 1Gb 4-port PCIe Adapter in a stand-up form factor. Both will deliver the same tested and trusted experience as the now discontinued LOM.
Additional information on the HPE ProLiant DL325 Gen Server, based on 2nd Gen AMD EPYC, can be found here: https://h20195.www2.hpe.com/v2/getdocument.aspx?docname=a00045498enw (Source: HPE)
I can’t disclose a source but HPe (and other manufacturers) get a kickback per SKU sold that includes an Intel chip.
Why the F would they do this? We were going to order a rack of these next week. Thanks STH for finding this. We checked the configs we’re looking at and they have this 366FLR.
HPE’s quality is going to shreds with stuff like this. We’d have bought a bunch of servers without enough PCIe slots for dual NVMe and a decent NIC. It’s like HPE doesn’t get it.
That’s a shady change. I like FlexLOM but i’m not giving up 1G ports for the slot.
I’m shocked they sent you a review sample with the Broadcom populated but they changed production models.
HPE changing NICs on people on the same model? It’s like they’re some cheap Taiwanese vendor
Is this a cost saving move? I’m sorry but I just don’t understand. The loss of an expansion slot for 1Gb networking is dumb. How much can this save? $5 or $10. Maybe the Intel is cheaper but you’re adding another card and labor to install.
Penny wise pound short. After the review we started looking at the dl325. Now maybe not. The extra expansion slot is what we were looking for and it was enough to make use consider HPE over PCIe Gen4 alternatives.
Seems like the same is with DL385 gen10.
NOTE: **Embedded 4x 1GbE Adapter is not equipped on new boards for 7xx2 series processors
It should be a standard.
Wow that’s dumb. Sybreeder I think they’re using the same motherboard on the DL385 so that follows.
I wouldn’t expect this from HPE. We’d been considering them with EPYC 7302P’s but if you’re losing FlexLOM then there’s no point.
I’m going to talk with our team about revisiting with the 2.5″ version of this that STH reviewed that has built-in 10G https://www.servethehome.com/supermicro-as-1014s-wtrt-review-a-1u-amd-epyc-7002-server/
If a 7xx2 series cpu is installed on a model that has 4x1gbe ports- will those 1gbe ports work or not?
Maybe it was removed for incompatibility reasons? It’s doubtful.
There are more PCIe lanes since you get the extra one with Rome (STH had this like 6 months ago confirmed)
So it doesn’t make sense that you’d lose lanes for the 7xx2 series
Be happy you got a free Intel because I would never buy Broadcom for production anymore, their firmware/drivers are responsible for a quarter of all our outages as MSP.
from what i read at HP. The Embedded 4x 1GbE LOM is not supported by the 7xx2 series CPU. Does anyone know what SSD drive will work with the dl325 besides HP drives I need 6 1tb drives.
Chuck, from experience, you are incorrect. I have an older DL325 model with a brand new Rome CPU, and the integrated NIC still functions fine.
Also, any SAS SSDs will work, but you need to get the REV 4.010 or newer caddies. If you search for caddies specifically for the Gen 10 DL325 they will try to scalp you on the big auction site. Leave out Gen10 and search for that revision number instead.