Dell EMC Plan to Address iDRACula Vulnerability

After our breakthrough coverage of the iDRACula vulnerability that has the potential to impact tens of millions of servers worldwide, including currently shipping 13th generation Dell EMC PowerEdge models, some immediate steps were outlined. We caught this post over at Reddit on the /r/sysadmin iDRACula thread.

A poster “concretebedlam,” from the Dell EMC iDRAC security team, posted a plan that Dell EMC is taking in the wake of this vulnerability disclosure. The poster also goes into detail why the Dell EMC PowerEdge 14th generation of servers is not vulnerable to iDRACula.

The Dell EMC iDRAC 13th Generation Plan

Here is the plan outlined via Reddit:

Few things we are going to do for 13G:

— as previously noted: immediate release of a new version that fixes the physical security issue. This is very easy to fix and we’ll have an updated release out soon for this.

— the files you’ve identified as persistent writable files, along with a few others we’ve found will all be moved back into the read-only root squashfs. That way, if you run the version with these fixes, there is no way a previous vulnerability can still persist. There were good reasons for them to be writable, but for all of them we’ve found better ways to fix the problems that having them writable solves. This fix and the following fixes are more complicated in that we have to address upgrade/downgrade issues, so will take a little longer to get out. (being circumspect about the exact files on purpose, sorry).

— for a small handful of files that still have to be editable for various reasons, we’ll generate known-good configurations on boot based on the current settings and compare hashes with the ones on the writable file system, overwriting if needed. (This is to save wear on the flash part by not writing data on every boot), but also preventing these exact kind of persistent vulnerabilities.

— we’ll have a security signoff process foreach release to ensure that nobody adds new files that can be exploited. (We’ve made significant progress in this area for each release 12g->13g->14g)

The good news is: None of these vulnerabilities are present in 14G. We have a bootloader that is signed with a keypair burned into the processor. The processor verifies the bootloader before loading it. The bootloader signature checks all of the uboot environment variables, the kernel, and the initramfs. The root file system is based on a dm-verity parition with the root hash stored in the (signature checked-) u-boot environment. We fix the boot EMMC partition to read-only, so you can’t replace the running firmware. We’ve moved most of our daemons to be running as non-root users (a process that continues), especially critical daemons such as the web server. And we’ve overlaid an SELinux policy on the system, and we are continuously updating the internals of the design to work better with SELinux.

So far as I am aware, the 14G IDRAC is the only BMC in the entire industry that simultaneously has full hardware root-of-trust, full SELinux protection and runs network-facing daemons as non-root.

Please email secure (at) dell.com if you ever find anything that you believe to be a security issue. We have a team of dedicated professionals who monitor that address and respond to issues.

(Source: Reddit)

STH’s Take and Final Words

First off, the way that Dell EMC is handling this is commendable. Although any system is potentially prone to attack, Dell EMC has people that are actively on this. Again, they had recreated and verified iDRACula vulnerabilities within 16 hours, overnight in the US.

The PowerEdge 14th Generation discussion is interesting because it discusses the hardware changes that have been made to stop this type of vulnerability. The next step is that the rest of the industry will need to catch up with Dell EMC’s lead with the PowerEdge 14th Generation servers. Dell EMC has a team that works on firmware security. Other small vendors have perhaps a resource working on this and do not have the hardware root of trust capabilities shipping in their products.

Key Resources on iDRACula

Here are a few key resources on iDRACula from STH:

• iDRACula Vulnerability Impacts Millions of Legacy Dell EMC Servers
• Broader Implications of iDRACula Vulnerability a Perspective