Today we are going to discuss a change to server security that is going to make waves in the home lab and secondary markets for servers and components in the future. During our recent Dell EMC PowerEdge C6525 review we briefly mentioned that AMD EPYC CPUs in the system are vendor locked to Dell EMC systems. This is not a Dell-specific concern. We have confirmed that other vendors are supporting the feature behind this. For the large vendors, their platform security teams are pushing to build more secure platforms for their customers, and that is going to have future impacts on the secondary server market and home labs.
In this article, we are going to cover the basics of what is happening. We are going to discuss the motivations, and why this is going to be more common in the future. Finally, we are going to discuss what those in the industry can do to keep the secondary server market operating well. If you work with partners or resellers who dip into used parts bins or even have the potential to purchase grey market CPUs, send them this article or accompanying video. The current market has a large disconnect between what some large customers are asking for, and large vendors are delivering on and what others in the market know is happening.
Accompanying Video
This is an important topic. To ensure that we can cover those who like to read/ skim and those who like to get information via audio, we have an accompanying video:
Feel free to pop open that video on YouTube and check it out or to send to those who prefer not to read.
Background: How we learned this was a “thing”
In 2018 we did a Dell EMC PowerEdge R7415 review and as part of that review, we started our normal process of trying different CPUs in the system. Early in that process, we used an AMD EPYC 7251 CPU, the low-end 8-core model, and noticed something curious. It would not work in our other test systems after.
After a bit of research, we found it was because Dell EMC was vendor locking the chips to Dell systems. We did not know exactly why, but we were told was a security feature. At this point, and even to this day two years later, not every vendor takes advantage of all of the AMD EPYC security features. What that practically means is that what we saw with the Dell EMC system is not what we saw with other systems. For example, we were able to interchangeably use CPUs in Supermicro and Tyan systems, but we could not use those systems once they went into a Dell EMC server.
We found we were not alone. Laboratories, VARs, and other organizations were finding that transferring AMD EPYC CPUs from one vendor’s system to another, was not the simple process it was on the Intel Xeon side. It did not always work.
We knew it was a security feature and thought that most who are buying servers would be informed of this by their sales reps or channel partners. After I personally got a lot of texts, e-mails, instant messaging, and comments on our C6525 video and article, I realized that this actually may be a situation where many people do not know what is going on.
This experience that we had, apparently is one that is not overly common yet. That makes sense because the systems that utilize the enhanced security levels are still largely new, and being used by their first buyers. Also, AMD still has a smaller market share than Intel. A big reason, by the way, that Intel Xeon does not have this issue is that they do not have the security feature that AMD has. Vendors have come out and stated that their AMD EPYC systems are more secure than their Intel Xeon systems, and this behavior is a byproduct of that enhanced security.
Next, we are going to dive into the feature of AMD processors (and what will be more common in future CPUs from other vendors.)
AMD EPYC Secure Processor Platform Secure Boot (PSB)
Let us start with the high-level slide. This is effectively the same slide on the AMD Secure Processor that we saw with the AMD EPYC 7001 series launch, but this is from the EPYC 7002 series. AMD EPYC CPUs may be x86, but they have an embedded Arm Cortex-A5 microcontroller that runs its own OS that is independent of the main system. This AMD Secure Processor is the backbone of AMD’s security push as it provides features such as key management and hardware root of trust for the platform.
AMD spends time patching this solution to make it more secure, but it is generally fairly hard to reach without some extremely low-level access in a system. We are going to come back to the “Enables hardware validated boot” line shortly, but it is important to understand that this secure processor underpins many of AMD’s best security features.
For example, at STH we use EPYC’s Secure Memory Encryption and Secure Encrypted Virtualization heavily. With AMD EPYC, we do not have to manually manage keys. Instead, the ephemeral keys are managed for us by the AMD Secure Processor. This is the basis for what is really the building wave of confidential computing offerings such as Google Cloud Confidential Computing Enabled by AMD EPYC SEV. Intel has its secure boot features and SGX that will be enhanced greatly with Ice Lake Xeons, but for now, AMD has this capability while Intel does not. When big vendors say AMD is more secure, the AMD Secure Processor is a cornerstone of those offerings.
Let us discuss that “Enables hardware validated boot” line. While traditionally CPUs just fire up in whatever platform they are in, AMD has intelligence in their CPU due to the Arm-based AMD Secure Processor. EPYC CPUs are designed to be a bit more intelligent about the platforms they are in, and interact with server platform security to act as this root of trust that would not be possible if they effectively just booted up in any system.
Here is a statement from AMD describing the AMD Platform Secure Boot.
The AMD Platform Secure Boot Feature (PSB) is a mitigation for firmware Advanced Persistent Threats. It is a defense-in-depth feature. PSB extends AMD’s silicon root of trust to protect the OEM’s BIOS. This allows the OEM to establish an unbroken chain of trust from AMD’s silicon root of trust to the OEM’s BIOS using PSB, and then from the OEM’s BIOS to the OS Bootloader using UEFI secure boot. This provides a very powerful defense against remote attackers seeking to embed malware into a platform’s firmware.
An OEM who trusts only their own cryptographically signed BIOS code to run on their platforms will use a PSB enabled motherboard and set one-time-programmable fuses in the processor to bind the processor to the OEM’s firmware code signing key. AMD processors are shipped unlocked from the factory, and can initially be used with any OEM’s motherboard. But once they are used with a motherboard with PSB enabled, the security fuses will be set, and from that point on, that processor can only be used with motherboards that use the same code signing key. (Source: AMD statement to STH)
That is a lot to take in. We asked HPE about this. Their response mirrored what the above was describing. HPE firmware, when a system is first turned on, performs this binding process where the AMD EPYC CPU expects to see HPE signed firmware. If you alter the HPE firmware on the system, the check fails and the system will not work. That means if your HPE motherboard fails, you can replace it and put your CPU in another HPE motherboard with signed HPE firmware. It also means if the server platform’s firmware is not signed by HPE, the processor will see it as evidence of tampering and not work.
Edit: 2020-09-09 – HPE clarified that they are doing this in a different manner than Dell after initially confirming that they were using the AMD PSB feature. After this went live, HPE sent us the following:
HPE does not use the same security technique that Dell is using for a BIOS hardware root of trust. HPE does not burn, fuse, or permanently store our public key into AMD processors which ship with our products. HPE uses a unique approach to authenticate our BIOS and BMC firmware: HPE fuses our hardware – or silicon – root of trust into our own BMC silicon to ensure only authenticated firmware is executed. Thus, while we implement a hardware root of trust for our BIOS and BMC firmware, the processors that ship with our servers are not locked to our platforms. (Source: HPE)
What is at least interesting there is that HPE was initially claiming feature parity with Dell to us, and from the comments on this article were saying they used this feature in sales pitches, but now are saying they are not blowing the eFuses.
Here is where the concern develops, and not necessarily for AMD, the OEM, or most of the initial customer base. Customers want more security. The OEMs want to create a secure hardware environment because that is what their customers want. AMD is implementing an advanced security solution beyond what Intel Xeons have giving the OEMs and end-customers what they want. Effectively, when these are sold as new systems, this is exactly what everyone involved wants.
If everyone is getting what they want, then where is the concern, that is what we are going to cover next.
Do you know whether the flash firmware on the motherboard is protected using AMD secure memory encryption? That would protect against the kind of circumvention measures seen in the fake Cisco routers reported on earlier.
In my opinion, this is a very complicated and wasteful way to solve the problem that the flash memory on the motherboard doesn’t have a write protect jumper. I know that jumpers don’t scale in a data center, but there could still be a way to physically write protect the motherboard flash while the server is in operation.
We’re actually looking at EPYC and HPE because of the security security features they have like this that you mention.
I get why this is something vendors and customers would want. However, I feel that this feature is something that really should be toggled when ordering said system ahead of time. Let the end users that want PSB enabled, order their systems from the factory as such. Otherwise it is left off for the rest of the ecosytsem.
This looks more like giving control of CPU to system vendors.
And it seems like any key leak from the vendor means doom for the “security” of the entire CPUs locked to that vendor? Is there any key revocation process in place?
There needs to be a revocation process! What happens when the big cloud providers are done with the hardware and wants to sell it off to get some return for it? It’s a huge waste and adds to an e-waste problem. Part reuse needs to be an option!
I think I’d rather purchase a TPM than have my CPU locked – and what about BIOS upgrades, if there’s any means to do that then the fuse is security by obscurity (or difficulty).
With a TPM it won’t boot without a password, but worse come to worse you still have all the pieces (and can start over). With a blown fuse there’s a tiny chance of losing an expensive CPU (and no mention of support / insurance in the event of damage).
Do they discount their CPUs because they know you’ll only be buying their servers. Would anyone buy something (home / car / stereo / cellphone) knowing that resale value or transferring ownership had severe limitations.
Why it couldn’t be an EEPROM that required keyboard (maybe even IPMI keyboard) entry of a password to upgrade the BIOS isn’t explained.
Not a fan of this ‘helpful’ feature.
Do you have a second source that AMD is enabling this feature you are calling “Platform Secure Boot”?
Guys the purpose of this was to get the discussion started. All great points on the implications.
do-better-journalism – we have statements about a publicly documented feature from AMD and its partners, and it is a documented feature. I am not sure what you mean here.
Likely lifecycle of these systems are as delivered from the factory – that CPU will live inside that Server for the entirety of it’s life. Someone who purchases the system after 3-4-5 years are paying a small fraction of the original list price – and will get what they pay for.
None of these systems will be upgraded at all during the period they are being used by their original purchaser. None. Buy them completely configured, then run them into the ground over that 3-4-5 year period – sell to some entity that sells used hardware – and forklift in new systems.
Only affects a tiny – 1% of the server market at most – as very few Epycs are being used in the first place, and a fraction of those are Dell.
It does suck for small operations like this who like to test enterprise gear – even limited use due to major architectural flaws like Epyc.
When it is not about money, it is about money.
Having the “fuse” be reversible/replaced with a hardware switch is weaker from a security standpoint: you are no longer protected against an insider threat physically undoing your protection.
I think a lot of people commenting are themselves in the secondary market and bemoaning that it’s made your life harder, but to be quite blunt, you are not the customer. At best you are an afterthought, a way to recoup some value from decommissioning old gear. For the large buyers, the security argument is going to be much stronger than making life easier for recyclers.
From an electrical engineering standpoint, the blowable fuse/circuit can be designed in such a way that various security features that depend on root of trust will not operate, effectively rendering inside attacks moot. Ex – the fuse can completely disable registers needed to read in a vendor security key required for the system to cryptographically sign that it is indeed the system it is saying it is with its hardware root of trust intact (ie, the registers will always report the key is 0 in all bits). The system will still boot (if other security features are disabled, lets assume the hypothetical attacker did so), but it will (Rightfully so) immediately give itself away on the datacenter network as having had its security compromised because it will be incapable of reporting itself with the correct cryptographic key since it is incapable of accessing it, while also being incapable of decrypting any locally stored data (ex in persistent dimms or on persistent storage such as SSDs) which was encrypted using the hardware root of trust key that it can nolonger read. The datacenter and the data on the system is thus secured from the attacker.
@Bob Dobbs
Locking the CPUs to the motherboard destroys much of the recycle value of old servers for parts. This affects any customer who was planning to eventually sell the servers as used surplus when they are no longer needed. Not only do people who buy them after 3 to 5 years pay less because it’s less useful, but the resulting waste means companies selling those machines make less from the surplus auction, which then needs to be taken into account in the original depreciation schedule.
Intel does have a comparable feature, its called Boot Guard, and it was rolled out in 2013 starting with Skylake. The firmware is signed by an OEM key. AMD burns the hash of the certificate corresponding to this key into the CPU, while Intel burns it into the Platform Controller Hub (PCH), which is a separate chip soldered into the motherboard. That’s why you’re able to swap CPUs with no issues for Intel systems.
Logically, the firmware belongs to the motherboard, so its root of trust should be tied to a fixed component on the motherboard, not some interchangeable component like the CPU. Tying it to the CPU results in the problems you encountered and there doesn’t seem to be any advantages in doing so. I’ve yet to figure out why AMD adopted this design.
As a consumer just hearing about this *processor* DRM damages the AMD brand in my eyes. The concept of having one-time-programmable ROM on the CPU that can be written to during normal use is ridiculous. This is a PC, not a game console. And it opens the system up to hackers where they will be able to permanently brick CPUs, with no possibility of recovery.
Until the AMD PSP has been cracked I will be seriously considering Intel for my next PC.
We buy Dell servers new and this along with the memory encryption are big reasons we are shifting some spend to AMD over Intel. I don’t care what happens to chips after we’re done with them.
If this can be done on an EPYC is can be likely done on a Ryzen CPU because they are all based on the same hardware IP (intellectual property) blocks. So once a hacker obtains access to the PSP on a Ryzen, assuming they know how to program the eFuse, they can brick the chip permanently. The only fix would be hardware replacement. Thanks AMD. Not only that, the PSP can be used a way of hiding malware on the system. So is this all really an improvement in system security, or is it an improvement in vendor control? And increased vendor control means higher profits for the vendor.
I don’t see what this provides that can not be achieved with existing technologies. Intel has had Trusted Execution Technology (TXT) for a decade or so (vulnerabilities in the implementation notwithstanding; I would not trust AMD to be immune against such anyway). TXT provides a way for a system to prove to a remote system that it runs the correct software across all levels. Also, with a TPM and measured boot + OS partition encrypted with the hash of the measured boot you can make sure that the system doesn’t boot if any layer (firmware, bootloader, etc.) is tampered with.
I don’t believe for a second that the purpose of this is not anti consumer.
If processor can be swapped to another motherboard of the same vendor then this is not a security feature but rather vendor lock-in.
So, another way to force companies to pay for ransomware makers? Now they could simply blow additional fuse essentially killing the machine.
Both PSP and IME had large security holes in the past, it would be safe to assume that they still have some.
AMD should ensure that power for programming the eFuse can be physically disconnected (via a jumper). No software running on the processor should be able to destroy it, ever. On the Xbox 360 it was possible to remove a resistor to prevent eFuse writes, if Microsoft could do that back in 2006 then AMD can certainly do so now.
This does not seem like a security feature, more like a risk. All of this stuff is closed source and security by obscurity, with the added benefit of destroying the resale value of your CPU. I don’t think customers asked for this. What we really need is to have access to all firmware source code and a way to verify INTERNALLY that the correct firmware is being run (or just reflash from a known good copy at every boot). Relying on a vendor for this seems very risky.
I also don’t see a security benefit in tying the CPU to a single mainboard vendor. It seems to me this feature was mostly demanded by Hardware vendors to the detriment of their customers. It’s also very generous of them to offer recycling (likely even for free, yay), I’m sure they do all this out of the goodness of their heart and concern for your security and the environment, not to take perfectly fine used hardware off the market to be able to sell more new hardware.
If you’re designing this, 15 minutes of thought lets you make the processor reusable.
One example approach: The whole point of a “hardware root of trust” is that it knows some crypto keys. Find yourself in a new system? Erase and regenerate those keys. Which is something you should be able to do anyway. Bingo. You’re reusable.
… and don’t BS me with “simple hardware security” pseudo-arguments, either. Blowable fuses don’t add security when all they do is trigger a massive software/microcode apparatus. A fuse can’t recognize a BIOS signing key; all it can do is inform some code that it needs to do so.
The people who design these things are ALWAYS driven by revenue first. Any actual security they provide is simply a convenient excuse that they can use to convince themselves and/or others that they’re helping.
I don’t know about Dell since we’re a HPE shop but I don’t think HPE’s firmware uses Intel Boot Guard but they use AMD PSB.
Someone at work asked if the Xeon servers use Boot Guard and it was awkward when the HPE guy says nope because it isn’t as secure.
We buy millions of dollars of servers each year and this is a feature we want.
Imo, I can understand the vendor point of view of niche use cases. For vendors, what about having a BIOS feature to disable PSB enablement? For most regular customers just toss a boot warning message about it being disabled, similar to how some BIOS configurations warn about TPM being disabled on boot.
In my opinion this is mostly a vendor lock-in than a security feature, and it makes money for AMD and DELL that will sell more new systems because the seconday market will shrink. At the same time we will see more complete system on the secondary market where you could upgrade your cpu only buying a new one, if it still exists, and the cpu without fuses blown will became precious. Think if you extend this approach to SSD disks, memory and devices in general.
For sure this approach is not eco-friendly and it will not help circular economy. About securiy, we will see: often when closed approach are used and the value of products is relatively high a solution to overcame this limitations is found. We will see.
Now, as a “primary” customer, we should check if this security model will impact in our companies (in some small companies and research center, it will impact whenever you needs to move components for whatever needs). If this is a concern simply don’t buy from vendors who are using lock-in mechanism. It’s something that is already happened in the smartphone market (where some vendors lock-in the bootloader).
As a “secondary” customer… customer of secondary market… potentially this could be a big problem leading to headcaches.
And just like with the printer cartridges, people who manage to bypass this ‘protection’ risk a FELONY lawsuit from AMD just as with Lexmark, due to the DMCA anti-circumvention provision. Yes, a FELONY for wanting to use a discarded CPU in a new motherboard.
The appropriate place for the circuitry is in a separate chip. Per se the CPU is not more or less secure if the board firmware has been hacked.
It is difficult to see thus as other than a means to limit aftermarket sale of Epycs while calling it a security feature. AMD is shooting itself in the foot here: every fool knows the aftermarket is a powerful way to bring in new customers in a market with relatively high cost of entry.
This has been a thing with Dell laptops for quite a while now. The article makes very good points about the second-hand market. I’d like to add to it with some experiences of my own that demonstrate how boneheaded this actually is.
As you’re aware, this type of locking is to support signed firmware. The issue for me is that Dell firmware has intentional backdoors. They have included a method for customers to remove UEFI setup/admin passwords by calling Dell Support, proving ownership, and obtaining a one-time code. While I understand they would rather not replace mainboards because a customer forgot their password, this system introduces multiple serious vulnerabilities.
In the firmware I have examined the algorithm used to generate this code is easily extracted from the system firmware itself. Writing a keygen/code generator is trivial. Additionally I do not believe it would be difficult to talk a Dell support rep into giving a code as needed. This makes UEFI setup passwords useless. Which makes secureboot useless as secureboot keys can be changed from setup(A good thing if you don’t trust the Microsoft key..).. Blows away the whole root of trust on the system.
I’ve contacted Dell on several occasions but they do not think this is a serious issue. On my previous laptops I just removed the UEFI module for this backdoor. On my new ones I’m stuck with this vendor-enforced backdoor.
This goes to show that locking people out of their own hardware is an absolutely terrible idea and can actually reduce security! I wish there was some way to push back against this sort of thing.
Interestingly enough, HPE responded today and are now saying they do not use the AMD PSB eFuses after I asked this last week. Added the statement on how HPE is implementing it differently than Dell, on page 1.
Congrats Patrick, you made it to the #1 tech site in Belgium and the Netherlands. :) I’m glad more people find STH and consider it to be the authority on server related tech news.
https://tweakers.net/nieuws/171956/amd-platform-secure-boot-van-epyc-cpus-is-slecht-voor-markt-2e-hands-servers.html
So this is actually a way to make sure you can never ever trust this server. Because the only way to trust what you run is if all the firmware is open. But your epyc is now fused to only run closed source blobs from your vendor?
Yeah, I’m in agreement with the majority sentiment here. This STINKS of vendor lock-in and money grabbing and it’s one that positively will ruin the third-party/fourth-party used parts market if all that has been said pans out true. And, as usual, IT DOESN’T ACTUALLY MAKE ANYTHING MORE SECURE! Just like the DMCA didn’t stop piracy and KYC/AML hasn’t stopped money laundering, this won’t stop the determined uberhacker from working out how this system works and completely circumventing it.
While I agree wholeheartedly on the motive to improve system security and firmware resilience to malicious threats targeted at the motherboard BIOS, using the CPU as the key to this security by means of non-reversible hardware fusing is a BAD IDEA. Surely, this kind of thing can be done WITHOUT locking the processor to a specific board and system brand and model. Seems like the main chipset would be a better place to do this given that the chipset is soldered to the motherboard whereas the CPU is a normally standardized interchangeable part that can go into any system. This is analogous to the previous issue I took regarding the whole concept of brand-specific processor SKUs that I mentioned in the previous post about “SoC containerization”.
AMD’s development teams have delivered some major advances over Intel as of late and it would be to their detriment to try and set this as a precedent. Of course, I’m saying this while knowing that enterprise buyers really couldn’t give two shits about the non-enterprise buyers and what they want and that all of the manufacturers are pursuing this concept in some form and to some extent. I’ll place blame on BOTH the manufacturers AND the enterprise market buyers for this as I feel they are ALL complicit in the whole “money first, environment second, end users’ wishes not at all” marketing model.
Now, you might be reading this and asking “Okay Stephen, what is it that YOU stand to lose in all this? Are YOU a third/fourth-party used parts vendor?”. Well, to be honest, no, I’m not a used parts vendor. However, I AM an amateur chip collector. It has been one of my hobbies for many years, probably since my days in vo-tech school taking classes for my Diploma in computer technology.
AND I RELY COMPLETELY ON THIRD-PARTY AND FOURTH-PARTY SOURCES FOR THE CHIPS AND PROCESSORS I BUY!
I rely on these so-called “grey market” (not a term I prefer to use) vendors for the parts I buy because it is usually the only way I can obtain otherwise hard to get CPUs and other devices. I don’t have lots of money to buy processors brand-new-in-box when they come out. It’s just too expensive. And being unemployed and broke since the end of January 2009 has done me NO FAVORS in dealing with that.
Currently, I buy my stuff from third and fourth parties on Amazon as it’s the only place that I know of that actually sells their gift cards in the local stores where I live. I have yet to see any eBay gift cards so I don’t use that source. I can often find decent chips for anywhere from as low as $3 to up to $20, some of them quite good parts like a 12-core AMD Opteron 6174 or a 6-core Intel Sandy Bridge Xeon E5-2620 in LGA-2011. These were expensive when new and now, after ten-plus years of waiting, I can buy them dirt cheap. That is the beauty of the third-party/fourth-party used parts marketplace. I can buy the processors I’ve always wanted (and there are many, many more I want that I don’t yet have) without having to look for a job that doesn’t exist, work my ass to death and still not have the money to buy them because, for example, Intel wants some $10,000 or so for a Xeon Platinum CPU or AMD wants over $3,000 for one of their Ryzen Threadrippers. Never mind the Cavium/Marvell Thunder X1s and X2s, the Qualcomm Centriqs OR Ampere’s shiny new Altra chips, all of which are currently in the “unobtanium” category right alongside IBM POWER-anythings and z/-anythings, MIPS-anythings, SPARC-anythings or any other mainframe/supercomputer/server processors you can divine from the ether.
This security “feature” of AMD’s will likely make any newer processors more scarce and much harder to buy in the future as more people catch on to this and start sending perfectly usable hardware straight to the e-waste pile. This will hurt me as an amateur CPU collector because this will mean that I may never be able to buy a specific model or type of processor due to them being unavailable. It’s already hard enough to buy used chips as it is just due to the fact that many of the ones I don’t yet have are now old enough to where there’s a really good chance that I might never be able to buy them on account of rarity due to the whole “scrap it for the gold” craze I’ve seen among electronics “collectors” (really just scrappers but who legally purchased, rather than stole, the items they scrapped). Who knows how much of our technological legacy we are losing to this behavior and attitude. Today’s commonplace blah-blah will be tomorrow’s sexy retro-chic (vintage ’80s Commodore 64, anyone?), BUT ONLY IF WE DON’T SCRAP ALL OF IT AND ONLY IF WE STOP ENGAGING IN DESIGN AND SALES PRACTICES THAT MAKE IT OKAY TO WASTE IT IN THIS WAY!! Not to mention the right-to-repair aspect to all of this…
Personally, I think new technology costs far, far too much. I cannot justify upgrading every year or two just because everybody else is doing so. This “security” move seeks to continue that wasteful model of “Trash your old machine! It’s only two years old and still works just fine, but trash it anyway because WE just released this NEW, MORE ‘SECURE’ one for you! You know you want it so just buy it already! Don’t worry, WE’LL TAKE VERY GOOD CARE OF YOUR OLD ONE when you ship it back to us, AT YOUR EXPENSE of course! HEH HEH HEH”. You enterprise buyers can say what you will, but you are a player in this nightmare. Just remember the little CPU collector guy who’s desperately trying to save a small piece of our technological legacy before it all ends up in the scrap recycler’s furnace.
AMD should realize they are tarnishing their own brand reputation with the inclusion of the PSP and the recent CPU lockdown.
Even though it’s a server CPU that’s affected by the lockdown, stories like this are definitely not well received by the enthusiast and gamer communities and draw attention to such anti-features like the PSP. Knowing that there’s a special processor inside the CPU specifically designed to prevent you from unlocking cores, etc. would NOT be good PR for AMD at all. I am using a Ryzen system right now and I regret buying it, I wish I went with Intel instead. At least the management engine has been cracked, unlike AMD’s AFAIK.
It’s about time we looked into a legal response to this behavior, just as with John Deere farm equipment, it will likely not stop unless fines are imposed or some kind of consumer boycott occurs.
Regarding the CPU lockdown, even Intel wouldn’t do such a thing. Surely isn’t it anti-competitive to lock the CPU to a specific system in this way? What would the EU think about this regarding e-waste and recycling? And I believe in Australia the ACCC would crack down very hard on such shenanigans?
These kind of DRM-like ‘security’ features starting being implemented first with phones and consoles and then has spread throughout the entire industry like cancer.
Many of the features of the AMD PSP could be implemented as hardwired logic, no need for a CPU for that. And thus no chance of malware being able to run undiscovered.
It’s like Orwellian doublespeak, in fact the Platform Security Processor might well be making the entire system less secure. Because we cannot inspect the content of the eFuse ROM how do we know if a state level adversary has placed code in there to weaken the system security?
Note: On the nVidia Tegra platform the eFuse ROM can contain executable code to patch the boot-up process, as Nintendo has done with the Switch console. It’s likely that AMD has such similar functionality.
So the PSP could be cracked, and then CPUs can be eFused with malware before shipping the server, and nobody would know that there’s an easily exploitable vulnerability now present.
I guess one of the real purposes of the PSP is to protect AMD’s security and prevent the user from unlocking disabled cores, boosting clock frequencies, retrieving HDCP keys, etc. on both CPUs and GPUs. So it’s partly to prevent the owner from doing what they want with the hardware.
A PSP or Intel ME related hack involving a SCADA system would not be discovered until it’s too late, with potentially extremely severe consequences. AMD is advertising the processor’s PSP as being a security device that is intended to enhance system security. If such a SCADA hack involving the PSP was to result in loss of life for example, what would AMD’s liability be in such circumstances, where the ‘security device’ itself has enabled the system to be hacked in the first place? Taking into account that the ‘security device’ cannot be disabled by the SCADA operator, so they have no choice to use it.
It doesn’t have to be directly controlling the safety critical SCADA equipment, it can be something as indirect as a contractor’s laptop containing Russian / Chinese PSP malware, and this allowing the secure SCADA network to be breached.
The fact is that because of this PSP and eFuse now processors might be considered storage devices, just like hard drives, of which there is a possibility for malware to be present. This is extremely concerning if you are at risk of state-level actors hacking your system (e.g. electrical / telecoms infrastructure operators).
That is why I believe the PSP and ME should be removed completely. Should that not be possible it should be replaced with a processor that is transparent to its internal operation. Once that is easily auditable by the user, and not trying to deliberately hide it’s internal operation from the owner.
Unlocking the EPYC CPU could be done with some kind of JTAG modchip, but this depends on what kind of JTAG security AMD has implemented.
If you want to know where to start, search GitHub for ‘KaveriPI’, if you unpack AMD BIOSDBG.EXE you can find a complete list of processor registers. This is all from 2015 but the PSP is documented in there.
There’s also a Microsoft Access database which has all the JTAG registers, but I don’t have the time to decode the meaning of the fields… It is likely that things have changed (a lot) since then but it still might be enough for a start.
Should the JTAG interface be protected then some kind of fault injection (possibly laser based, unfortunately) might be required to open it up. I guess some of the eFuse bits can be overwritten, maybe there’s a combination which can remove the lock. An innovative recycling company could work on making a jig to automate this somehow…
Some PSP JTAG stuff here (publicly available material from GitHub in 2015, fair use applies):
41469,3529,164000,164999,”SMU_PSP_efuse_ovr_tried”,,1,0,0,0,50,”0000″,0,
41470,3529,164000,164999,”SMU_PSP_FRA_pass_ld_err”,,1,1,1,0,50,”0001″,0,
41471,3529,164000,164999,”SMU_PSP_FRA_pass_ld_cor”,,1,2,2,0,50,”0002″,0,
41472,3529,164000,164999,”SMU_PSP_efuse_pdmb_aes_dis”,,1,3,3,0,50,”0003″,0,
41473,3529,164000,164999,”SMU_PSP_efuse_pcpu_dis”,,1,4,4,0,50,”0004″,0,
41474,3529,164000,164999,”SMU_PSP_efuse_ccp_cyph_dis”,,1,5,5,0,50,”0005″,0,
41475,3529,164000,164999,”SMU_PSP_efuse_FRA_en”,,1,6,6,0,50,”0006″,0,
41477,3529,164000,164999,”SMU_PSP_efuse_proto”,,1,7,7,0,50,”0007″,0,
41478,3529,164000,164999,”SMU_PSP_efuse_secure”,,1,8,8,0,50,”0008″,0,
41552,2352,164000,164999,”SMU_PSP_hard_resetb”,,1,31,31,0,50,”101F”,0,
41553,2352,164000,164999,”SMU_PSP_early_resetb”,,1,30,30,0,50,”101E”,0,
41554,2352,164000,164999,”SMU_PSP_slv_mbus2_reset”,,1,29,29,0,50,”101D”,0,
41555,2352,164000,164999,”PSP_SCAN_MODE_STICKY”,,1,28,28,0,50,”101C”,0,
41568,2352,164000,164999,”PSP_AEB_307_PCPU_RST_DLY_TDR_en_pclk”,,1,15,15,0,50,”100F”,0,
41569,2352,164000,164999,”PSP_AEB_304_PCPU_FORCE_rst_en_pclk”,,1,14,14,0,50,”100E”,0,
41579,2352,164000,164999,”PSP_Resetn”,,1,8,8,0,50,”1008″,0,
43605,555,164000,164999,”PSP_ENABLE_SPARE”,,0,1,1,0,50,”0001″,0,
43642,555,164000,164999,”PSP_SPARE”,,0,7,14,0,50,”0007″,0,
Search Twitter for “EPYC schematics” and find the post by RetiredEngineer to get the JTAG pads.
From what I know already with some of their old chips is that there is a short period of time after deasserting the reset signal where ‘protected’ JTAG registers may be accessed before the SMU (an older on chip CPU for housekeeping/etc) starts to lock things down. So you can disable the SMU before it gets a chance to shut down access. There is a slim chance that that persists on their newer hardware.
Sorry for the spam, just so outraged at CPU DRM and that reusing old CPUs might become a crime under the DMCA (that’s what blew my top).
@ argb
Oh God, I wasn’t even thinking about SCADA systems until you brought it up! Jesus, that’s even WORSE than the damage this could potentially do to the used parts market! And then there’s the gamers and home HPC nerds that this will also hurt. Because you know the tech industry suffers from a very BAD CASE of “monkey see, monkey do” syndrome. Whatever one company rolls out, EVERYBODY IS GONNA WANT SOME OF! Why stop at servers? LET’S PUT THIS “SECURITY” FEATURE IN ALL OF OUR PROCESSORS!!! Kind of puts my Mom’s classic “cat poo” meme right to shame!
Nothing good is going to come out of this and everything you have said is EXACTLY the narrative of the Right to Repair movement. Everyone from Louis Rossmann on down is saying the same things. But alas, we are all at the mercy of our governments. You know governments don’t like to listen to the citizenry. I guess it’s going to take a nation’s spy agency tearing into the national grid of another before our stupid businesses and government “leaders” finally get a clue that this shit can’t be stopped the way they’re doing it. But it sure is effective at taking our freedom to use our own property the way we see fit. Sorry, but I won’t be subjugated so easily and I hold up the damn Pirate Bay as my proof. DMCA and “digital rights management” HAS NOT ONCE stopped them. And I believe the AMD platform security processor thing ain’t gonna stop an uberhacker any more than a speeding bullet to the head would.
No need to apologize for spamming. I wouldn’t call it that anyway. You’re venting and THAT’S A GOOD THING!! It means you’re not holding it in, allowing it to tear up your insides. In fact, I’m kind of enjoying reading your take on this. For what it’s worth, I saved a full copy of your comments, including the JTAG stuff you put up, just in case the “Powers that Be” decide that you’re a “criminal” to be dealt with and your stuff gets deleted.
I’m with you. There NEEDS to be some way for EVERYBODY to push back on this stuff. I’m worried that most of this won’t get to the proper people, if you get my drift.
@Stephen Exactly – it’s taking away users’ ability to do what they want with hardware that they own (with a small chance of being prosecuted criminally – this is utterly shocking).
At least we need the DMCA’s scope limited to cover copy protection *only*, if there are other reasons for cracking the protection (e.g. for curiosity or to unlock cores or boost the clock frequency), than that should be entirely legal. Just like we can take apart our vehicles and there is no penalty for doing that.
I’ve got a lot more than just the JTAG, everything taken from public sources on the Web. On some of their older chips I have gotten code execution on the SAMU (Security Asset Management Unit), which has now been replaced by the PSP. However the SAMU wasn’t involved in the boot process (on PC CPUs and GPUs), it was just an optional DRM processor which you didn’t have to use if you didn’t want to. It’s quite fun to watch this mini-CPU running your own bare metal code BTW…
BTW The SAMU is an AM32 CPU core which is a heavily modified open-source Lattice Mico32 processor. It’s instruction set is completely different from the LM32 but if you know what to search for and which file to dump with a hex editor, you can find a complete description of all the instructions.
https://www.wired.com/2015/04/dmca-ownership-john-deere/
What’s the post code for a locked down epyc? Suspecting I’ve been dealing with this issue.
This is vendor lock-in at its best. Instead of getting used grey-market cpus you’ll have to pay 5x the price and buy them direct from Dell at a premium (and they’ll likely bump up the prices because they’ll know you have no other choice).
IMHO, this should be investigated as antitrust practice.
How long until a government busybody demands backdoor access to this “secure platform”? How after that until the backdoor gets leaked? I’m writing this on a computer built around an EPYC chip. It will be my last.
Can yout check the Threadripper Pro, it looks like they have the same vendor lock.
Lockdown is a big thing these days, and the excuse is always “security”… in the world of homosapiens there are all kinds of viri which give govs reason to enforce lockdowns… and in the world of computers there are also all kinds of viri which giv vendors reason to enforce lockdowns…
…what many are beginning to realise is that ‘lockdown’ is a politically correct vessel to achieve control.
Back in the old days we understood the need to build natural resiliency against viri, and in the world of computers we understood that everyone on a level playing field benefited from proven technology, not like all the fiasco with custom firmwares making SSDs go pop after 32768 hours lmao.
Intel played a clean game keeping away from lockdowns.
AMD has opened the door to a very messy future, thank you AMD.
My next dog will be named, wait for it, drum roll…
Lockdown.
Easy just don’t buy from Dell, since they are the ones doing it.
Herm… I remember seeing a strange case of the same thing using Dell servers running Windows Server with Intel CPUs. Once I tried to repurpose the server to Unix, the security features kept from booting, and I had to disable all the BIOS security features to run my Unix OS on it. Dell was already practicing vendor lock-in with Microsoft… there was this big uproar about it too… ? So its not only CPU lock-in, it’s also OS lock-in.
Just as I was about to suggest R6525s and R7525s to my company I see this, looks like we will be going Intel and Supermicro instead who thankfully don’t exercise this “”security feature”” (vendor lock).
This only creates more ewaste and is an obvious attempt to kill off the second party and further markets so they can get them to buy new servers. This affects both the first party given that these servers and chips will devalue much faster as well as anyone further down the line who wants to buy them.
Anyone who is trying to make the security argument like above and doesn’t care about the next guy is brainwashed and should seriously start caring about the impact of computing as a whole on the environment. It is obvious this has nothing to do security on the premise alone that it isn’t a motherboard lock, I even expect to see in the future that the lock won’t even be burnt automatically into new CPUs either and you will be forced to buy your CPU from your motherboard/server vendor otherwise it won’t work. Don’t worry, it’s for security!
There is going to be many ways to improve AMD’s platform security that don’t intentionally create ewaste, maybe AMD should start by open sourcing their PSB code too. Or maybe let people completely disable it, like the US government does with Intel ME because “it is a security hole”.
This is not a feature we want, we do resell our hardware here to recoup initial investment costs and without a doubt this will devalue it significantly. Nor will it increase security given that PSB is not open sourced. I’d also really like to see how few instances of servers not having this specific feature has actually allowed attackers access into the system. Given that other Epyc board vendors are not implementing it, I doubt it is even a single one.
Well it turns out that researchers have managed to totally hack the PSP by just reducing the supply voltage to the processor. Even on the latest Zen 3 architecture. Nothing exotic required.
https://arxiv.org/abs/2108.04575
Some interesting stuff in that doc, I wonder why does the PSP need anything to do with WiFi (p. 49)?
0x3d WLAN Umac
0x3e WLAN Imac
0x3f WLAN Bluetooth
0x88 WLAN firmware copied to MPM memory by MFD and then MPM will send this to WLAN
Just imagine if that WLAN firmware was remotely exploitable somehow..
Actually it’s a rebranded MediaTek MT7921K.
Basically:
Computer manufacturer: Hey AMD, lets help making our hardware as useless as possible on second hand market.
AMD: Hi PC manufacturers, lets help making our processors useless on second hand maket
AMD & Computer manufacturer: $$$$ as greed can get….
Everyone else: :-/