This week, F-Secure Labs published some of their work looking for backdoors in counterfeit Cisco switches. The switches in question were counterfeit Cisco Catalyst 2960-X switches. For those who are unaware, the Cisco Catalyst 2960-X is a very popular switch and the particular model investigated are the Cisco WS-2960X-48TS-L models a mainstay 48x 1GbE + 4x SFP uplink model. Cisco sells these switches for much more than something like a MikroTik CRS354-48G-4S+2Q+RM, and for good reason given their support and reputation. Still, mature technology with high margins means that counterfeiters are in the supply chain.
Fake Cisco Catalyst 2960-X Switches
F-Secure had access to a genuine Catalyst 2960-X model along with two counterfeit models. The counterfeits were discovered when they suddenly stopped working with software updates. See if you can guess which one is real and which is fake based on the below three shots.
Of course, go read the paper (link below) but the one on the left above is the genuine Cisco switch. That requires opening a switch up which few (if any) are going to do. Looking at the exterior of the chassis, there are some subtle differences:
There the markings and LED indicators are off, but only by a small degree. Likewise, below one can see some slight differences as well:
The bottom line here is that it would be difficult to tell which switch is genuine and which is not based on the exterior view. Also, these companies are engineering PCB so getting better at exterior color matching and such is a relatively easier problem to solve.
In the paper, the F-Secure team discusses some of the hardware modifications that were made in an attempt to circumvent hardware/ software checks for validity. Here is the most egregiously obvious example:
That was found on the bottom of the switchboard which is even harder to spot. Realistically, very few take apart a Cisco Catalyst switch to see the bottom of the PCB.
The F-Secure team highlights a few more examples. We wish that they would have had more product photography to see if there were other telltale signs of a counterfeit. The team also goes into detail about how they searched for backdoors in the firmware yet did not find any.
Final Words
If you are interested in this area, the F-Secure report is 32 pages and great. Check it out here. This is also a reason that we are seeing computer manufacturers, components suppliers, and large customers focus on securing the supply chain. We have even seen this at the component level with Counterfeit Intel Xeon E5 CPUs Invade Amazon and Investigating fake Intel i350 network adapters. For a company such as Cisco, having these products in the field that can fail can damage its reputation especially if the customer is unaware they are using a counterfeit device.
While F-Secure did not find backdoors, making this a hardware sale profit-oriented endeavor, it is not out of the question that these can be used for other purposes in the future.
Maybe Cisco themselves are doing this in an effort to thwart reselling of used gear. They simply pollute the supply on eBay and and release a news article such as this, and the problem eventually fixes itself (or so they think)…
@Steve: Are you serious?
I must say i am pretty impressed by the skill of those (reverse) engenering.
Could they be “Ghost shift” / “Third shift” counterfeits?
Remind me of the Intel NIC clones that are working copies. Impressive.
Why would they need to add more backdoors when there are already tons of security bugs in the original Cisco software?
I dunno, if these are so well reverse engineered copies to match performance while offer same packet filtering/firewall features as real Cisco than what are they doing making clones???
@Paul, I imagine that most cost nowadays is in software/firmware development. So even if you built the exact same hardware platform but were able to steal the software, you are way ahead. No recouping the original development cost and don’t have to pay for maintenance teams to fix security issues/bugs. It’s getting where I order from the manufacturer’s website if possible to avoid counterfeit issues, even if the cost is higher than eBay/Amazon.
Amazon can’t even supply me a genuine candle or Fruit of the Loom undershirt these days…
Unfortunately, I’ve had a case like this before. We had bought some used switches from a online dealer and after an extensive test we used the switches for customers who did not want to pay the full price of new Cisco ToR switches.
4 years later one of the switches failed and we got a new one as a replacement and wanted to make a firmware update for the existing switches. We bought the update directly from Cisco and behold, all 7 remaining switches stopped working.
So we called Cisco, who found out that the 48 port switches were 24 ports in a previous life and had an original 48 port housing. Cisco was kind enough to give us two weeks to return the counterfies or they would confiscate the switches. With an agreement with the police and some effort on our part (and the dealer’s), we were able to avoid a lawsuit and have not been a Cisco partner since then …
Timo
“have not been a Cisco partner since then …”
Sounds to me like the issue was buying used from an online reseller – not with Cisco.
Seriously doubt that Cisco threatened to confiscate the switches, or that law enforcement was involved – and doubt there was ever a chance of a lawsuit.
Cool story, though.
Hey Patrick!
Great article. I wanted to share something. My employer purchased some “Cisco” switches about 4 years ago from eBay. The outside of them look suspicious so I took them apart and found EXACTLY what was seen in this article. Was very cool to have it all validated. I didn’t, at the time, have the time to run any debugging on the little bodge chip on the back of the PCB. Cool that someone else did take the time to work it all out. Here’s my reddit thread where we discussed it for quite a while. Unfortunately it’s archived so I can’t post new comments but I did edit the original post with a link to your article for posterity sake. Hope that’s okay. That’s for detailing all of this!! Great work!!
https://www.reddit.com/r/networking/comments/4iwa5f/possible_counterfeit_cisco_equipment_wphotos/
Good job on letting the counterfeiters what they need to fix and to look more authentic.
Nobody opens a switch – why? :D OK, i haven’t repaired a Cisco switch yet, but if i got a faulty one, i would try it :D