Network switch firmware is a pretty attractive place to put a backdoor — just ask the NSA employees reportedly pictured here opening intercepted boxes of Cisco switch shipments like kids on Christmas morning.

But who's responsible for securing that firmware in bare metal switches, the commodity hardware that many expect will form the backbone for software-defined networks?

It's one of the major network security questions to come out of last week's Black Hat conference in Las Vegas, with implications for the long-term success of SDN as a whole. Software vendors want hardware solutions, switch makers are just shipping bare metal, and fingers are pointing everywhere.

The drama centers around the Open Network Install Environment (ONIE), open source switch firmware from the Facebook-backed Open Compute Project. ONIE ships standard on many bare metal switches as the boot loader for compatible network operating systems.

The problem, as Hellfire Security's Gregory Pickett revealed at Black Hat, is that several of those network operating systems had a loophole, allowing attackers with low-privilege network OS account access to replace ONIE with malicious firmware. And once it's in the firmware, reinstalling the OS won't get rid of it.

Pickett, the SDN security expert who first discovered OpenDaylight's Netdump vulnerability last year, in his talk singled out network OS products from vendors Big Switch Networks, Cumulus, and Mellanox as potentially vulnerable to the ONIE exploit (after notifying the vendors themselves, of course).

The vulnerability is tricky to exploit. You have to already be inside the data center, either physically or through a compromised account, to take advantage of it. But the persistence it offers is scary. Pickett's talk at Black Hat drew the largest crowd I've ever seen for an SDN-related talk at a non-SDN conference.

Big Switch and Cumulus tried to use their blogs to pre-empt the heat before Pickett even released his research.

"The issue is not SDN (or even networking) specific," wrote Big Switch CTO Rob Sherwood. "ONIE provides the same security approach for networking that PXE offers in the server world."

"This issue applies to proprietary vendor switches’ firmware as well," Cumulus CTO Nolan Leak wrote.

In other words: Cisco does it too!

In fact, it turns out Cisco does do it too — the company revealed just this week that hackers have been caught pulling the same firmware swap on its IOS devices.

Here's the thing: when it's a Cisco problem, it's a Cisco problem. Their hardware, their firmware, their network OS, and their profit margins. If something goes wrong, there's one neck to throttle.

Pickett, who believes that SDN is the future of networking, is dismayed about the lack of security accountability he perceives in the space. When we met last week in Las Vegas, the evening before his big presentation, he seemed caught off guard by the defensive reactions that were already emerging from network OS vendors.

"Not one vendor has owned up to their responsibility in this," he told me in an email this week.

That's not just a technology problem — it's a marketing problem. Incumbents including Cisco, which has recently ramped up its focus on security, are happy to point out that they only have one neck. It's one of the strongest cases to make for staying loyal to a large vendor's gear in the face of cheaper, more flexible alternatives.

As those alternatives spread, though, the consequences of weak security accountability could be severe.

"SDN has the potential to turn the entire Internet into a cloud," Pickett wrote in his latest report. "But there is a hole in the middle of it that could easily be filled by the likes of the NSA or, worse yet, nation-states like China.

"Let's not let that happen."