Project Glasswing is the current poster child for how AI is impacting the ability for cybersecurity platforms to bolster detection and defense, but even the most ardent supporters of using AI to power such efforts are aware of potential pitfalls.

Cisco Chief Product Officer (CPO) Jeetu Patel tackled this issue directly during the vendor’s recent Live event in Las Vegas, asking both Drew Hintz, product security lead at OpenAI, and Anthony Grieco, chief security and trust officer at Cisco, for their three-year “disaster scenario” view on AI’s impact over the cybersecurity ecosystem, with a dramatic painting on what a lack of adoption might mean.

“I think the disaster scenario is that we don't sort of make it over the curve. If we can survive sort of this initial filter, if we get to that three-year mark where the technology is diffused, I think we’ll be in a good shape,” Hintz said, before starkly adding, “but I think in that spirit that along the way, that a lot of things sort of get burned down, that there's this sort of like wildfire clearing event that happens.”

Hintz explained that while “some of the pillars, like OpenAI and Cisco that have built these things up will survive, all the new trees have sort of survived this force clearing, but a lot of” others “might not survive,” pointing to a previous anecdote that his children’s school district was the recent victim of a cybersecurity attack.

“If they don't adopt these things and they don't get their things in order, and then if we see attackers running around with, you know, open-source models that don't have the same guardrails … like it's sort of a race,” Hintz added.

Grieco offered up a similarly blunt worst-case scenario, noting that while the broader market is heading toward a “dynamic” future “driven by what is happening in the moment, detected by agents” and happening at “machine speed,” there is risk.

“Worst case … look, the adoption of technology is never uniform, and there are places where the adversaries get that advantage and they leverage that advantage. This is the tail as old as time. There will be pain, and in the worst-case scenario, there will be pain, and that can be felt through digital systems that are compromised,” Grieco said. “You'll see nations, you'll see criminal organizations, you'll see all of the traditional actors in the adversarial space will take advantage of that capability, and that won't be a fun time.”

However, Grieco did add that “the most likely scenario, I think the broader that we get the adoption, I think the better we're going to have a shot at ending up in a happier place. I do think that that space where we see this uneven distribution of this technology is the thing that we've got to collectively focus on is how do we get folks across that chasm.”

Hintz provided a similar view on a more likely scenario and the importance of collaboration toward that goal. This includes OpenAI’s work with the open-source community and its work with vendor partners to help secure their platforms that can in turn help secure a broader set of organizations.

“But there's going to be some pain. There are going to be some places that don't embrace it, that don't adopt it, they get hit by it, and it's not going to be fun for them,” Hintz said. “And I think that we, as a community, have a responsibility to step in and help these institutions, these organizations that do get burned by it, and help them rebuild, and help them sort of rise out of the ashes in this wildfire to rebuild so they can make it to that three-year mark where everything is significantly more secure.”

Accountability and training

However, one big challenge toward reaching that happier place is in recognizing responsibility when something does go wrong. Patel noted that many organizations are not willing yet to adopt AI-run cybersecurity platforms because they don’t trust those platforms and are not sure how to deal with accountability if something were to go wrong.

Hintz did note that remains a “difficult question to answer,” but that he views a better angle to be toward “blameless postmortems.”

“If something goes wrong, you don't look around and say, ‘who's in trouble for this, who do you point fingers at,’ it's like, ‘how do we prove it, how do we support,’ and I think in these cases everyone has responsibility to improve things,” Hintz explained.

Grieco added that there also needs to be recognition that “humans make mistakes too,” and that this trust challenge of wanting something to be perfect before it’s adopted.

“This is what I worry about, to the point we were raising before, of the adoption curve and how do people lean into this and leverage technology is they're asking for it to be perfection, forgetting that humans are themselves not perfect and make mistakes on a daily basis,” Grieco said.

Hintz added further fuel to this human-AI dichotomy by pointing to one specific challenge in trying to reach this end goal: proper training.

Specifically, Hintz noted that while there are models in place that allow AI-run systems “duke it out inside a simulated environment to see how good they are … the thing that we can’t really measure are some of these sort of real-world, red-team offensive cyber operation-type things.”

Hintz explained that he has run red teams where they battle simulated attackers that are trying to compromise a company.

“But they're just things you can't do as a defender at a company,” Hintz said. “You can't go and sort of like extort employees at your company. You can't go and like destroy a bunch of data. You can't go and like take this real user data and ship it off to the cloud somewhere. You have to do all this in sort of a simulated way. Sure, you can compromise accounts. Sure, you can compromise servers. … But you can't go that last mile that a really mean attacker is going to do in order to test that. I think that's probably one of the limitations that we can't test with models is we get that last sort of exploitation with a really malicious activity.”