As businesses increasingly look to integrate generative artificial intelligence (AI) into their cybersecurity strategies, CrowdStrike CTO Elia Zaitsev provides insights into the benefits, potential risks and essential questions they should consider.

Zaitsev explained that generative AI is currently being used in various areas to enhance cybersecurity capabilities, spanning from natural language processing (NLP) that allows junior analysts and less technical users to give open-ended requests and tasks, summarization that distills large datasets into concise and relevant information, explainability that simplifies complex security topics for better understanding and action by analysts, to content creation that can help write a response script or offer guidance.

“There are a lot of different exciting applications for this technology. Take that all and stitch it together what we call generative workflows,” he told SDxCentral. “So instead of ask a question and get an answer, ask one task and get an output, chain them.”

Zaitsev said sophisticated generative AI platforms can help analysts move through all the steps for one case from data analysis, and understanding the situation to taking actions, in seconds or minutes instead of hours.

However, Zaitsev pointed out there are potential risks that organizations should be aware of when applying generative AI to their cybersecurity infrastructure.

“From a technical perspective, the underlying risks are no different when you're applying generative AI technologies to security as other domains,” he said. “It comes down to what is the impact of those risks when you're dealing with a security use cases, as opposed to like a consumer talking to an AI system and asking it to create them a diet.”

“If you are asking it to make security decisions, implement policies take action in your environment across your entire enterprise, and you get a hallucination, it's a bit of a bigger deal,” Zaitsev added. “Not only can [generative AI system] be wrong, but they're very confidently wrong.”

He suggests organizations should take very careful steps to deal with the hallucination and ensure the AI systems are auditable and their actions understandable. But, “if you're relying on a black box solution that isn't designed correctly, you don't have that visibility.”

In addition, Zaitsev cautions the potential risks in two ways: the first one is information and data leakage to unauthorized services or misuse in third-party AI model training; the second is prompt injection attacks that are similar to SQL or database injection attacks, where malicious inputs to AI systems can lead to unauthorized or unintended actions.

As cybersecurity vendors increasingly integrate large language models (LLMs) and generative AI capabilities into their products, Zaitsev recommends that customers pose these questions to their vendors:

1. AI system visibility: Where does each question that is asked to these AI systems go? Where and how is the data processed?
2. Data security and privacy: Who has access to the data? Are third-party services involved? If the vendor uses third parties, what are the agreements and controls in place?
3. Handling of hallucinations: What measures and controls are in place to identify and rectify hallucinations like false positives?
4. System auditability and transparency: Is there any input and output validation? How traceable are the AI's decisions? What steps does the AI system take, when it makes a decision, output or recommendations? How much inspection can users add along the way?
5. Data usage: How is customer data used in training the AI models? How are the questions being asked by the end users being used? Are those questions kept private? Will the vendors train new models with the data?
6. Security control bypass:  Are the generative AI systems bypassing cybersecurity controls? Do they operate with the same level of permissions and ability to access sensitive data as the end user? Or do they have superuser privileges that inadvertently bypass controls that have been put in place?