Microsoft teamed up with OpenAI to publish new research that sheds light on the ways in which threat actors are harnessing the power of generative artificial intelligence (genAI) and large language models (LLMs) to augment their ongoing cyberattack operations.

The two companies shared that they have detected and disrupted attempts by adversaries tracked by Microsoft, such as Forest Blizzard, Emerald Sleet, Crimson Sandstorm, Charcoal Typhoon and Salmon Typhoon, leveraging LLMs and AI tools like ChatGPT to refine and enhance their cyber offensive capabilities. The research focused on identified activity associated with known threat actors, including prompt injections, attempted misuse of LLMs and fraud.

“This important research exposes incremental early moves we observe these well-known threat actors taking around AI, and notes how we blocked their activity to protect AI platforms and users,” Vasu Jakkal, corporate VP of security, compliance, identity and management at Microsoft, wrote in a blog post.

“Cybercriminals and state-sponsored actors are looking to AI, including LLMs, to enhance their productivity and take advantage of platforms that can further their objectives and attack techniques,” she added.

Microsoft and OpenAI detailed in their blogs how threat actors use OpenAI services for querying open-source information, translating, finding coding errors and running basic coding tasks:

• Charcoal Typhoon used AI services to dig into various companies and cybersecurity tools, debug code, craft scripts, and create content likely for phishing attacks.
• Salmon Typhoon turned to AI for the translation of technical papers, mining publicly available information on multiple intelligence agencies and regional threat actors, aiding in coding, and research techniques to conceal processes within systems.
• Crimson Sandstorm sought AI assistance in scripting for application and web development, generating content likely for spear-phishing campaigns and investigating common ways for malware to slip past detection measures.
• Emerald Sleet exploited AI capabilities to identify experts and organizations dealing with defense matters in the Asia-Pacific region, learn about known vulnerabilities, support basic scripting needs, and prepare drafts likely for phishing campaigns.
• Forest Blizzard primarily used AI services for open-source research into satellite communication protocols and radar imaging tech, alongside scripting assistance.

Notably, the two companies report that they haven’t found particularly novel or unique AI-enabled attack or abuse techniques due to threat actors’ usage of AI.

“Although threat actors’ motives and sophistication vary, they share common tasks when deploying attacks,” Jakkal said. “These include reconnaissance, such as researching potential victims’ industries, locations, and relationships; coding, including improving software scripts and malware development; and assistance with learning and using both human and machine languages.”

Microsoft and OpenAI extended the use of AI by hackers research to map and classify LLM-themed tactics, techniques, and procedures (TTPs) into the MITRE ATT&CK framework and MITRE ATLAS knowledgebase.

These TTPs include the following:

• Intelligence gathering: Deploying LLMs to gather actionable intelligence on technologies and potential vulnerabilities.
• Scripting capabilities: Use LLMs to generate or refine scripts that could be used in cyberattacks, or for basic scripting tasks.
• Malware development: Leverage LLMs in the development lifecycle of malicious tools and programs like malware.
• Social engineering: LLMs are being harnessed to assist with translations and communication.
• Vulnerability research: Use LLMs to understand and identify potential vulnerabilities in software and systems.
• Payload crafting: Employing LLMs to create and refine payloads for deployment in cyberattacks.
• Anomaly detection evasion: Use LLMs to help develop methods that help malicious activities blend in with normal behavior or traffic to evade detection systems.
• Security feature bypass: By leveraging LLMs, attackers find ways to circumvent security features like two-factor authentication.
• Resource development: Leveraging LLMs in tool development, tool modifications, and strategic operational planning.