AI researchers at Google have developed VaultGemma, a small-scale AI model specially designed to prevent memorization and potential leakage of specific training data.

With businesses using potentially sensitive data to train or fine tune models for specific use cases, breaches or model manipulation could see threat actors extract underlying training information through carefully crafted prompts.

In a bid to create inherently more secure AI, researchers at Google and their DeepMind counterparts teamed up to create a model designed using a different approach to data privacy preservation: differential privacy.

Existing practices to protect training data include data anonymization, where sensitive information is transformed before training. The researchers, however, employed a technique that adds carefully calibrated mathematical noise during the training process itself to prevent the model from memorizing specific data points, while still allowing it to learn general patterns.

Compared to traditional anonymization, which modifies the data beforehand, differential privacy leaves training data unchanged, instead altering how the model learns from it.

VaultGemma provides what the researchers called “sequence-level” protection, meaning if sensitive information appears in only one training sequence (1,024 tokens), the model essentially cannot access or reproduce that information.

“If information relating to any (potentially private) fact or inference occurs in a single sequence, then VaultGemma essentially does not know that fact: the response to any query will be statistically similar to the result from a model that never trained on the sequence in question," the researchers explained in a blog post. "However, if many training sequences contain information relevant to a particular fact, then in general, VaultGemma will be able to provide that information.”

At one billion parameters, VaultGemma is significantly smaller than leading models, though its size means it’s substantial for many edge devices. In terms of performance, it performs similarly to OpenAI's five-year-old GPT-2 model.

What it lacks in capabilities, VaultGemma represents the largest open model trained with differential privacy from scratch, helping to establish scaling laws to help optimize future, larger privacy-preserving AI models.

Google has been exploring differential privacy for some time, applying the concept to machine learning models back in 2023. According to the research team, previous attempts to bring differential privacy to large-scale language models posed "significant challenges," primarily because it requires much larger batch sizes and more computational resources than traditional training methods.

To overcome these obstacles, the Google and DeepMind researchers developed scaling laws capable of helping model developers determine optimal training configurations for differentially private models. They found that rather than training large models with modest batch sizes, differential privacy works better with smaller models trained using much larger batch sizes.

Its latest model is fully open source, with anyone able to access VaultGemma via repository platforms like Hugging Face and Kaggle.

AI security scope widens

Google’s efforts come as AI and related deployments are emerging as a new attack front for threat actors.

Cybersecurity vendors are already looking for ways to shore up the merging plane.

F5 is among the early movers, having unveiled plans for the $180 million purchase of acquire CalypsoAI, which offers a platform aimed at protecting AI systems, applications, and agents at the inference layer.

Security is also being baked into the hardware the powering AI models, like Nvidia’s GPU Confidential Computing (GPU-CC) feature, which safeguards sensitive data during processing.

However, such an offering hasn’t always worked, with researchers recently uncovering significant security gaps in GPU-CC, including the potential for threat actors to potentially compromise data flowing between confidential virtual machines (VMs) and GPUs. Such flaws have reportedly since been fixed.

Google’s own Gemini CLI AI offering was the subject of a major security flaw uncovered in July that could have allowed hackers to execute arbitrary malicious code on a user’s machine without their knowledge.

Analysts at cybersecurity firm Tracebit discovered that the pen-source AI agent for command line terminals could be circumvented through malicious instructions hidden within seemingly innocent files that when executed would allow an attacker to take access of a developer’s entire terminal.