Google launched a cloud-based processing platform aimed at enhancing privacy for AI use cases.

Private AI Compute was developed by the hyperscaler specifically for its Gemini cloud models, leveraging its tensor processing units (TPUs) alongside Titanium, Google's custom hardware system, to offload infrastructure tasks.

According to Google, the compute model uses remote attestation and encryption to connect devices to the Titanium-secured, sealed cloud environment, allowing Gemini models to securely process data within a specialized, protected space.

“This ensures sensitive data processed by Private AI Compute remains accessible only to you and no one else, not even Google,” wrote Amin Vahdat, VP/GM, AI & Infrastructure, Google Cloud.

Under the hood, CPU and TPU workloads utilize an AMD-based hardware trusted execution environment (TEE) that encrypts and isolates memory and computation from the host.

For TPU workloads processing large language model (LLM) training, the platform uses a hardened TPU platform that delivers privacy and security properties comparable to a typical TEE via Titanium Intelligence Enclave.

For attestation of CPU/TPU nodes, a secure communication channel based on the Noise protocol is established, connected to a frontend server where mutual attestation of verified binaries is performed over the connection.

The frontend server then creates a noise and application layer transport security (ALTS)-encrypted channel with other services in the scalable inference pipeline and also model servers operating on the secured TPU platform.

“These services and servers are provisioned only upon successful attestation validation. The overall trust in the Private Al Compute system is established by transitive trust between these servers,” a Google technical brief outlines.

Private AI Compute deployments that require analytics or aggregated insights use privacy-enhancing technologies (PETs) such as confidential federated analytics to ensure that only anonymized statistics, such as differentially private aggregates ‘clouded’ by random noise, are visible to Google.

In confidential federated analytics, raw data is processed by open-source software running within hardware TEEs with a hardware root of trust based on the Titan chip. The privacy was said to ensure confidential federated analytics are transparent and verifiable by external parties, and is secured by the open-source Oak protocol, as paired with AMD SEV-SNP (Secure Encrypted Virtualization – Secure Nested Paging).

Private Al Compute also isolates user data in virtual machines (VMs) to contain vulnerabilities, with CPU workloads hosted in confidential VMs.

In addition, it hardens systems against physical exfiltration with memory encryption and input/output memory management unit (IOMMU) protections.

The system also doesn’t allow privileged access to user data. Clients establish trust with a system endpoint through validating the endpoint server’s identity. Isolation of authentication and authorization is achieved through inference using anonymized tokens.

“The system is designed so that inputs, model inferences, and computations are only kept as long as needed to fulfill the user’s query. Attackers cannot access past data. User data is processed in a protected execution environment at the time of inference request and discarded when the user session is completed,” wrote Google.

“Arbitrary execution (e.g. shell access) is not possible on nodes running in the Private Al Compute system … Frontend services run in a confidential virtual machine … [ensuring] user data protection during processing because the workload in a guest virtual machine is protected from the host and the code is verified via attestation.”

IP binding, meanwhile, removes the ability for a bad actor to link a user’s IP address to a specific query.

The underlying binaries of Private Al Compute are available for review on Google systems such as Android, Private Compute Core, and its machine learning (ML) suite.

Pixel users are able to see when Private Al Compute is being used on their devices via network logs, with the platform powering contextual suggestions and transcription tools as an early use case.

The system has received third-party accreditation from NCC Group.