Large language models (LLMs) and generative AI are no doubt the future for enterprise. In time, it’s safe to say that every organization will be using the technologies in some way.

At the same time, many remain wary of the evolving tools: Even if used correctly, they can expose sensitive data and put organizations, their employees and customers at risk.

Confidential computing company Opaque Systems aims to tackle this challenge: The company announced today that it is enhancing its platform with zero trust data clean rooms (DCRs) optimized for Microsoft Azure confidential computing.

These new functionalities will allow separate organizations to securely analyze and collaborate on combined raw sensitive data while maintaining confidentiality and allow organizations to use LLMs without revealing their data.

Opaque will offer a deep dive into these new capabilities at the inaugural Confidential Computing Summit to be held next week (June 29) in San Francisco. The company is co-hosting the event with the Confidential Computing Consortium (CCC), a Linux Foundation project.

“Adoption of LLMs and the usefulness of LLMs can really skyrocket if organizations are able to harness this technology without having to worry about the risk of exposing their data,” Opaque Systems CEO and cofounder Rishabh Poddar told SDxCentral. “Confidential computing I think is the answer.”

Confidential computing and data clean rooms

With confidential computing, computation is performed in a hardware-based, attested Trusted Execution Environment (TEE) that prevents unauthorized access or modification of data while it is in use.

This transcends the typical practice of encrypting data at rest in storage and also in transit across a network, but not while use in memory — which is caused by limitations in conventional computing infrastructure.

“It provides a hardware black box within which you can keep data protected,” said Poddar. “Data remains protected and encrypted at runtime, too.”

Similarly, data clean rooms allow different entities to share data for joint analysis under strictly defined rules. Any personally identifiable information (PII) is compliantly anonymized. This allows different entities to collaborate on combined datasets while ensuring compliance with data laws and regulations.

A variety of use cases then become possible, Podder pointed out. For instance, multiple advertisers and marketers could work together on sensitive data to measure ad campaign effectiveness or personalize consumer targeting. Or, financial institutions can collaborate in detecting fraud and insurers can join forces to identify duplicate claims. All this with each party only able to see the data they directly own.

Data clean rooms and confidential LLMs to enable business insights

The Opaque platform’s zero trust DCRs enable secure, multi-party analytics on fully encrypted confidential data secured in TEEs. Customers can quickly create clean rooms and perform multi-party analytics and AI on data that doesn’t compromise its confidentiality. This also helps ensure that data and insights are only accessible to authorized parties, Podder explained.

The company’s confidential LLM Inference allows organizations to run LLM models within Opaque and know that their queries and data remain private and protected and are not exposed to the model or service provider or used in unauthorized ways.

“This allows organizations to start putting confidential data to use,” said Podder.

Multiple layers of data security — including secure hardware enclaves and cryptographic fortification — are built into the platform protect data against potential cyber-attacks or breaches. The tool also incorporates a policy framework to govern permissions and access to allow confidential and non-confidential data to be analyzed in tandem.

With Opaque, organizations can individually encrypt data, upload it, combine datasets, gain insights and train models, “all while making sure that individual data is not revealed to anyone,” Podder said.

Leveraging LLMs, not fearing them

Several organizations of late have grown wary of generative AI and LLM use within their enterprise. Samsung has gone so far as to ban use of ChatGPT after discovering that workers had leaked sensitive data; several prominent companies including Apple and JPMorgan have followed suit. Verizon has also said that ChatGPT is not accessible from their corporate systems.

The chatbot “can put us at risk of losing control of customer information, source code and more…as a company, we want to safely embrace emerging technology,” the company reported in a public statement.

Podder pointed out that many organizations would like to use LLMs or generative AI but have valid concerns when it comes to data privacy. If they want their employees to leverage the technologies, they have to posit questions and prompts based on their data.

The same issue comes up during training and fine-tuning of models: They need to be fed data; some of which could be public, some of which could be proprietary or sensitive in nature.

“When I’m using an LLM, it puts my data at risk,” said Poddar. “I don't want to expose this to the LLM service provider. Confidential computing helps us realize the full power of generative AI and LLMs. This is how we can still benefit from these technologies without revealing data.”

Building a community around confidential computing

Confidential computing is still an early technology, but a rapidly evolving one. By one estimate, its market size will reach nearly $60 billion by 2028, representing a stunning compound annual growth rate of 62.1%.

But many organizations are still unclear on what exactly the technology is or does — which is what prompted next week’s Confidential Computing Summit.

The event will feature keynotes and sessions with leaders from Microsoft, Intel, VMware, Google Cloud Platform, Fortanix, Meta, Google and others. The goal of the event, Poddar said, is to connect technologists, users and academics to “evangelize” around the technology and discuss use cases.

“Confidential computing is in its early days,” he said. “It’s a very powerful and exciting technology that we as an industry can do more to educate people about.”

CCC outreach chair Nick Vidal, for his part, called the summit a “unique opportunity to explore cutting-edge technology in data privacy and how to secure the confidentiality of data in the cloud.”

Opaque Systems president and cofounder Raluca Ada Popa — who co-founded and co-directs the RISELab and SkyLab at UC Berkeley and helped build the open-source LLM Vicuna — will open the event with a keynote exploring confidential computing’s “immense opportunity” and discussing how it can protect user privacy when interacting with LLMs.

“This enables LLMs to be trained on confidential or proprietary data,” she said, “unlocking the tremendous potential of LLMs.”