Snowflake announced the private preview of Snowpark Container Services, in a move to extend data programmability and its compute infrastructure to support multiple programming languages, access to third-party software, and enhanced security and governance for hosting of large language models (LLMs) and model training.

The new Container Services provides “further flexibility by generalizing our compute platform. That is the key benefit,” noted Torsten Grabs, senior director of product management at Snowflake. “Now we can run a full stack application end to end, from the bottom of your stack of the data layer all the way up to the UI on Snowflake with the containerized compute support.”

The underlying infrastructure for Snowflake's container services is Kubernetes-based, Grabs noted, which is a popular open-source platform for container management and orchestration. Containers are self-contained software packages that allow programmers to package part or all of an application into a single object.

The Snowpark Container Services allows the containers to be deployed to the Snowflake platform, and then the containerized application can be managed and orchestrated.

Snowflake claims this allows users to run a variety of workloads more securely within Snowflake, including full-stack applications and LLM model training, which expands the scope of Snowpark and provides flexibility in building and deploying applications.

Snowpark is designed for developers to program in their language of choice and then execute, extract, load and transform (ELT) and/or extract, transform and load (ETL) data modeling, data preparation and analytics on Snowflake. The platform already supports SQL, JavaScript, Scala, Java and Python.

With Snowpark Container Services, developers have the flexibility to build in any programming language, beyond the existing supported ones. “Additional programming languages will be supported on those containers and running inside the Snowflake compute boundary in the customer account,” Grabs said.

Developers can also package and distribute their containerized applications to the Snowflake marketplace. “Snowpark Container Services will get vendors (for instance, for machine learning models) the ability to wrap a containerized model as a native application, and distribute to the Snowflake marketplace,” he added.

To accelerate computing and improve software integration for the Snowpark Container Services, Snowflake is partnering with Nvidia to leverage the chip company's GPUs, particularly for machine learning (ML) workloads, and to bring Nvidia AI Enterprise and NeMo LLM framework to the Snowflake Data Cloud.

Nvidia AI Enterprise, which is the software pillar of the vendor’s artificial intelligence (AI) platform, includes over 100 frameworks, pretrained models, and development tools, including PyTorch for training, Nvidia RAPIDS for data science, and Nvidia Triton Inference Server for production AI deployments. The integration with Snowflake's Data Cloud will expand Snowflake's AI and ML capabilities, executives claim.

Snowflake also said it will host and run Nvidia’s NeMo platform in the Data Cloud, which enables users to use their proprietary data on the Snowflake platform to tailor LLMs that power business-specific applications and services, including chatbots, summarization and intelligent search.

Besides Nvidia, Snowflake has partnered with numerous third-party software and application providers, such as Alteryx, Dataiku, SAS and Hex, to enable customers to access these products and solutions within their Snowflake account using Snowpark Container Services.

For example, Snowflake users can run Hex’s Notebooks for analytics and data science, use AI platforms and ML features from Alteryx, Dataiku and SAS to run more advanced AI and ML processing, and manage these data workflows with Astronomer's platform powered by Apache Airflow — all within Snowflake, the vendor touted.

Other partners include AI21 Labs, Amplitude, CARTO, H2O.ai, Kumo AI, Pinecone, RelationalAI and Weights and Biases.

“Last year, we shared our broad vision on bringing the computation to the data and turning Snowflake into an application platform,” said Snowflake SVP of Product Christian Kleinerman. “Snowpark Container Services enables us to accelerate that mission of Snowflake as an application platform for a variety of languages, program models and use cases; LLM being a big part of it, but it's so much broader.”