Nvidia has updated its CUDA software platform, adding a programming model designed to simplify GPU management.

Added in what the chip giant claims is its “biggest evolution” since its debut back in 2006, version 13.1 features CUDA Tile. Instead of managing thousands of low-level threads, the addition lets AI developers work in high-level “tiles of data.”

CUDA Tile is effectively a combination of a compiler tool with language features for optimization, potentially making it much easier to write programs for GPUs across large sets of data like arrays, vectors, and tensors.

It creates what essentially are big blocks of data collected by the compiler, which then brings them together to figure out the best way to run the code on a GPU, including mapping it to tensor cores for better performance.

“It's very common in GPU programming to be operating on big bulk tensors and vectors, and arrays of data, and [CUDA Tile] makes it easier to get performance and portability for those types of programs,” explained Stephen Jones, one of the architects behind Nvidia CUDA.

An Nvidia technical blog goes into more detail: “Currently, in SIMT (single instruction, multiple threads) programming, you specify kernels by partitioning data and defining each thread’s path of execution. Using CUDA Tile, you can bring your code up a layer and specify chunks of data called tiles. You specify the mathematical operations to be executed on those tiles, and the compiler and runtime determine the best way to launch that work onto individual threads. The tile model abstracts away the details of using specialized hardware such as tensor cores, and your tile code will be compatible with future GPU architectures.”

CUDA Tile initially works in Python, the “go-to” programming language for AI and data science, and back-end development, according to Stack Overflow.

The decision to start CUDA Tile off in Python wasn’t just based on its proliferation, either. Python developers often leverage libraries like NumPy, which already allow for data types to be specified like matrices and then executed as bulk operations.

“It's just come out in Python, and I think that maps beautifully to the AI development frameworks and so on, things like PyTorch for developers to work with this, even for the frameworks to target,” Jones added.

Nvidia has plans to bring CUDA Tile to other languages, with support for C++ confirmed, though no timeframe for that release was given.

CUDA Tile is supported on Nvidia Blackwell products only, specifically those running compute capability 10.x and 12.x. Nvidia confirmed that future versions of its CUDA platform will, however, add support for more architectures.

Nvidia's efforts to democratize AI development

Upon launching CUDA Tile, Jones said the new addition was designed for “both novices and experts.”

It comes as Nvidia is looking for more ways to make its flagship computing stack more accessible and easier for AI developers to tinker with.

The chip giant recently introduced Grove, an open source Kubernetes API designed to help developers run workloads more efficiently across thousands of GPUs.

Grove offers autoscaling components that turn complex orchestration needs into what are essentially simple Kubernetes pods, enabling them to be scaled together and more efficiently.

Nvidia made Grove available via GitHub, as well as a modular component within its Dynamo platform, which is being used by the big four cloud hyperscalers to help them ease complex orchestration.

Grove, Dynamo, along with the addition of CUDA Tile, form part of Nvidia’s ongoing efforts to try and democratize AI development, amid an influx of interest from both developers and enterprises looking to jump on the growing demand for the technology.

The chip giant is even offering a desktop supercomputer that’s powered by its Blackwell hardware, a move founder Jensen Huang described as “placing an AI computer in the hands of every developer to ignite the next wave of breakthroughs.”

“One of the really exciting things about CUDA, or in any engineering, is when you put something out there, and people use it in 100 ways you never even imagined. And I'm just really excited to see the explosion of ways this is going to get used,” Jones added.