Microsoft updated its artificial intelligence (AI) infrastructure and Azure Kubernetes Service (AKS) at Ignite 2023 in a move to optimize every layer of the data center hardware and software stacks that power high-performance AI workloads.

More than a year into what Microsoft Chief Communications Officer Frank Shaw described as a "collective AI transformation," IT teams are experiencing entirely new ways of working and are sending strong signals that AI is thriving across various industries and workflows.

"As we accelerate further into this era of AI, we are rethinking cloud infrastructure," Shaw told reporters last week. These updates fall in line with the cloud hyperscaler's strategy of "taking a systems approach," he said.

Microsoft announced two new custom data center chips designed around a holistic view of hardware and software systems, and both will be available in Azure next year.

Microsoft Azure Maia is an AI-optimized GPU designed to run cloud-based training and inferencing on AI workloads like OpenAI's large language models (LLMs), ChatGPT or GitHub Copilot. And for general purpose workloads, Microsoft is launching Azure Cobalt – a cloud-native chip based on ARM architecture that improves performance, energy efficiency and cost effectiveness for traditional workloads.

The hyperscaler also announced the general availability of Azure Boost, an infrastructure enhancement that moves virtualization processes like networking, storage and host management, which are typically performed by host servers, to hardware and software that the vendor claims is optimized for those processes. As a result, the Azure Boost system improves data center storage and networking performance for the fastest performance rates available on the market, Shaw said.

Specifically, Azure Boost boasts remote storage performance of 12.5 Gb/s throughput, 650,000 input/output operations per second and local storage performance of 17.3 Gb/s throughput with 3.8 million input/output operations per second.

Plus, Microsoft has expanded its collaborations with AI leaders AMD and Nvidia to help customers train, build and deploy AI workloads. Featuring the Instinct MI300X, which is AMD's latest GPU, Microsoft's ND MI300 v5 virtual machines (VMs) are meant to speed up AI workload processing for both high-range AI model training and generative AI (genAI) inferencing.

The cloud hyperscaler also introduced the preview of its NC H100 v5 VM based on a variant of Nvidia's H100 GPU. This VM series is targeted at midrange AI training and genAI inferencing and improves reliability, performance and efficiency, according to Microsoft.

Nvidia's H100 GPU promises higher memory per GPU than previous iterations and, by storing more data per GPU in the VM, IT teams can optimize data processing and workload performance as a whole.

Microsoft also added AI and machine learning (ML) workload capabilities to its AKS container orchestration platform to reduce the need for manual configuration and lower the costs of AI. The platform's new Kubernetes AI toolchain operator automates the deployment of LLMs on AKS by choosing the most appropriately-sized infrastructure for the task.

This update simplifies the process of dividing AI inferencing across multiple lower-GPU-count VMs to increase the amount of cloud regions those workloads can run in. It also decreases wait times for higher-GPU-count VMs and the overall cost of using AI. Microsoft also added preset models with AKS-hosted images to the platform, which is targeted at reducing the time it takes to set up inference services.

These AKS updates are now generally available.