Processor designer Arm Holdings is integrating its compute platform directly into a new GitHub Copilot extension in a move designed to benefit the 20 million developers building on Arm using Github.

The integration uses Copilot’s artificial intelligence (AI)-powered code suggestions to help developers write, test, and optimize code more efficiently on Arm's development platform. The Arm Extension for GitHub Copilot will become available by the end of the year as part of the GitHub Marketplace.

Arm is most well-known as a provider of silicon IP and custom systems-on-chips inside of billions of devices. Its business model involves IP design and licensing rather than manufacturing and selling of actual semiconductor chips.

Arm also produces software development tools for AI applications, focusing on performance, efficiency, and ease of deployment across various Arm-based platforms.

Arm Kleidi and KleidiAI: centralizing development Arm Kleidi is a new suite of tools designed to centralize AI software development across different frameworks (such as TensorFlow, PyTorch, and ONNX) and hardware platforms. It aims to simplify the process of optimizing and deploying AI models on Arm devices.

KleidiAI, a key component of Arm Kleidi, is designed to accelerate AI workloads on Arm CPUs. It achieves this by providing developers with compute kernels optimized for Arm's architecture, including features like NEON, SVE2, and SME2.

Q&A with Arm SVP of AI Alex Spinelli SDxCentral sat down with Alex Spinelli, SVP of AI and developer platforms and services at Arm, to discuss the developments.

SDxCentral: Can you outline what Arm brings in terms of benefits for AI developers?

Alex Spinelli: Arm’s focus in AI development is centered around empowering developers to create exceptional user experiences by democratizing machine learning (ML). We do this by ensuring that machine learning can run efficiently across a wide range of devices, from mobile platforms to cloud infrastructures. Our efforts revolve around optimization using what we call “micro kernels,” which we've integrated into major ML frameworks like TensorFlow and PyTorch. This allows developers to get the best performance possible from their models, regardless of the hardware they’re deployed on.

SDxCentral: How does KleidiAI help developers optimize AI models?

Spinelli: Kleidi is a set of optimization tools, particularly micro kernels, that help ensure that ML models perform optimally on Arm CPU architectures. We've integrated these optimizations directly into popular frameworks like TensorFlow and PyTorch so that developers can take advantage of them without extra work. By optimizing how models run on CPUs and ensuring that they map efficiently to the hardware, we help reduce the overall complexity for developers. This is especially important for mobile platforms, where efficiency is critical due to the limited resources available.

SDxCentral: How does Arm’s AI development strategy improve CPU efficiency, particularly on mobile platforms?

Spinelli: Our approach is to maximize CPU efficiency by embedding deep optimizations within ML frameworks. By working with platforms like TensorFlow and PyTorch, we’ve embedded optimizations that allow models to run smoothly even on devices with limited processing power, such as mobile phones. This ensures that developers don’t have to rework their models when deploying them on mobile platforms—they can trust that performance will be strong across a wide range of devices.

SDxCentral: As a developer, do I need to make any significant adjustments to use Arm’s platform for AI development?

Spinelli: The good news is that developers don’t have to make any significant adjustments. The necessary optimizations are already baked into the latest versions of popular ML frameworks like TensorFlow and PyTorch. As long as you’re using the latest versions, you’ll benefit from these enhancements automatically.

SDxCentral: How does Arm’s solution integrate with other large language models such as Hugging Face?

Spinelli: The majority of models used today are based on frameworks like TensorFlow and PyTorch, which we have deeply optimized. Hugging Face, for example, uses many of these underlying models so developers using it will also benefit from our optimizations as long as they’re working within those popular frameworks. Our focus is to ensure that developers have the broadest possible support and performance across their AI deployments.

SDxCentral: How does Arm’s AI development roadmap look?

Spinelli: Our roadmap includes continuing to expand Kleidi optimizations to cover even more devices and platforms. We’re committed to making ML and AI easier for developers, working closely with frameworks, OEMs, and cloud providers to ensure that performance is maximized across various environments.

SDxCentral: Does Arm offer tools to assist developers in writing AI code?

Spinelli: While our current efforts focus on optimizing how models run across different platforms, we aren’t specifically targeting code assistance tools for AI development at this point. However, if developers are using devices running Arm systems with the latest frameworks, they’ll benefit from our optimizations indirectly. We're laying the foundation for a broader ecosystem that includes improved performance for various AI tools, including code assistance in the future.

SDxCentral: What advice would you give developers and IT managers using Arm’s AI tools?

Spinelli: Developers and IT managers should ensure they are using the latest versions of frameworks like TensorFlow and PyTorch to take full advantage of our optimizations. Also, consider the diversity of devices in your deployment strategy, from mobile to cloud, and leverage these optimizations to get the best performance. Arm is focused on simplifying AI development and we will continue to expand our support to make this technology accessible and efficient across more devices.

SDxCentral: What is the next step for developers looking to start with Arm’s AI solutions?

Spinelli: Download the latest versions of the AI frameworks you use, such as TensorFlow or PyTorch, and you’ll automatically benefit from the optimizations we’ve integrated. Going forward, we’ll continue to expand the platform and introduce more features to make AI development even easier.