Artificial intelligence (AI) is expected to produce increased operational efficiencies but will a new approach toward cloud utilization be needed to manage soaring energy demands?

Analyst firms are predicting the increased use of AI will drive data center power requirements through the roof.

IDC in a recent report projected AI data center energy consumption will surge at a 44.7% compound annual growth rate (CAGR) over the next several years, sucking down 146.2 terawatt-hours of energy by 2027, “with AI workloads consuming a growing portion of total data center electricity use.” The firm also noted that electricity accounted for 46% of total enterprise spend on running their data centers and 60% for service provider data centers.

ABI Research in its own report noted that AI applications and training models are estimated to use up to 20% of the data center electricity.

This surge will require data center operators to rethink how they manage AI traffic, including where in their cloud environments they decide to run certain AI-related workloads.

Induprakas Keri, SVP and GM of hybrid multicloud at Nutanix, explained that a lot of that focus today is on the training of foundational models, which typically happens just once. This requires a large amount of resources, but only for a limited amount of time, which makes it perfect for public cloud systems like Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP).

“However, if you look at the future of energy consumption in AI, a lot more will happen both inside the enterprise and at the edge,” Keri added.

This architectural shift will happen as enterprises look to bolster those foundational models with more specific data to fine tune their AI platforms. This will move from running AI enhancement models within an enterprise firewall to running inferencing at the network edge.

“When you think about the energy consumption across the spectrum, the foundational model is something that you spend energy on once,” Keri explained. “The refinement that happens inside the enterprise and the training and the refinement augmentation, or retrieval-based augmentation that happens several times, maybe 100 times. The inferencing is something that happens a million times, and so while most of the attention today, for example, is focused on it takes 90 megawatts or six months to train a foundational model, the real action will be how do you make inferencing more efficient.”

One path toward gaining edge efficiency is by using cloud-native architectures like containers. This model allows for the sharing of compute and storage resources, with capacity moved to where it’s needed and away from where it’s not required.

“If you can reduce the cost of the inferencing by 30% and you run that model a million times; if you compress the model; if you have more efficient storage, if you're able to turn off cores that are not being used for processing because you're processing just sparse matrix operation; if you have a container platform that's very efficient in terms of resource consumption, all of those allow you to run the inference and engine more efficiently, all of which leads to the better power utilization,” Keri said.

Keri also noted that while it’s important to realize how different cloud architectures support AI initiatives, it's also important to have an open mind toward those different models.

“AI really becomes this modern, almost the hybrid multicloud app that you can't really afford to have religion about,” Keri said. “What you really want to enable is the flow of the model starting to all the way from the foundational model, to the training that happens, to the inference that needs to happen either in the data center or at the edge. And you need to be able to facilitate that model flow and data flow to really facilitate the AI pipeline.”

Keri added that while AI management might be daunting for some, it’s important that enterprises start working toward the challenge. This means having a clear mission statement as to what challenge AI is being used to solve and to start the process as soon as possible, even if it’s just in a limited basis.

“What they want to do is to stand up something quickly enough that allows them to experiment, demonstrate some amount of business value, and then scale based on that,” Keri said. “This is one of the situations where you can't actually have a three-year plan. You have to be pretty agile, and you have to say, ‘OK, I want to accomplish how I can demonstrate, for example, benefit to the support organization in six weeks. And based on that benefit, I'm going to scale up something that is going to apply to my entire 5,000-person support organization.’”

But this all needs to happen with an open cloud architecture mindset that will allow for the most efficient use of resources.

“Don't get religious about this,” Keri concluded. “Work with someone that explicitly understands the need for portability, the need for infrastructure to evolve, and fully understands the end-to-end AI value chain, meaning the need to span private cloud, public cloud, and the edge. I think that's really important. Don't get bogged down by religion.”