Distributed inference at scale

Dave McCarthy, Research Vice President for Cloud and Infrastructure Services at IDC, joins SDxCentral’s Kat Sullivan to discuss how the AI cloud stack is evolving as companies move from model training to distributed inference at scale. Key discussion points include:

  • Why the industry is shifting from large-scale training infrastructure to distributed inference environments
  • How agentic AI and inference workloads are driving demand for scale-out, distributed compute architectures
  • The trade-offs companies face when distributing workloads across cloud, edge, and data center environments
  • How cost, performance, security, and digital sovereignty considerations are shaping AI infrastructure strategy