Microsoft offered a glimpse inside its $3.3 billion data center project in Mount Pleasant, Wisconsin, providing insight into the network powering what it claims to be the "world’s most powerful data center."
The facility, dubbed Fairwater, is nearing completion. Its three buildings span some 315 acres and will house approximately 500 full-time employees. Microsoft plans to spend a further $4 billion on a similar facility in the state, with other identical data centers under construction across the U.S.
Unlike its existing Azure sites, which are optimized to run smaller, independent workloads, the Fairwater facility is designed to work as a singular AI supercomputer underpinned by a single, flat networking infrastructure that connects multiple pods of racks housing hundreds of thousands of GPUs from Nvidia.
In terms of rack layout, systems deployed in Microsoft’s Wisconsin data center are laid out in a two-story configuration to reduce latency brought on by increased physical distance. That means each rack is networked to adjacent systems, and to those above or below them.
Microsoft claims the wholly interconnected site is capable of 10-times the performance of the world’s fastest supercomputers and will support AI training and inference workloads “at a level never before seen.”
The site is also connected to Azure’s global network via Microsoft’s Wide Area Network (WAN) to support large-scale distributed training across the globe.
“This is a fundamental shift in how we think about AI supercomputers,” Scott Guthrie, Microsoft’s EVP for cloud and AI, wrote in a blog post. “Instead of being limited by the walls of a single facility, we’re building a distributed system where compute, storage, and networking resources are seamlessly pooled and orchestrated across datacenter regions. This means greater resiliency, scalability, and flexibility for customers.”
The Fairwater site has its own entirely separate facility, the size of five football fields in length, simply to store and process the vast amount of data used and generated by the site’s AI cluster.
Each Azure storage solution on-site is capable of sustaining more than 2 million read/write transactions per second – enabling exabyte-scale storage.
Solutions like BlobFuse2, which acts as a virtual file system driver, helps provide low-latency access for GPU node-local training to ensure GPUs in the facility seldom idle.
Some 90% of the facility uses a closed-loop liquid cooling system while the remaining 10% of servers use outdoor air for cooling and some water, but only during hot weather.
Microsoft is also funding a new 250MW solar project in nearby Portage County to match the facility with renewable power purchase agreements (PPAs).
Comments