Keynotes at last week’s Hot Interconnect conference were very much focused on the data center — not just the philosophizing by Tom Edsall, CTO of Insieme, but also overviews of the Google B4 network and the Open Compute Project‘s newfound networking obsession, which had also been presented recently at the Open Networking Summit and Interop, respectively.
Another presentation described a different data-center architecture, one modeled around optical networks, the sandbox that Plexxi has been playing in. This, too, was presented before — you can read the basics in Wired — but I thought the details made it worth reporting here.
George Papen, who’s leading a U.C. San Diego team doing this research, presented the idea. He’s been calling it a hybrid network, for its combined use of optical circuit switching and traditional packet switching. It’s intended for the large, “scale-out” data centers, as a way to address scale: to cut down on the number of optical transceivers needed while also letting some traffic cut through the network more directly.
Playing the Data Center Circuit
“Circuit switching” refers to a network connection where the route stays intact during the session, as in a traditional telephone call. A packet network, by contrast, sends packets along different paths depending on the routers’ moods.
Scheduling those circuits in a virtualized data-center network would be a monumental task, and that’s where the hybrid model comes in: The network operator can get away with scheduling just a portion of the traffic. The circuit-scheduling algorithm doesn’t have to be perfect or comprehensive. It can be shortened depending on how much time you want the network to spend on it.
“Whatever is left over, that you didn’t get right, you send over the packet network,” Papen said.
The reason you can get away with that is because certain connections persist — that is, when you watch individual processors, you find they spend long amounts of time talking to exactly one other processor, Papen said. In other words, there are correlations in the noise (it’s a very “80/20 rule” situation), and the correlations resemble what you’d see in high-performance computing.
So, the correlated traffic can be connected in circuits, and the rest of the traffic can continue to take its chances on the packet network. You can even substitute “high-priority” for “correlated.”
That’s the trick. Optical circuit switching can be used to overcome the scale of the data center, while the hybrid, partial-scheduling model prevents the circuit switching itself from becoming too burdensome.
Papen and his team have demonstrated switching times of 11.5 microseconds, which is fast enough to suggest the idea doesn’t have to be confined to the data-center core; it can work further toward the edge. Whether that means it can apply to top-of-rack switching “is an active research issue,” Papen said.
Rings Around the Data Center
The UCSD prototype, called the Microsecond Optical Research Datacenter Interconnect Architecture (Mordia), consists of six optical nodes connected in a ring. Each node has four endpoints. Using wavelength-selective switches — gizmos that allow particular wavelengths of light to pass through — each node picks the wavelengths intended for its endpoints; the other wavelengths keep traveling down the ring to the next node.
Physically, it’s a ring, but the whole setup forms a logical mesh. It’s an alternative to using a commercial optical switch — the problem being that such systems have switching times in the milliseconds, too slow for most uses.