Cloudflare handles about 20% of the world’s web traffic, but the details of its extensive backbone remained undisclosed, until now. The vendor today unveiled its machine learning (ML)-based Traffic Manager while explaining how Cloudflare connects between data centers worldwide, moves around traffic and determines the most efficient route across the public internet or its private fiber networks.

The vendor, known for its content delivery network (CDN) and diverse security services, claims it operates in 300 cities worldwide, interconnecting with over 12,500 networks, including major internet and cloud service providers and enterprises.

To support this expansive global network, Cloudflare has established a large network of dedicated backbone connections, using both the public internet and private fiber optics to achieve reliable connectivity between different points across the world.

Cloudflare CTO John Graham-Cumming told SDxCentral that the vendor has “a large network of fiber optics, which allows us to have traffic over our very reliable connections.”

In operating this large-scale network, challenges like equipment failure and data center downtime inevitably occur. “When that happens, we may not have enough attendants to serve every person going through security in every location,” the Cloudflare team wrote in a blog post.

Previously, Cloudflare's network engineers manually managed such disruptions, modifying anycast routes to divert user traffic to alternate data centers.

“It was a real pain, and not only was it manual, but it was very crude because we couldn't move traffic around so easily, so you've got large blocks of traffic being moved around,” Graham-Cumming said. “Our network has reached a size and scale where humans can't run it successfully. We need machines to help us.”

The manual work was not only a burden to the network operations team, but it also resulted in a subpar experience for the customers, as the engineers needed to take time to diagnose and re-route traffic. That’s why they developed a tool called Traffic Manager to manage traffic coming into the Cloudflare network.

Cloudflare’s Traffic Manager was designed to balance supply and demand seamlessly across its global network.

The tool's artificial intelligence (AI) and ML capabilities help Traffic Manager automatically detect data center user access troubles, and withdraw anycast routes from the data center until users no longer see issues. Once it receives notification that the impacted data center can absorb traffic again, it puts the anycast routes back.

“Because Traffic Manager is plugged into the user experience, it is a fundamental component of the Cloudflare network: it keeps our products online and ensures that they’re as fast and reliable as they can be,” the team explained.

“It’s our real-time load balancer, helping to keep our products fast by only shifting necessary traffic away from data centers that are having issues. Because less traffic gets moved, our products and services stay fast.”

Based on the team's rough estimate, using the service could save hours per day. Using Traffic Manager can be the difference between taking seconds versus minutes to solve a problem, eliminating downtime for the end user. Graham-Cumming said user downtime is "just not acceptable."

To augment the efficiency of Traffic Manager, Cloudflare has introduced the Traffic Predictor, a tool designed to predict traffic flow shifts based on real-world tests.

Every time Cloudflare adds a new data center or a new peering session, the distribution of traffic changes. Plus, the vendor has 12,500 peering sessions across more than 300 cities, so it’s difficult for a human to keep track of or predict how the traffic will move around the network.

The Traffic Predictor is designed to carry out an ongoing series of real-world tests to check where traffic actually moves. This testing system simulates removing a data center from service and measuring where traffic would go if that data center wasn’t serving traffic, according to the team.

“We're able to take our own probing of the internet so we know where stuff will move. And then we use the machine learning to say: If this thing were to happen, the machine learning algorithm can predict what the new load would be somewhere else,” Graham-Cumming said.

Traffic Predictor also allows the Cloudflare team to preconfigure Traffic Manager policies to move requests out of failover data centers to “prevent a thundering-herd scenario: where a sudden influx of requests can cause failures in a second data center if the first one has issues.”

“With Traffic Predictor, Traffic Manager doesn’t just move traffic out of one data center when that one fails, but it also proactively moves traffic out of other data centers to ensure a seamless continuation of service,” the team wrote.