AUSTIN, Texas — If you want to talk about scale, Richard Haigh has a couple of numbers for you.
He says that, while the New York Stock Exchange can pass well more than 5 million transactions a day, sports-betting exchange Paddy Power Betfair — the operation for which he’s global head of reliability and operations — surpasses 100 million.
That makes it a pretty good environment for testing whether OpenStack and software-defined networking (SDN) can survive at scale. Both technologies are at the heart of a new private cloud that Betfair is just launching. The first two applications went live last week, and the next 200 will migrate during the next year-and-a-half, Haigh told SDxCentral.
It’s the culmination of an eight-month planning process — lightning-fast, compared with telecom operators. Haigh told the story in a Thursday morning session at the OpenStack Summit here, after giving SDxCentral a sneak preview the previous afternoon.
Haigh likes to think of it as a DevOps story. Betfair, the result of this year’s £5 billion (US$7.3 billion) merger of Paddy Power and Betfair, had already broken down the walls between developers and operations. That’s the familiar tale of DevOps — but what’s discussed less, Haigh says, is what happens next, when this merged team has to take a mature company through its next growth phases.
That’s what Betfair’s new cloud is really all about, Haigh thinks. To hear him describe it, Betfair’s new cloud will be a test of whether DevOps itself can scale.
The original Betfair, founded 16 years ago, operates just like a stock exchange, acting as a real-time matchmaker to sports bettors. One player offers odds on, say, a football match, and another player takes that bet.
We commonly talk about the financial sector requiring massive, agile networks. Paddy Power Betfair is no different — in fact, it outdoes financial exchanges, partly because it never closes. And just as with financial exchanges, Paddy Power Betfair has attracted bots that conduct high-speed trading. Its peak traffic so far has been 135 million transactions in a day, Haigh said.
Charged with building a greenfield network, Haigh polled analysts about SDN and got a mixed response — interesting, given that every major equipment provider talks about SDN as if it’s a sure thing. Fifty percent of the analysts said it wasn’t mature enough; the other 50 percent said he had to go with SDN, Haigh said.
Haigh eventually gave SDN the thumbs-up, picking Nuage Networks — now a segment of Nokia — as his vendor. The network, based on a leaf/spine architecture, also runs the Red Hat Enterprise Linux OpenStack Platform.
Stability was priority for the new cloud, but Betfair was also enticed by the DevOps promise of agility — letting developers create new applications more quickly. The new cloud was to let the developers just code the network, in a sense. If they want a subnet, they should be able to, figuratively, push a button and get one.
It’s something both halves of the company had been striving for. “They both want to enable the development team to self-serve.”
The test environment alone had 450 hypervisors — more than some vendors’ commercial deployments, Haigh says. All told, he expects to end up running 1,300 hypervisors, across the combined test and production environments.
At that scale, Betfair’s cloud has been a learning experience for Red Hat and Nuage, according to Haigh. Betfair has been developing code along the way, sharing it with the vendors.
“Just last week, I think we published 45 Ansible functions to enable SDN,” he said. (Ansible was acquired by Red Hat last fall.)
Haigh isn’t sharing Betfair’s experience just for bragging rights, though. Betfair has a long history of open source involvement, and Haigh wants to share what’s been learned in this process. “We like the idea that you might have tens of thousands of people around the globe who have the same problems.”
Photo by Kendall Waters, courtesy of the OpenStack Foundation.