During his Amazon re:Invent keynote yesterday, Amazon CTO Werner Vogels made a few significant announcements related to developers.
He noted that AWS is launching a series of open source projects under the collective name of Blox, and he announced some enhancements to Lambda that imply that developers are doing increasingly serious work with so-called serverless functions.
Blox would appear to be a different take on what Kubernetes and Mesos are trying to do, in terms of container scheduling and orchestration. It’s actually meant to be a collection of tools. The first two are a cluster state service and a scheduler that can launch daemons for each container you create.
The point is to let developers work without having to care about infrastructure. One of AWS’ prize customers, Netflix, was on stage to tout the new project.
“Blox is aimed exactly at this space that we’re heading to,” said Neil Hunt, Netflix’s chief product officer. “We really don’t want or need to operate all of this infrastructure. We firmly believe it’s best to let AWS do that heavy lifting.”
Netflix began the shift to containers just this year and has developed some of its own software: a container runtime of its own, called Titus, and a scheduler called Fenzo. Parts of Titus will be integrated into Blox, Hunt said.
It’s hard to know what to make of Blox, considering it’s going to include other projects that don’t exist yet. But it’s fair to say that creating an open source project is an unusual step for AWS, which hasn’t had to share its technology to this point. One question will be whether other companies will be willing to contribute to Blox, given that the implication is that all that work will be fueling AWS services.
The scheduler inside Blox “sounds like an approach similar to Mesos, in that it says: Maybe you know better than us how you want to schedule stuff,” says Brian Gracely, director of product strategy for Red Hat. Mesos, unlike Kubernetes, is a two-tier scheduler, a setup that makes it useful for application-specific work.
Red Hat happens to be in the Kubernetes camp — the company was an early contributor to the open source project and uses Kubernetes in its OpenShift platform-as-a-service (PaaS) — and Gracely is quick to point out that developers seem to be siding more with Kubernetes than Mesos.
“We’ve seen how that model plays out in the marketplace. It’s a great approach if you have people who know how to do scheduling, but that’s a hard skill. It’s a CS [computer science] skill, a distributed-engineering skill.”
Lambda Grows Up
It’s noteworthy that AWS announced some enhancements to Lambda as well, and that Vogels saved these for the end of his presentation. AWS customers have been enthusiastic about Lambda, he said, and the announcements imply that the service is being used for more and more sophisticated tasks.
Lambda runs so-called serverless functions. They’re not literally serverless; Lambda lets developers run individual functions on a fraction of a server, rather than having to rent an entire server from AWS.
Vogels announced Lambda@Edge, which makes Lamba available at CloudFront locations — the sites of Amazon’s content delivery network (CDN). This brings Lambda closer to the edge of the network and closer to the user.
The goal is the same as that of a CDN, namely, to shorten the transit times for traffic. As Vogels pointed out, even AWS can’t do anything to improve the speed of light.
Developers are also starting to chain together Lambda functions, either combining them into one service (à la NFV service chaining or the way containers are used) or running jobs in parallel. That means someone or something has to handle issues such as keeping track of state, notifying the user of errors, and auditing all this work to make sure the functions actually ran as planned.
Vogels’ final announcement of the day was AWS Step Functions, a service built to handle all of that work. Step Functions is generally available as of yesterday.
Photo by Jake Hills, via Pexels.