The Kubernetes community is extending the reach of the container orchestration platform into the field of machine learning.

Kubeflow is an open source project that supports machine learning stacks on Kubernetes. The project is housed within the Kubernetes project, which is part of the Cloud Native Computing Foundation (CNCF). CNCF is, of course, housed within the Linux Foundation.

Kubeflow includes the JupyterHub platform for creating and managing Jupyter notebook servers that are used by data science and research groups; a Tensorflow Customer Resource for managing compute resources to a specific cluster size; and a Tensorflow Serving container to house the machine learning work.

Container Solutions, which is one of the vendors working on the project, described Kubeflow as a “mashup of Jupyter Hub and Tensorflow.”

“This project marks the beginning of the end of the data scientist and/or software engineer as disparate roles,” said Philip Winder, an engineer and consultant at Container Solutions, in a blog post. “Like DevOps has merged operations and development, DataDevOps will consume data science.”

Kubeflow also takes advantage of the Ksonnet project, which is a configuring application running on Kubernetes. For Kubeflow, Ksonnet is being used to enable the movement of workloads between development, test, and production environments.

Machine learning focus

During its recent re:Invent conference, Amazon Web Services (AWS) launched its SageMaker fully managed, end-to-end machine learning service. During a keynote address, AWS CEO Andy Jassy said the product was designed to ease developer access into machine learning.

“Most companies don’t have expert machine learning practitioners,” Jassy said. “Machine learning is still too complicated. If you want to enable most companies to be able to use machine learning, you have to solve the problem of making it accessible for everyday developers and scientists.”

Google, which birthed the Kubernetes platform, in May unveiled its next-generation Tensor Processing Units (TPUs). Google CEO Sundar Pichai said the product would make Google Cloud Platform (GCP) “the best cloud for machine learning.”

Kubernetes expansion

The Kubeflow project also continues to expanded use of Kubernetes outside of its original container orchestration focus.

Ryan van Wyk, assistant vice president of cloud platform development at AT&T, recently said the carrier plans to use Kubernetes as part of the management layer for its next-generation AT&T Integrated Cloud (AIC) platform set to roll out next year.

The most recent Kubernetes update, which was released last week, included a focus on increasing extensibility of the platform. Eric Chiang, senior engineer at CoreOS, explained in a blog post that the community ran specific tests during the development phase targeted at broader adoption.

“Notably, this data validates the ongoing goal to split the Kubernetes monolithic repository into smaller, more consumable projects,” Chiang explained.