Enterprises are flocking to containers and Kubernetes to automate more of their application deployments. However, ensuring that the technology’s benefits can be harnessed with minimal risk requires thorough Kubernetes security to consistently protect these deployments. The reality is that container-based virtualized environments in private and public clouds can be just as susceptible to hackers and exploits as more traditional ones. Inadequate container security is increasingly leading to strikes related to ransomware, cryptomining, data theft, and service disruption attacks.

Attackers looking to compromise enterprise assets may also target exploits in Docker and Kubernetes themselves. For example, the recent hack of Tesla’s public cloud — stemming from a Kubernetes exploit — demonstrates this rising threat.

Kubernetes’ Security Challenges

Kubernetes presents unique security considerations due to the hyper-dynamic nature of the containers it orchestrates, including:

• A significant internal traffic increase. With containers deployed dynamically across hosts and even across clouds, internal traffic from communication between these containers drastically increases, and it absolutely needs to be monitored to prevent attacks concealed within this traffic.
• A larger attack surface. Every container environment offers hackers an attack surface and vulnerabilities to exploit. Also, Kubernetes and Docker themselves have to be considered for the attack surfaces they might present.
• Traditional security can’t keep up. Given the rapidly changing nature of the container environment, classic firewalls and other tools/models are unable to provide the security needed. Instead, security automation is required.

Assessing Kubernetes security

You can perform an assessment of how secure your containers are during run-time by asking your Kubernetes team these critical questions:

• Do we possess visibility into Kubernetes pods being deployed (in order to view communications between application pods or clusters)?
• Can we detect nefarious activities within internal container-to-container traffic
• Can we monitor activity within pods and containers and recognize potential exploits?
• Have we looked at Kubernetes cluster access rights to assess the risk of potential insider attacks?

At the same time, it’s also necessary to automate security processes so that they do not present any obstacles for application development and DevOps teams. When it comes to containerized deployments, you’ll want to ask your team in charge of Kubernetes security the following:

• What can we do to ensure an accelerated security approval process that allows developers to rapidly deploy new code into production?
• How can we streamline monitoring and security alerts to ensure our focus is on the most pertinent threats?
• How can we isolate certain containers or network connections within the Kubernetes environment?

Kubernetes: The Basics

Before digging in a little deeper on Kubernetes security, it’s important to review some of its foundational structure and functionality. Kubernetes is a tool for container orchestration that automates container deployment, updates, and monitoring — and has broad support from all major container management and cloud platforms, including Red Hat OpenShift, Docker EE, Rancher, IBM Cloud, AWS EKS, Azure, SUSE CaaS, and Google Cloud.

Some key Kubernetes terminology includes:

• Master node: The server that manages the Kubernetes worker node cluster and the deployment of pods on nodes.
• Worker node: Servers (also known as slaves or minions) that normally run application containers and Kubernetes components like proxies and agents.
• Pods: In Kubernetes, a pod is a unit with its own IP address that can be deployed and addressed. A pod usually contains a single container but can have more.
• Services: A service functions as a proxy to its underlying pods. Requests can be load balanced across replicated pods.
• System components: Managing a Kubernetes cluster requires the use of components such as the API server, Kubelet, and etcd – each of which may be targeted as an attack surface.

Kubernetes Networking

Core to Kubernetes networking is the fact that each pod has a unique routable IP address. Through its network plug-in, Kubernetes handles the routing of internal requests between hosts, ensuring they arrive at the IP address of the correct pod. For external access to these pods it’s crucial to use a service, load balancer, or ingress controller (with Kubernetes again providing internal routing to the right pod).

Pods use a network overlay to communicate with one another, with load balancing and designation network address translation (DNAT) serving to establish accurate connections. Packets can be encapsulated to add headers addressed to the correct destination, with that encapsulation removed when the packet arrives. However, with Kubernetes handling this overlay networking dynamically, the task of monitoring and securing network traffic becomes quite challenging.

Attack Vectors

External attacks can arrive through the network. Internal attacks can be prompted by insiders, and this includes unwitting victims of phishing attacks whose systems are compromised and used by malicious outside parties.

Examples of Kubernetes container attacks:

• Compromised containers. Attackers get into a container by taking advantage of application misconfigurations or vulnerabilities. Once they have a foothold, attackers will test for opportunities to further infiltrate the network, file system, or process controls.
• Unauthorized connections between pods. Once a container is compromised it can try to communicate with other pods to either probe for weaknesses or perform an attack. Some security can be achieved by using Layer 3 network controls to whitelist pod IP addresses. However, if a trusted IP address is compromised, Layer 7 network filtering would be required to recognize an attack utilizing it.
• Exfiltration of data from a pod. Attacks designed to steal sensitive data will often employ a combination of methods, such as the use of a reverse shell in a pod that connects to a command and control server and sends data via network tunneling.

Dangerous Attacks Execute a ‘Kill Chain’

A kill chain is a multiple-stage series of actions taken by attackers, which often results in the most damaging attacks on Kubernetes containers. These actions can occur in rapid succession over just seconds, or they can infiltrate a system slowly over months.

Since so many different resources are used to orchestrate these attacks, detecting the stages of a kill chain requires several layers of security monitoring capabilities. To best detect these events in a production environment, it’s essential to monitor these three attack vectors:

• Network inspection. Effective network monitoring makes it possible to witness attackers’ initial entry via a network connection, subsequent lateral movements as they expand their attacks, and final activity as they steal data.
• Container monitoring. By monitoring syscall, file system, and process activity within each container, you can detect application or system exploits as they occur by recognizing suspicious processes (or attempts at privilege escalation and container breakout).
• Host security. Traditional host or endpoint security tools can help to recognize exploits of the kernel or system resources, but they are only adequate if capable of Kubernetes and container awareness.

Attacks That Target Kubernetes Infrastructure

Attackers may also target Kubernetes resources and deployment tools, such as the Kubernetes API Server or Kublets, in order to gain access to data or control over running pods. For example, by hacking or stealing the API Server token, bad actors can pose as an authorized user with admin privileges and then introduce malicious containers or interfere with other essential applications. (The aforementioned Tesla exploit took advantage of an open Kubernetes console.)