Engineers and contributors to the Linux operating system developed the Linux container (LXC for short, named for the abbreviation for its system driver) as a means of isolating a namespace for distributed applications in Linux that required a more secure operating environment. Linux admins can use Linux containers to create a layer of separation between the operating system kernel and the application layer.
The Roots of the Linux Container
The application architecture we now know as containerization began with a Linux concept called control groups, or cgroups. This is a prevalent concept today, and still convenient for Linux applications even outside of containerization. Cgroup refers to the clustering of tasks together with their related resources, and the delegation of a virtual namespace and virtual file system to those tasks. This way, resource usage from an application can be limited and tracked.
LXC began as a mechanism for virtualizing control groups, effectively enabling the hosting of virtual machines (VMs) without the overhead of a hypervisor. As the picture above (from IBM) demonstrates, LXC removes the need for the entire Linux OS to be replicated within VMs, reducing density and increasing efficiency.
It also doesn’t unnecessarily burden system administrators, who can quickly spin-up an LXC instance of an application running in an isolated space. Within this space, applications still need access to some resources from the outside world. While the kernel runs the container package, it does not communicate with its contents directly, relying instead upon signals to be sent through the network.
Automation and Configuration of LXC
A configuration file serves as an LXC container’s manifest, explaining how the user ID of the host kernel maps to the user ID within the isolated namespace. Any commands sent inside the container are then run as that user. LXC was intended to be used in such a way that the users of a container can remain completely unprivileged, ensuring that any operations performed there are limited to the container. That having been said, it is indeed feasible to assign a user ID to a container with admin privileges.
The lxc-create command builds the contents of a container from the configuration file from a Linux kernel command line. It can then be launched with the lxc-start command, and stopped with lxc-stop. Communication between the kernel and the container is facilitated by a small runtime package inside the LXC container, called liblxc.
Contrary to the common notion of virtual machine management, containers are essentially designed to be temporary, “ephemeral” entities. They’re instantiated in memory, run for as long as necessary, and removed when completed.
A type of process automation can be achieved by coupling LXC with a configuration management system, such as Salt. This enables a complex network to leverage LXC’s configuration files. Salt goes so far as to replace LXC’s basic command syntax with commands of its own, making it capable of automating the building and deployment of multiple LXC containers using templates.
From LXC to Docker
The concept of leveraging Linux containers for purposes of application portability came about when a PaaS platform provider called dotCloud began using LXC containers as an easy means for transporting preconfigured workloads from customer systems to its cloud-based platform. LXC was convenient up to a point. Docker became the open source project emerging from dotCloud, to improve the automation and distribution of container-based workloads.
Docker added the concept of a public registry from which container images could be downloaded. At first, Docker Inc. was formed to distribute LXC containers through this registry. At the beginning of the platform’s popularity, the company introduced a different container format that was more responsive to automated build commands. This format was built by, and presented to, the open source community and was never proprietary.
Nonetheless, the Docker container format does have competitors, including CoreOS’s rkt (“rocket”) format, as well as LXC containers themselves. Now the Docker container format has been given to an independent agency: the Open Container Initiative, funded by the Linux Foundation.
As a competitive move in 2015, Linux provider Ubuntu began producing LXD (pronounced “lex-dee”), a way to deploy and manage virtualized LXC containers using a small hypervisor. LXC acts as the command line tool, enabling LXD to run almost seamlessly under the hood, while adding facilities for portability and scalability that make the older format more competitive with Docker.