Cloud

Insights

The CNCF project categories (Table 1) move up the stack from runtime to observability and analysis, with intermediate layers build upon each other for a complete cloud native development and production environment.

Table 1: CNCF project categories

Source: Infosys

The CNCF landscape is a valuable reference for end users to add capabilities to their platforms.

The following use cases address runtime and Wasm projects, comprising both CNCF and non-CNCF projects.

This document also looks at the WasmEdge (sandbox) project that is part of the Wasm category and offers a Wasm container runtime. This project is already adopted by the Docker+Wasm Technical Preview product from Docker.

The cloud native glossary, an open-source repository where anyone can contribute, describes these concepts in detail.

About CNCF landscape runtime

The runtime category covers foundational technologies on which Kubernetes and other higher-level abstractions are built. It is divided into container runtime, container native network, and container native storage.

Given the foundational nature of these technologies, a technology stack will include both CNCF and non-CNCF projects. For example, a typical stack may utilize Docker (non-CNCF), Kubernetes (CNCF) with containerd (CNCF), and cri-o for container runtimes. In addition, both networking and storage have multiple proprietary and non-proprietary implementations, implying the runtime of any cloud native program at this stage will comprise a mix of CNCF and non-CNCF projects.

Different implementations of Kubernetes can play well together as a part of a technology stack if they are compliant with common standards, such as container network interface (CNCI, CNCF), open container initiative (OCI, non-CNCF), and container storage interface (CSI, non-CNCF).

Container runtime

Docker (non-CNCF) is the most popular tool to build container images, both locally and in the pipeline. Teams using Red Hat tooling look at Podman (Red Hat sponsored and open-source) as a drop-in replacement for Docker. Given most image builds are done via pipeline, we can also utilize a build image such as Kaniko to avoid creating a container with root permissions (required for Docker).

All the container tools above adhere to the OCI specifications. So, teams can use a different tool such as Docker to build containers and then run them on Kubernetes with cri-o as the container runtime.

Table 1: Container tooling

Both containerd (G) and cri-o (I) enable running workloads on Kubernetes nodes. They are lightweight and optimized to work with Kubernetes clusters. They are the defaults in Kubernetes starting v1.24 and conformant to OCI specifications.

Figure 1: Cri-o architecture

Containerd has several clients for different purposes (as shown in Table 1). The most user friendly is nerdctl for general use, while crictl and ctr are for debugging. Client libraries are also available for scripting.

Cloud native network

A microservice architecture requires a reliable and fully functional networking layer. This section of the CNCF landscape has projects catering to standards such as Container Network Interface (CNI) (I) and implementations such as Cilium (I).

The CNI project defines a standard to configure networking in containers. Kubernetes CNI plugins conform to this standard, but vendors may provide additional capabilities.

When setting up Kubernetes clusters on the cloud, use the well-tested and supported CNI plugin provided by the cloud vendor. For on-premises Kubernetes, use the default plugin supported by the vendor of the Kubernetes distribution you are using (e.g., Red Hat for OpenShift, SUSE for Rancher, VMWare for Tanzu Kubernetes Grid).

Cilium (I) is an eBPF-based CNI implementation that allows sandboxed programs to run in the Linux kernel with security precautions enabled. It is used for observability, networking, security, and other Kernel capabilities. Earlier, kernel capabilities were extended using root-required kernel modules with limitations.

Figure 2: eBPF use cases

Cilium is a large CNCF project that uses eBPF to provide a wide range of networking and security features for Kubernetes.

It offers a CNI plugin, network policies, observability, service mesh, security features, and a separate UI dashboard called Hubble. It was the first service mesh to run without sidecars, offering easy upgrades, reduced resource consumption, and other such benefits. Istio (I) is now developing a sidecar-less model, also known as Ambient Mesh.

Figure 3: Cilium capabilities

Cloud native storage

Kubernetes was originally designed for stateless workloads and non-critical stateful applications such as caches requiring less reliability. This is because Kubernetes concepts, such as ephemeral pods and unplanned reschedules, are not ideal for traditional databases. Kubernetes also lacked the ability to tune and access storage at an OS level, which is necessary for tuning traditional databases such as PostgreSQL.

Cloud native database offerings, such as Cockroach DB, have re-architected their products to deal with these limitations. Traditional database projects, such as PostgreSQL and MySQL, are also catching up by offering operators for horizontal scaling and the ability to run in Kubernetes clusters. This means that lower-level storage must also be able to scale and be reliable enough to support these stateful workloads.

CNCF projects Rook (G), Longhorn (I) and CubeFS (I) offer storage that works closely with Kubernetes and typically utilizes node storage for fast, reliable, redundant storage allocation, reclamation, and maintenance.

For any cluster running stateful workloads, selecting a cloud-native storage provider is critical. Network File System (NFS) is not recommended for production-level clusters, even if companies have good infrastructure, including backups and network optimization. The first recommendation is to use storage options integrated with the cloud or on-premises vendor, as they are well-supported and thoroughly tested with the platform.

While choosing an open-source storage provider for Kubernetes, choose one of the above options for better control and tuning.

Notable sandbox entries

This section explores sandbox projects with momentum or interesting concepts.

K8up (S) is in the same space as Velero (non-CNCF). It resides as an operator in Kubernetes and backs up storage from within a given namespace. This includes persistent volumes (PVs) with the data stored by applications running in that namespace. The underlying program is restic, a standard, battle-tested open-source tool for incremental backups and snapshots.

WasmEdge Runtime (S) is a high-performance, sandboxed runtime for WebAssembly programs. Wasm is a decade-old technology designed for browsers. However, the properties of a lightweight runtime for browsers, including security and performance, translate well to a server environment. Thus, Wasm is now leading a renaissance on the sever side with different languages such as Rust and Go that can be compiled to and run as Wasm.

WasmEdge runtime is a WASI (WebAssembly System Interface, non-CNCF) compliant engine and intended to run on the server side, including in resource-constrained edge devices. This project is already being used by customers as part of the Docker+Wasm technical preview.

Submariner (S) is a multicluster networking solution that allows multiple Kubernetes clusters to communicate with each other. It can help organizations isolate resources from different teams, but it can also lead to problems. For example, pods in one cluster may need to go through multiple network hops to talk to pods in another cluster. The flat network space created by Submariner, where pods from different clusters can communicate directly, resolves this issue.

Recommendations and best practices

As a part of the runtime layer, projects are using the options below to create container images. While Kubernetes has standardized on containerd + cri-o, any OCI-compliant image is compatible with this runtime. Projects have many options to go from code to a built container image.

Many tools can convert code to OCI-compliant container images. Tools, such as tilt, exponentially boost overall productivity and are worth exploring.

Other CNCF projects in this layer are best used through cloud and product vendors. For example, Longhorn (I) is offered as a packaged product with the Rancher Kubernetes distribution.

Contributing to CNCF projects

All CNCF projects are open source under Apache License 2.0 and hosted on GitHub. Every project has a GitHub repository with the README file providing instructions to start using the project, to start developing for the project, and contributing.

While Kubernetes is a large project with a stable governance structure and detailed steps for contribution, things may differ slightly in terms of structure and steps for other CNCF projects. Read, explore, and gain sufficient knowledge to become comfortable working with the build and codebase of your chosen project. Many projects assume a Linux or Linux-adjacent environment such as Mac or Windows Subsystem for Linux (WSL) and use tooling that works well on a Linux-like environment.

The first issue will take the longest, but subsequent issues will gradually become easier.

Get to know the people involved in the project and attend weekly technical committee meetings. The more you know about and interact with other technical experts, the more you benefit.

The predominant development language for CNCF hosted Incubating, Graduated projects is Go, and other languages are Rust, C++, Typescript, Shell script and Python. Refer to the below table for languages by project.

Most CNCF projects use shell scripts, makefiles, dockerfiles, and helm charts.

References