A Brief History of Kubernetes Security

Kubernetes has two primary mechanisms for controlling accesses within the cluster.

  1. Role-Based Access Contol (RBAC) for limiting API Requests
  2. SecurityContexts for limiting system-local privileges, such as root and UID/GID

API Requests and Role-Based Access Control (northbound requests)

Northbound requests are requests that are sent to the Kubernetes API to perform an action to the cluster. Ususally these requests create, delete and update data stored in the cluster backend, normally ETCD. (NOTE: There are special sub-resource requests such as oc logs ... in which case the request is forwarded to the a Kubelet instace, but this is not in scope for this post)

Access scope of these northbound requests are controlled using RBAC policies, which consist of Roles, RoleBindings and Identities. These resources are discussed further in the list below:

  • Roles define the permissions provided to an Identity for a given subject or resource when an Identity is associated with the Role. These permissions provide the ability to create, read, update, and delete resources within the cluster. For more information about the available permissions (verbs) that can be allocated, see Kubernetes: Using RBAC Authorization

  • RoleBindings are used to create a relationship between Roles and Identities. RoleBindings associate a Role with a given Identity and ensure that the Identity is provided the authorization(s) of the associated Role(s).

  • Identities are not a resource-type in Kubernetes but instead are provided by the authentication provider to the authorization provider generally not stored in the cluster. Kubernetes supports a number of external authentication processes which will provide the API the Username and Group Associations of the identity upon successful authentication. Internally, this could be through the use of x509 certificates, in which case the Common Name (CN) and Organisation (O) values are used for the Username and Group Associatations respectively. Alternatives authentication mechanisms include OAuth and ServiceAccountTokens.

Note: Roles and RoleBindings are limited to a single Namespace and have cluster-wide alternatives, ClusterRoles and ClusterRoleBindings. These alternative resources are used to provide authorization to an Identity to all Namespaces within the cluster.

RBAC policies only govern the authorization of requests to the Kubernetes API and do not provide additional containment benefits for containers running on the hosts.

SecurityContexts and Host-level Security (southbound requests)

Southbound requests are requests to the Kernel via syscalls requesting items such as file-accesses, additional capabilities, privilege escalation and much more.

The SecurityContexts defines the boundaries placed around the Pod, and its containers processes, limiting the capabilities and requests that can be performed by the process.

Taking a snippet from the Kubernetes Documentation:

A security context defines privilege and access control settings for a Pod or Container.

SecurityContexts can be used to configure the Discretionary Access Control (DAC) permissions of the Pod such as the Linux UID/GID context of the processes within the Pod.

These values configure limits on which syscalls can performed by the processes in the Pod and which capabilities these processes have available.

The capabilities available for a process are controlled by the SecurityContext component of the Pod definition while the DAC and capabilities configurations are defined within each container definition.

For more details about the options that SecurityContexts provide, use the following:

# For Pod-wide configurations
$ kubectl explain pod.spec.securityContext --recursive

# For Container-specific configurations
$ kubectl explain pod.spec.containers.securityContext --recursive

As SecurityContexts can provide a Pod with the ability to become root on the Node, is there a way to leverage RBAC in the API to enforce limits on which SecurityContexts can be configured by given RBAC users?

Pod Security Policies and SecurityContextConstraints

To ensure that everyday users of the cluster were not able to perform elevated southbound requests, the industry developed admission controllers that would intercept all requests traversing the API and perform actions on the requests, such as rejecting or mutating the request before it is persisted to the backend storage.

Two common admission controllers and related custom-resources were:

  1. Pod Security Policies (PSP) are the community solution to the SecurityContext problem. PSPs consist of a custom resource which defines a list of acceptable values for the PodSecurityContext and an enforcing admission controller that rejects all Pods without associated PSPs. PSPs are associated to Pods through requesting user and ServiceAccount RBAC permissions. Once an Identity is provided the use permission for one or more PSP resources, the Pod will be provided the allowances to run in accoradance with the chosen PSPs. The order of precedence for the PSP-lists prefers non-mutating PSPs but if a mutating PSP is required to run the Pod, this will be ordered by PSP name. See Policy Order. NOTE: PodSecurityPolicies have been deprecated in favour of configurable admission controllers, discussed later in this blog.

  2. Security Context Constraints (SCC) are an OpenShift mechanism and provide mostly the same granularity as PSPs with a different design philosophy. SCCs are provided through a combination of an enforcing admission controller and custom resource defining limits on the requestable SecurityContext for a Pod. Associations between SCCs and Identities is directly written into the SCC custom resource under scc.groups and scc.users. When a Pod is able to use multiple SCCs, the SCC chosen in the order of:

    • Ordered by highest priority (a field set in the SCC definition)
    • Ordered from most restrictive to least restrictive
    • Ordered by name

    For more information about SCCs, see the Official OpenShift Documentation.

Generic Admission Controllers and Webhooks

During the development of SCCs and PSPs, admission controllers were required to be written in Go, register themselves with the Kubernetes APIServer and be built into the APIServer binary. As the Kubernetes APIServer matured, the mutating admission webhook controller and validating admission webhook controller were included into the Kubernetes API allowing for dynamic definition of admission control through the use of out-of-tree controllers registered using Kubernetes resources, mutatingwebhookconfigurations and validatingwebhookconfigurations respectively. These objects define Kubernetes Service endpoints that should be used to perform admission checks and the admission controllers can be hosted within the cluster.

Leveraging this webhoook controller, there are many configurable admission controllers such as the Open Policy Agent(OPA) - Gatekeeper which uses the OPA Rego policy language to describe compliance policies for structured data. These Rego policies are then enforced by the Gatekeeper on all requests that it receives, as Kubernetes objects can be represented as structured data.

The Rego language supports querying for data in the structure, asserting requirements and mutating requests before they are admitted to the clsuter.

PodSecurityPolicies have been deprecated and will be removed in Kubernetes v1.25 in favour of configurable admission controllers.

Additional Readings

If you found this blog interesting, you may find the following code interesting: