Kubernetes security for application developers
Kubernetes brought a lot of innovation into the application development and operations, these changes have effects on application and infrastructure security. It might be hard to reason about kubernetes security since this problem have to be addressed on multiple layers. In this post I will provide overview and references to the available security tools and procedures from the application developer perspective.
As mentioned in the official documentation it might be easier to reason about application security on kubernetes by separating this problem into multiple layers:
Infrastructure. This level is a foundation of a cluster and it includes typical server isolation and security procedures.
k8sspecific problems include proper network isolation for the communications between the control plane, nodes and
Cluster. This level address security of a
k8sdistribution and it’s components: TLS encryption setup for communications between components, authenticating and authorizing requests to a
k8sAPI and other
k8ssettings that can affect deployed applications.
Container runtime. On this level we asses vulnerabilities inside container images, trust chain between hosts and image repositories, container runtime settings that allow escaping from the container’s namespace.
Application. This is the field most developers will be familiar with.
k8sprovides several tools that directly impact application’s runtime and it might be useful to familiarize with those.
I will go through each level below and will try to highlight things that I think is important to be aware of.
Infrastructure security is a complex field but a lot of problems in
this level are not entirely unique to
k8s. Kubernetes nodes and
control plane servers should still adhere to security procedures of
regular deployments. This means that OS and software should be up to
date, servers should have a proper network isolation and security
tools provided by OS or infrastructure provider should be used. This
level is a foundation for other layers: if it will be possible to
break into a host it will be possible to avoid security mechanisms
from other levels.
It’s important to properly isolate control plane and nodes, ensure
that APIs of the control plane’s processes as well as
not exposed to public networks. Process for setting up network rules
will vary depending on the deployment but general guidance is provided
in the Accessing
Cluster security can be hard to reason about and will require quite a
bit of knowledge about kubernetes administration. Good starting points
for this is the Concepts and
the Securing a
sections of the official administration guide. I found the Kubernetes
Way to be
a useful tool to get familiar with
k8s components and setup
procedures, it includes a good walkthrough for the base security
k8s distribution will most likely provide the sane
security defaults but will not protect cluster from applying unsafe
configurations values to the
can be used for auditing such settings automatically. All
checks are also published as a separate checklist style document, it
can be used as a reference for potential security misconfigurations.
The Control Plane-Node
section of the documentation is a good starting point for securing
k8s communication channels. The main highlights from that section:
TLS should be properly setup and enforced, certificate validation for
API server to
kubelet communication has to be enabled explicitly,
appropriate authentication and authorization mechanisms should be
used for all API requests, additional measures might be necessary for
API server to nodes, pods and services communication.
Beyond control plane and node settings it’s important get familiar with RBAC authorization and use fine-grained restrictive roles often to minimize security risks when tokens are leaked.
falco also provides a good integration with the
k8s audit log and supports alerting rules if necessary.
It’s important to ensure that containers are providing safe runtime for applications, containers using expected images and it’s not possible to escape from container’s namespace.
A lot of the security procedures for a host OS also apply to a
container OS. Using static analysis for images with
trivy will help automating
some of those checks. Some of the problems may be avoided by using
OS-less base images like
scratch for statically compiled language or
by using minimal images provided by
It’s important to ensure that pulled images were not altered after the build, most container runtimes provide a way verify signatures of images. For docker registries and images you can follow Docker Content Trust guide.
It’s possible to escape container’s namespace or get control over the container runtime daemon with privileged permissions, elevated capabilities or certain host mounts. This enables large number of attacks: from taking control over container runtime to potentially being able to reconfigure cluster settings. Pod Security Policy can be used to prevent this across a cluster.
Application runtime protection quite often falls into domain of
application developers. I’m not going to focus on common application
security techniques like static and dynamic analysis but instead will
k8s tools and settings that can be used to strengthen
As mentioned above setting up
RBAC accounts with minimal permissions
per deployment will minimize impact if such credentials will be
leaked. Such accounts work well with other security features like
PSP. Applications that require
cluster-admin role (Kubernetes
dashboard for example) should be properly secured or avoided where
All pods automatically
service account credentials which give pods access to the API
server. In many cases pods don’t require these credentials, it’s
recommended to set
automountServiceAccountToken: false on all
resources unless such credentials are necessary.
Kubernetes Secrets have some limitations and it’s generally recommended to familiarize with the security properties and risks. Secret encryption has to be enabled and Sealed Secrets are useful when it’s necessary to commit manifests to a version control system.
can be used for constraining container runtime. Default permissions
are quite open and usually not necessary for large number
applications. General good practices for
PSP include restricting
privileged containers, unnecessary risky mounts and device
access. It’s also recommended to restrict root and privilege
escalation and have a read-only file system inside
seccomp profiles can
be used to define fine-granular restrictions for applications running
inside containers. Pod’s
can be used to define the same settings on the pod level.
Network isolation for pods can be ensured via
resources. Most common network plugins (
restrict communication between pods which means that all cluster pods
can discover and communicate with each other, alternative
should be used.
NetworkPolicy is scoped by a namespace and contains
list of rules that define allowed traffic to selected pods. Implicit
deny rule will be applied for traffic that was not explicitly
whitelisted upon policy deployment. General recommendation is to
minimize allowed traffic to specific ports and pods instead of broader
per namespace policies.
While not strictly related to the security, resource limits on pods can prevent certain attacks and overall a good practice for managing deployment environments.
It might be hard to keep an eye on all settings that were mentioned above. kubeaudit can help with automated manifests testing, it can also do checks against a container or a cluster.
As I mentioned at the start of this post, it might be hard to reason
k8s security as a whole. Once broken down to a smaller pieces
it’s noticeable that while problem is still complex it can be
addressed gradually and provided tools are quite powerful for securing
application runtime. It’s definitely worth spending some time learning
a bit more to avoid common pitfalls.