Mastering the CKS Kubernetes Exam — 1/6 — Cluster Setup

As you prepare for the Certified Kubernetes Security Specialist (CKS) exam, it’s crucial to focus on securing your Kubernetes cluster effectively.

This is the first part of 6 posts, showing all the domains and competencies for this exam.

Today’s competency is Cluster Setup, which constitutes 10% of the exam:

This post will guide you through some fundamental security practices, including network security policies, CIS benchmarks, ingress security, protecting node metadata, minimizing GUI access, and verifying platform binaries.

1. Use Network Security Policies to Restrict Cluster-Level Access

Network policies control the traffic flow between pods in your Kubernetes cluster. By defining these policies, you can limit which pods can communicate with each other and external services.

Example:

Create a NetworkPolicy to allow traffic only from a specific namespace:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-specific-namespace
  namespace: default
spec:
  podSelector:
    matchLabels:
      role: backend
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          name: trusted

In this example, only pods in the trusted namespace can communicate with pods labeled as role: backend in the default namespace:

You will be allowed to check the documentation during the exam, so make sure you are used to the interface and the way you search on it.

Here are more examples of networking policies:

https://kubernetes.io/docs/concepts/services-networking/network-policies/

2. Use CIS Benchmark to Review Security Configuration

The Center for Internet Security (CIS) Kubernetes Benchmark is a vital tool for ensuring that your Kubernetes cluster adheres to security best practices.

By following the CIS Benchmark, you can systematically evaluate and enhance the security of Kubernetes components such as etcd, kubelet, kubedns, and kubeapi.

Where to Download the CIS Benchmark

The CIS Kubernetes Benchmark is available for free from the CIS website. You’ll need to register for an account to download the benchmark guide, which provides detailed recommendations and checks for Kubernetes security configurations.

How to Use the CIS Benchmark

• kube-bench: This tool automates the process of running the benchmark checks and generates reports.

1. Download kube-bench:

https://github.com/aquasecurity/kube-bench

You can also use docker if you don’t want to download it:

docker run --rm -it --net=host docker.io/aquasec/kube-bench:latest

Perform a Security Assessment against master and nodes:

kube-bench run --targets master,node

It Compares your cluster’s current configuration against the benchmark’s recommendations.

The benchmark provides a checklist of recommended settings and configurations that you should validate.

Example: Reviewing API Server Security Configuration

In the 2.1 picture, one of the controls in the benchmark relates that API Server allows anonymous-auth as you can see on 1.2.1. If you check your API Server:

2.1

If we check the kube-api manifest file:

cat /etc/kubernetes/manifests/kube-apiserver.yaml

in the spec section, we add the line to fix the problem found in the benchmark:

spec:
  containers:
  - command:
    - kube-apiserver
    - --advertise-address=192.21.217.6
    - --allow-privileged=true
    - --anonymous-auth=false  <---ADD THIS LINE
    - --authorization-mode=Node,RBAC
    - --client-ca-file=/etc/kubernetes/pki/ca.crt
    - --enable-admission-plugins=NodeRestriction
    - --enable-bootstrap-token-auth=true
    - --etcd-cafile=/etc/kubernetes/pki/etcd/ca.crt
    - --etcd-certfile=/etc/kubernetes/pki/apiserver-etcd-client.crt
    - --etcd-keyfile=/etc/kubernetes/pki/apiserver-etcd-client.key
    - --etcd-servers=https://127.0.0.1:2379

If we run again now:

kube-bench run --targets master,node

• We verified that it passed the benchmark:

You should do the same with the other warning and failings to make your cluster more secure.

3. Properly Set Up Ingress Objects with Security Control

Recommended lab:

Ingress Secure | Killer Shell CKS | Killercoda

Ingress objects manage external access to services within your Kubernetes cluster. The proper configuration ensures secure access.

Example: Define an Ingress resource with TLS termination:

This example requires you to create a minikube environment. If you have a PC with at least 4GB of RAM, you should be fine:

Install Minikube:

curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
sudo install minikube-linux-amd64 /usr/local/bin/minikube && rm minikube-linux-amd64

Start minikube:

minikube start

Add the ingress addon to the cluster:

minikube addons enable ingress

Create a sample deployment to respond to the ingress requests:

kubectl create deployment web --image=gcr.io/google-samples/hello-app:1.0

Expose the deployment as a NodePort on port 8080 in the pod:

kubectl expose deployment web --type=NodePort --port=8080

Finally, let’s create the ingress without TLS certificate:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: insecure-ingress
spec:
  ingressClassName: nginx
  rules:
    - host: hello-world.example
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: web
                port:
                  number: 8080

Insecure request on port 80 is sent:

curl --resolve "hello-world.example:80:$( minikube ip )" -i http://hello-world.example

As you can see, the pod is working, as well as the ingress:

Now let’s move on and secure this ingress with a certificate. For this lab, we don’t have a valid certificate, so we will generate a self-signed one, which will allow us to securely access the pod (443 port):

Generate a self signed key and certificate:

openssl req -new -newkey rsa:4096 -x509 -sha256 -days 365 -nodes -out t
ls.crt -keyout tls.key

Now, generate a secret that will be used by the ingress:

kubectl create secret tls tls-secret --key tls.key --cert tls.crt

In the ingress now, we add the tls section, which will use the secret we created earlier:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: secure-ingress
spec:
  tls:                      #ADD
  - hosts:                  #ADD
    - hello-world.example   #ADD
    secretName: tls-secret  #ADD
  ingressClassName: nginx
  rules:
    - host: hello-world.example
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: web
                port:
                  number: 8080

Here we use -k flag to skip certificate validation and -v for verbose mode:

curl -kv - resolve "hello-world.example:443:127.0.0.1" -i https://hello-world.example

As you can see, the certificate is being used by the ingress for TLS termination. This Ingress configuration uses TLS to secure communication between clients and your application.

4. Protect Node Metadata and Endpoints

This domain is also related to networking policies.

Protecting node metadata and API endpoints helps safeguard your cluster from unauthorized access.

In our example, the following endpoint must be protected:

http://169.254.169.254/latest/meta-data/

Action: Implement IAM policies and network security groups to restrict access to this metadata.

Creating Networking Policies To Allow/ Deny Access to the Endpoint

Ensure that your node metadata API is exposed only to the pods that have the security: metadata-access label.

To do so, we first deny egress access to all pods to that IP address on port 80:

metadata-deny.yaml

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: deny-metadata-access
  namespace: default
spec:
  podSelector: {}  #<- HERE WE SELECT ALL PODS
  policyTypes:
  - Egress
  egress:
  - to:
    - ipBlock:
        cidr: 0.0.0.0/0
        except:
        - 169.254.169.254/32
    ports:
    - protocol: TCP
      port: 80

Now, we allow only the pods on default namespace that match the label security: metadata-access to access that address on port 80.

metadata-allow.yaml

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-metadata-access
  namespace: default
spec:
  podSelector:
    matchLabels:
      security: metadata-access
  policyTypes:
  - Egress
  egress:
  - to:
    - ipBlock:
        cidr: 169.254.169.254/32
    ports:
    - protocol: TCP
      port: 80

5. Minimize Use of, and Access to, GUI Elements

Limiting the use of graphical interfaces reduces the attack surface of your cluster.

Example:

To restrict access to the Kubernetes dashboard, modify its deployment configuration:

kubectl edit deployment kubernetes-dashboard -n kubernetes-dashboard

In the editor, adjust the configuration to restrict or disable GUI access based on your security policies:

  template:
    spec:
      containers:
      - args:
        - --namespace=kubernetes-dashboard  
        - --authentication-mode=token        # change or delete, "token" is default
        - --auto-generate-certificates       # add
        #- --enable-skip-login=true          # delete or set to false
        #- --enable-insecure-login           # delete
        image: kubernetesui/dashboard:v2.0.3
        imagePullPolicy: Always
        name: kubernetes-dashboard

6. Verify Platform Binaries Before Deploying

Always verify the binaries and images used in your cluster to ensure they come from trusted sources.

Example:

From the official website:

Download Kubernetes

Let’s Check the image SHA before deploying kube-api binary:

Download the binary:

dl.k8s.io/v1.31.0/bin/linux/amd64/kube-apiserver

There is a checksum file right after the file, download it:

Here we can use a tool called sha256sum and compare the provided value in the website with the one from the actual binary:

echo '9016f6048ff9827ef58934e98f28a8026634c10b4e6fcc1df49451038a23a9aa kube-apiserver' | 
sha256sum --check

The output should be an OK:

kube-apiserver: OK

If the values are not the same, you’ll receive the following warning:

kube-apiserver: FAILED
sha256sum: WARNING: 1 computed checksum did NOT match

How do you know which sha you should use? When you download the file, it shows as an extension, like this:

It can also be sha512, in this case you should use sha512sum tool.

The goal is to ensure it always matches the expected value.

Conclusion

A secure cluster setup is vital for protecting your Kubernetes environment from potential vulnerabilities and threats. Best of luck with your CKS preparation!

You can read part 2/6 here, right now!

Feel free to reach out if you have any questions or need further assistance. Happy securing!

Follow me on: Linkedin