There is an application running in a Kubernetes cluster. Goal is to protect this application from any modifications, except if those modifications are coming from predefined actor.
Setup Requirements
To start let’s set up the Kind cluster and Klock. I assume you already have installed Kind. Now let’s create a cluster and install all requirements:
- Create cluster:
kind create cluster - Install cert-manager:
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.8.2/cert-manager.yaml - Install Klock :
helm repo add rnemet https://rnemet.dev/helm-charts
helm repo update
helm install klock rnemet/klock
Setup Scenario
My application has one Pod and one ConfigMap. By default the Klock supports locking Deployments, Pods, Secrets, and ConfigMaps. So, I’m covered there.
Let’s deploy our application, one Pod and one ConfigMap:
kubectl run my-pod --image nginx --labels aura=red
kubectl create configmap my-cm --from-literal nikola=tesla
kubectl label cm my-cm aura=red
Add one more Pod:
kubectl run other-pod --image nginx
Check what we have in default namespace:
❯ kubectl get pods,cm
NAME READY STATUS RESTARTS AGE
pod/my-pod 1/1 Running 0 5m52s
pod/other-pod 1/1 Running 0 112s
NAME DATA AGE
configmap/kube-root-ca.crt 1 18m
configmap/my-cm 1 4m20s
While our application is:
❯ kubectl get pod,cm -A --selector aura=red
NAMESPACE NAME READY STATUS RESTARTS AGE
default pod/my-pod 1/1 Running 0 7m2s
NAMESPACE NAME DATA AGE
default configmap/my-cm 1 5m30s
Let’s add one service account(SA) to be used for our so-called operator to manage our application:
kubectl create serviceaccount jonny-op
If you try to list pods with jonny-op:
❯ kubectl run operator --image roffe/kubectl --overrides='{"apiVersion":"v1","spec":{"serviceAccount":"jonny-op"}}' -it --restart Never -- /bin/sh -c "kubectl -n default get pods"
No resources found.
Error from server (Forbidden): pods is forbidden: User "system:serviceaccount:default:jonny-op" cannot list resource "pods" in API group "" in the namespace "default"
pod default/operator terminated (Error)
Here I’m trying to list all pods in default namespace with our fake operator. As operator run with SA jonny-op it can not list pods. What about our application:
kubectl delete pod operator
❯ kubectl run operator --image roffe/kubectl --overrides='{"apiVersion":"v1","spec":{"serviceAccount":"jonny-op"}}' -it --restart Never -- /bin/sh -c "kubectl get pod,cm --selector aura=red"
No resources found.
Error from server (Forbidden): pods is forbidden: User "system:serviceaccount:default:jonny-op" cannot list resource "pods" in API group "" in the namespace "default"
Error from server (Forbidden): configmaps is forbidden: User "system:serviceaccount:default:jonny-op" cannot list resource "configmaps" in API group "" in the namespace "default"
pod default/operator terminated (Error)
Doh… Let’s fix this by creating Role and RoleBinding:
kubectl create role app-op --verb list --verb get --verb create --verb update --verb delete --verb patch --resource pods --resource cm
kubectl create rolebinding jonny-app --role app-op --serviceaccount default:jonny-op
And check:
❯ kubectl delete pod operator
pod "operator" deleted
❯ kubectl run operator --image roffe/kubectl --overrides='{"apiVersion":"v1","spec":{"serviceAccount":"jonny-op"}}' -it --restart Never -- /bin/sh -c "kubectl get pod,cm --selector aura=red"
If you don't see a command prompt, try pressing enter.
NAME READY STATUS RESTARTS AGE
pod/my-pod 1/1 Running 0 26m
NAME DATA AGE
configmap/my-cm 1 25m
Notice: you need to delete operator pod each time because it can not be updated.
Run Scenario
Me, as external actor can modify both Pod my-pod and ConfigMap my-cm. Well, I created the cluster and I’m an admin in this case so that is natural. As shown, we can do the same in the default namespace running a workload with SA jonny-op, or operator. But my goal is just and only jonny_op can do it. And only for my application.
All resources in my application are grouped with the label aura=red. So, let’s create a lock:
kubectl apply -f - <<EOF
apiVersion: klock.rnemet.dev/v1
kind: Lock
metadata:
name: lockred
spec:
operations:
- UPDATE
- DELETE
matcher:
aura: red
exclusive:
name: jonny-op
EOF
Now let’s try to update Pod my-pod:
❯ kubectl label pod my-pod aura=blue --overwrite
Error from server (denied, there is a lock: map[aura:red]): admission webhook "klocks.rnemet.dev" denied the request: denied, there is a lock: map[aura:red]
Ok… What about Pod other-pod:
❯ kubectl label pod other-pod aura=blue --overwrite
pod/other-pod labeled
So, I as admin can label(UPDATE) Pod that is not labeled with aura:red. What about SA jonny-op:
❯ kubectl run operator --image roffe/kubectl --overrides='{"apiVersion":"v1","spec":{"serviceAccount":"jonny-op"}}' -it --restart Never -- /bin/sh -c "kubectl label pod my-pod here=was-jonny"
pod/my-pod labeled
❯ k get pods --show-labels
NAME READY STATUS RESTARTS AGE LABELS
my-pod 1/1 Running 0 91m aura=red,here=was-jonny
operator 0/1 Completed 0 25s run=operator
other-pod 1/1 Running 0 87m aura=blue,run=other-pod
What about ConfigMap my-cm?
❯ kubectl label cm my-cm admin=was-here
Error from server (denied, there is a lock: map[aura:red]): admission webhook "klocks.rnemet.dev" denied the request: denied, there is a lock: map[aura:red]
❯ k delete pods operator
pod "operator" deleted
❯ kubectl run operator --image roffe/kubectl --overrides='{"apiVersion":"v1","spec":{"serviceAccount":"jonny-op"}}' -it --restart Never -- /bin/sh -c "kubectl label cm my-cm here=was-jonny"
configmap/my-cm labeled
❯ kubectl get cm --show-labels
NAME DATA AGE LABELS
kube-root-ca.crt 1 108m <none>
my-cm 1 94m aura=red,here=was-jonny
Same. So, I managed to protect my application from other actors in the cluster, while SA jonny-op can operate with it.
Conclusion
Even if this is more manual example, it shows that is possible to make exclusive lock in Kubernetes with Klock.