A Pod would be the smallest deployable unit one can create and manage inside Kubernetes(K8s). In practice, rarely you create a Pod directly. Instead, one would create a Deployment, a StatefulSet, a DaemonSet, a Job, or a CronJob. These are higher-level constructs that would create Pods for you.
Here I’m covering common tasks that one would do with a Pod in daily work, like getting logs, opening a shell inside a container, debugging a Pod, etc.
If you don’t have access to other K8s clusters where you can execute these examples, go to my quick intro to k3d. I assume that you
already know what
kubectl is, and you have it installed.
All these examples are run locally with kind. To add resources to the cluster run:
#!bin/sh # kubectl create ns apps kubectl create ns dbs kubectl create ns workers kubectl create ns kafka kubectl create deployment -n apps nginx --image nginx:stable-alpine kubectl create deployment -n workers nginx --image nginx:stablex-alpine kubectl create deployment -n dbs postgres --image postgres:15.2-alpine
You are looking for a Pod. All you know is an application name that is running in a Pod. How do you find it? As already mentioned, Pods are usually created by higher-level constructs. And usually, they are named by an application that they are running. If that is the case pods will include the application name in their name. One way would be:
> kubectl get pods -A | grep <application-name>
❯ kubectl get pods -A | grep nginx apps nginx-9dc8cc55b-9lcpg 1/1 Running 0 2m52s workers nginx-7f685bb9bb-4wh6p 0/1 ImagePullBackOff 0 100s
kubectl get pods would search for Pods in the current namespace. If you want to search in all namespaces use
-A flag. The output needs
to be filtered by
grep to find the Pod that you are looking for. In this case, Pods are running in
workers namespaces. To see the
default namespace use
kubectl config get-contexts command:
❯ kubectl config get-contexts CURRENT NAME CLUSTER AUTHINFO NAMESPACE cloud-run-dev-internal cloud-run-dev-internal cloud-run-dev-internal default dev dev bgates default * kind-kind-test-cluster kind-kind-test-cluster kind-kind-test-cluster
If you follow the best practices for labeling resources
you can get search for Pods by labels. In these examples I’m using the
> kubectl get pods -l app=<app-name>
❯ kubectl get pods -A -l app=nginx NAMESPACE NAME READY STATUS RESTARTS AGE apps nginx-9dc8cc55b-9lcpg 1/1 Running 0 19m workers nginx-7f685bb9bb-4wh6p 0/1 ImagePullBackOff 0 18m
Of course, Pods can have different labels that can be used for searching.
Get additional informations about a Pod/Debug Pods
To learn more about a Pod you can get its manifest with
kubectl get pod <pod-name> -o yaml command. I find this rather unpractical so I use
kubectl describe pod <pod-name> command. It gives me more information about a Pod in a human-readable format:
❯ kubectl describe pod -n apps nginx-9dc8cc55b-9lcpg Name: nginx-9dc8cc55b-9lcpg Namespace: apps Priority: 0 Service Account: default Node: kind-test-cluster-worker/172.18.0.2 Start Time: Sat, 25 Feb 2023 18:29:08 +0100 Labels: app=nginx pod-template-hash=9dc8cc55b Annotations: <none> Status: Running IP: 10.244.1.3 IPs: IP: 10.244.1.3 Controlled By: ReplicaSet/nginx-9dc8cc55b Containers: nginx: Container ID: containerd://4a09db88bcc07dcaab7f5cef4277cb7695cdeff63524e0b037cde8cb1f03eaa2 Image: nginx:stable-alpine [...snip...] Node-Selectors: <none> Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 24m default-scheduler Successfully assigned apps/nginx-9dc8cc55b-9lcpg to kind-test-cluster-worker Normal Pulled 24m kubelet Container image "nginx:stable-alpine" already present on machine Normal Created 24m kubelet Created container nginx Normal Started 24m kubelet Started container nginx
Sometimes your pods can not start. So you do not have logs to check. Usually, these are cases when Pods are in CrashLoopBackOff or
ImagePullBackOff state. Then describe a Pod and look under the
Events section. It will give you information about what is going on.
❯ kubectl get pods -A -l app=nginx NAMESPACE NAME READY STATUS RESTARTS AGE apps nginx-9dc8cc55b-9lcpg 1/1 Running 0 43m workers nginx-7f685bb9bb-4wh6p 0/1 ImagePullBackOff 0 42m
ImagePullBackOff state? Let’s check:
❯ kubectl describe pods -n workers nginx-7f685bb9bb-4wh6p Name: nginx-7f685bb9bb-4wh6p Namespace: workers Priority: 0 Service Account: default [...snip...] Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 42m default-scheduler Successfully assigned workers/nginx-7f685bb9bb-4wh6p to kind-test-cluster-worker2 Normal Pulling 41m (x4 over 42m) kubelet Pulling image "nginx:stablex-alpine" Warning Failed 41m (x4 over 42m) kubelet Failed to pull image "nginx:stablex-alpine": rpc error: code = NotFound desc = failed to pull and unpack image "docker.io/library/nginx:stablex-alpine": failed to resolve reference "docker.io/library/nginx:stablex-alpine": docker.io/library/nginx:stablex-alpine: not found Warning Failed 41m (x4 over 42m) kubelet Error: ErrImagePull Warning Failed 40m (x6 over 42m) kubelet Error: ImagePullBackOff Normal BackOff 2m40s (x172 over 42m) kubelet Back-off pulling image "nginx:stablex-alpine"
As you can see the image
nginx:stablex-alpine does not exist. That is why the Pod can not start.
To get logs from a Pod you can use
kubectl logs <pod-name> command. It will give you logs from all containers in Pod. If you
want to get logs from a specific container inside a Pod you can use
-c <container-name> flag. For example:
❯ kubectl logs -n apps nginx-9dc8cc55b-9lcpg -c nginx /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/ /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh 10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf 10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh /docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh /docker-entrypoint.sh: Configuration complete; ready for start up 2023/02/25 17:29:09 [notice] 1#1: using the "epoll" event method 2023/02/25 17:29:09 [notice] 1#1: nginx/1.22.1 2023/02/25 17:29:09 [notice] 1#1: built by gcc 11.2.1 20220219 (Alpine 11.2.1_git20220219) 2023/02/25 17:29:09 [notice] 1#1: OS: Linux 5.15.49-linuxkit 2023/02/25 17:29:09 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576 2023/02/25 17:29:09 [notice] 1#1: start worker processes 2023/02/25 17:29:09 [notice] 1#1: start worker process 33 2023/02/25 17:29:09 [notice] 1#1: start worker process 34 2023/02/25 17:29:09 [notice] 1#1: start worker process 35 2023/02/25 17:29:09 [notice] 1#1: start worker process 36
It will output logs on stdout. If you do not want to repeat this command all the time you can use
-f flag to follow logs. Then
when ever new logs are added to the Pod you will see them in your console.
Execute a command in a Pod/Open shell in a Pod
Sometimes you need to execute a command in a Pod. Like to see all environment variables, so you do something like
kubectl exec <pod-name> -- <command>:
❯ kubectl exec -n apps nginx-9dc8cc55b-9lcpg -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin HOSTNAME=nginx-9dc8cc55b-9lcpg NGINX_VERSION=1.22.1 NJS_VERSION=0.7.7 PKG_RELEASE=1 KUBERNETES_PORT_443_TCP_ADDR=10.96.0.1 KUBERNETES_SERVICE_HOST=10.96.0.1 KUBERNETES_SERVICE_PORT=443 KUBERNETES_SERVICE_PORT_HTTPS=443 KUBERNETES_PORT=tcp://10.96.0.1:443 KUBERNETES_PORT_443_TCP=tcp://10.96.0.1:443 KUBERNETES_PORT_443_TCP_PROTO=tcp KUBERNETES_PORT_443_TCP_PORT=443 HOME=/root
But if you want to open a shell in a Pod and interact with it you can use
kubectl exec -it <pod-name> -- <command> command. For example:
❯ kubectl exec -n apps nginx-9dc8cc55b-9lcpg -it -- sh / #
Close shell with
exit command or