A Pod would be the smallest deployable unit one can create and manage inside Kubernetes(K8s). In practice, rarely you create a Pod directly. Instead, one would create a Deployment, a StatefulSet, a DaemonSet, a Job, or a CronJob. These are higher-level constructs that would create Pods for you.

Here I’m covering common tasks that one would do with a Pod in daily work, like getting logs, opening a shell inside a container, debugging a Pod, etc.

If you don’t have access to other K8s clusters where you can execute these examples, go to my quick intro to k3d. I assume that you already know what kubectl is, and you have it installed.

All these examples are run locally with kind. To add resources to the cluster run:

#!bin/sh
#
kubectl create ns apps
kubectl create ns dbs
kubectl create ns workers
kubectl create ns kafka

kubectl create deployment -n apps nginx --image nginx:stable-alpine
kubectl create deployment -n workers  nginx --image nginx:stablex-alpine
kubectl create deployment -n dbs postgres --image postgres:15.2-alpine

Find/List/Filter pods

You are looking for a Pod. All you know is an application name that is running in a Pod. How do you find it? As already mentioned, Pods are usually created by higher-level constructs. And usually, they are named by an application that they are running. If that is the case pods will include the application name in their name. One way would be:

> kubectl get pods -A | grep <application-name>

For example:

❯ kubectl get pods -A | grep nginx
apps                 nginx-9dc8cc55b-9lcpg                                     1/1     Running            0          2m52s
workers              nginx-7f685bb9bb-4wh6p                                    0/1     ImagePullBackOff   0          100s

Command kubectl get pods would search for Pods in the current namespace. If you want to search in all namespaces use -A flag. The output needs to be filtered by grep to find the Pod that you are looking for. In this case, Pods are running in apps and workers namespaces. To see the default namespace use kubectl config get-contexts command:

❯ kubectl config get-contexts
CURRENT   NAME                                                          CLUSTER                                                       AUTHINFO                                                      NAMESPACE
          cloud-run-dev-internal                                        cloud-run-dev-internal                                        cloud-run-dev-internal                                        default
          dev                                                           dev                                                           bgates                                                        default
*         kind-kind-test-cluster                                        kind-kind-test-cluster                                        kind-kind-test-cluster

If you follow the best practices for labeling resources you can get search for Pods by labels. In these examples I’m using the app label:

> kubectl get pods -l app=<app-name>

For example:

❯ kubectl get pods -A -l app=nginx
NAMESPACE   NAME                     READY   STATUS             RESTARTS   AGE
apps        nginx-9dc8cc55b-9lcpg    1/1     Running            0          19m
workers     nginx-7f685bb9bb-4wh6p   0/1     ImagePullBackOff   0          18m

Of course, Pods can have different labels that can be used for searching.

Get additional informations about a Pod/Debug Pods

To learn more about a Pod you can get its manifest with kubectl get pod <pod-name> -o yaml command. I find this rather unpractical so I use kubectl describe pod <pod-name> command. It gives me more information about a Pod in a human-readable format:

❯ kubectl describe pod -n apps nginx-9dc8cc55b-9lcpg
Name:             nginx-9dc8cc55b-9lcpg
Namespace:        apps
Priority:         0
Service Account:  default
Node:             kind-test-cluster-worker/172.18.0.2
Start Time:       Sat, 25 Feb 2023 18:29:08 +0100
Labels:           app=nginx
                  pod-template-hash=9dc8cc55b
Annotations:      <none>
Status:           Running
IP:               10.244.1.3
IPs:
  IP:           10.244.1.3
Controlled By:  ReplicaSet/nginx-9dc8cc55b
Containers:
  nginx:
    Container ID:   containerd://4a09db88bcc07dcaab7f5cef4277cb7695cdeff63524e0b037cde8cb1f03eaa2
    Image:          nginx:stable-alpine
[...snip...]
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  24m   default-scheduler  Successfully assigned apps/nginx-9dc8cc55b-9lcpg to kind-test-cluster-worker
  Normal  Pulled     24m   kubelet            Container image "nginx:stable-alpine" already present on machine
  Normal  Created    24m   kubelet            Created container nginx
  Normal  Started    24m   kubelet            Started container nginx

Sometimes your pods can not start. So you do not have logs to check. Usually, these are cases when Pods are in CrashLoopBackOff or ImagePullBackOff state. Then describe a Pod and look under the Events section. It will give you information about what is going on. For example:

❯ kubectl get pods -A -l app=nginx
NAMESPACE   NAME                     READY   STATUS             RESTARTS   AGE
apps        nginx-9dc8cc55b-9lcpg    1/1     Running            0          43m
workers     nginx-7f685bb9bb-4wh6p   0/1     ImagePullBackOff   0          42m

Why is nginx-7f685bb9bb-4wh6p in ImagePullBackOff state? Let’s check:

❯ kubectl describe pods -n workers nginx-7f685bb9bb-4wh6p
Name:             nginx-7f685bb9bb-4wh6p
Namespace:        workers
Priority:         0
Service Account:  default
[...snip...]
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason     Age                    From               Message
  ----     ------     ----                   ----               -------
  Normal   Scheduled  42m                    default-scheduler  Successfully assigned workers/nginx-7f685bb9bb-4wh6p to kind-test-cluster-worker2
  Normal   Pulling    41m (x4 over 42m)      kubelet            Pulling image "nginx:stablex-alpine"
  Warning  Failed     41m (x4 over 42m)      kubelet            Failed to pull image "nginx:stablex-alpine": rpc error: code = NotFound desc = failed to pull and unpack image "docker.io/library/nginx:stablex-alpine": failed to resolve reference "docker.io/library/nginx:stablex-alpine": docker.io/library/nginx:stablex-alpine: not found
  Warning  Failed     41m (x4 over 42m)      kubelet            Error: ErrImagePull
  Warning  Failed     40m (x6 over 42m)      kubelet            Error: ImagePullBackOff
  Normal   BackOff    2m40s (x172 over 42m)  kubelet            Back-off pulling image "nginx:stablex-alpine"

As you can see the image nginx:stablex-alpine does not exist. That is why the Pod can not start.

Get logs

To get logs from a Pod you can use kubectl logs <pod-name> command. It will give you logs from all containers in Pod. If you want to get logs from a specific container inside a Pod you can use -c <container-name> flag. For example:

❯ kubectl logs -n apps nginx-9dc8cc55b-9lcpg -c nginx
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
2023/02/25 17:29:09 [notice] 1#1: using the "epoll" event method
2023/02/25 17:29:09 [notice] 1#1: nginx/1.22.1
2023/02/25 17:29:09 [notice] 1#1: built by gcc 11.2.1 20220219 (Alpine 11.2.1_git20220219)
2023/02/25 17:29:09 [notice] 1#1: OS: Linux 5.15.49-linuxkit
2023/02/25 17:29:09 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
2023/02/25 17:29:09 [notice] 1#1: start worker processes
2023/02/25 17:29:09 [notice] 1#1: start worker process 33
2023/02/25 17:29:09 [notice] 1#1: start worker process 34
2023/02/25 17:29:09 [notice] 1#1: start worker process 35
2023/02/25 17:29:09 [notice] 1#1: start worker process 36

It will output logs on stdout. If you do not want to repeat this command all the time you can use -f flag to follow logs. Then when ever new logs are added to the Pod you will see them in your console.

Execute a command in a Pod/Open shell in a Pod

Sometimes you need to execute a command in a Pod. Like to see all environment variables, so you do something like kubectl exec <pod-name> -- <command>:

❯ kubectl exec -n apps nginx-9dc8cc55b-9lcpg -- env
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=nginx-9dc8cc55b-9lcpg
NGINX_VERSION=1.22.1
NJS_VERSION=0.7.7
PKG_RELEASE=1
KUBERNETES_PORT_443_TCP_ADDR=10.96.0.1
KUBERNETES_SERVICE_HOST=10.96.0.1
KUBERNETES_SERVICE_PORT=443
KUBERNETES_SERVICE_PORT_HTTPS=443
KUBERNETES_PORT=tcp://10.96.0.1:443
KUBERNETES_PORT_443_TCP=tcp://10.96.0.1:443
KUBERNETES_PORT_443_TCP_PROTO=tcp
KUBERNETES_PORT_443_TCP_PORT=443
HOME=/root

But if you want to open a shell in a Pod and interact with it you can use kubectl exec -it <pod-name> -- <command> command. For example:

❯ kubectl exec -n apps nginx-9dc8cc55b-9lcpg -it -- sh
/ #

Close shell with exit command or Ctrl+D.