StatefulSet Force-Delete Corrupts PersistentVolume

kubectl is not just a command-line tool — it is a Kubernetes API client. Every command you type translates into HTTP requests to the kube-apiserver. Understanding this relationship explains why certain commands are fast (cached reads), why others are slow (watch calls), and why some fail with 'connection refused' when the API server is unreachable.

The common misconception is that kubectl is a deployment tool. It is a state inspection and mutation tool. Deployment is one use case. The real value is observability — seeing what the cluster is doing right now, what it did in the past, and why a specific Pod is stuck. Engineers who master kubectl's inspection commands debug production incidents in minutes. Engineers who only know apply and delete debug by redeploying and hoping.

This cheatsheet is organized by intent: what are you trying to do? Find a resource? Debug a failure? Roll back a deployment? Each section includes the commands, the output you should expect, and the production gotchas that bite when you use them at scale.

What a kubectl Commands Cheatsheet Actually Does

A kubectl commands cheatsheet is a curated reference of the most essential kubectl commands for interacting with Kubernetes clusters. It condenses the full kubectl CLI — which has over 100 verbs and thousands of flag combinations — into a focused set of operations that cover 90% of daily tasks: pod inspection, resource creation, log retrieval, and debugging. The core mechanic is mapping human-readable commands (like kubectl get pods -o wide) to API calls against the Kubernetes API server, which then returns structured JSON or YAML output.

In practice, a good cheatsheet groups commands by intent: cluster info, workload management, troubleshooting, and resource mutation. It highlights the most common flags (-o wide, -n <namespace>, --all-namespaces, -w for watch) and shows how to combine them for real workflows. For example, kubectl describe pod gives you events and status, while kubectl logs --tail=50 -f streams recent logs — two commands that together resolve most pod failures in under 30 seconds.

Use a cheatsheet when you need to move fast without memorizing every flag. It’s not a replacement for the official docs or kubectl --help, but it eliminates context-switching during incidents. Teams that internalize a cheatsheet reduce time-to-diagnosis by 40-60% in production outages, because they don’t stop to look up syntax under pressure.

Getting Information

The get and describe commands are your primary observability tools. They read cluster state from the API server without modifying anything — safe to run in production at any time.

kubectl get lists resources in a compact table format. It is fast because the API server caches the response. kubectl describe shows the same resources with full detail including events, conditions, and annotations — this is where the debugging signal lives.

The --output flag is more powerful than most engineers realize. -o jsonpath lets you extract specific fields programmatically. -o custom-columns builds custom dashboards in your terminal. -o name outputs only resource names, perfect for scripting.

# List resources
kubectl get pods
kubectl get pods -o wide          # + node and IP
kubectl get pods --all-namespaces # all namespaces
kubectl get deployments
kubectl get services
kubectl get nodes
kubectl get all                   # pods, services, deployments in current namespace

# Get YAML/JSON of a resource
kubectl get pod myapp-pod -o yaml
kubectl get deployment myapp-deployment -o json

# Detailed view with events — essential for debugging
kubectl describe pod myapp-pod
kubectl describe node my-node

# Watch resources update in real time
kubectl get pods -w

# Custom columns output
kubectl get pods -o custom-columns=NAME:.metadata.name,STATUS:.status.phase

# ── PRODUCTION-GRADE EXTRAS ──────────────────────────────────────────────

# Extract a specific field with jsonpath
kubectl get pod myapp-pod -o jsonpath='{.status.podIP}'
kubectl get pods -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.status.podIP}{"\n"}{end}'

# Get all pods with their resource requests
kubectl get pods -o custom-columns=NAME:.metadata.name,CPU_REQ:.spec.containers[0].resources.requests.cpu,MEM_REQ:.spec.containers[0].resources.requests.memory

# Find pods without resource limits (security/compliance check)
kubectl get pods -A -o json | jq -r '.items[] | select(.spec.containers[].resources.limits == null) | .metadata.namespace + "/" + .metadata.name'

# Get events sorted by time — critical for post-incident analysis
kubectl get events --sort-by='.lastTimestamp' -A

# Get events for a specific resource
kubectl get events --field-selector involvedObject.name=myapp-pod

# Check API server health
kubectl get --raw /healthz
kubectl get --raw /readyz
kubectl get --raw /livez

Debugging

The debugging commands — logs, exec, port-forward, cp — are how you interact with running (or crashed) containers. These commands go through the kubelet on the target node, not directly to the container runtime.

kubectl logs retrieves stdout/stderr from the container. The --previous flag is critical — it retrieves logs from the container instance that just crashed, which is the only way to see why a CrashLoopBackOff Pod is failing.

kubectl exec opens a shell inside the container. This is the Kubernetes equivalent of SSH. It requires the container to be running and have a shell binary (bash or sh). Distroless containers do not have a shell — use ephemeral debug containers instead.

kubectl port-forward creates a tunnel from your local machine to a Pod or Service inside the cluster. It bypasses Ingress, LoadBalancer, and NodePort — useful for accessing services that are not exposed externally.

# Container logs
kubectl logs myapp-pod
kubectl logs myapp-pod -c container-name  # specific container
kubectl logs myapp-pod --previous          # logs from crashed container
kubectl logs -f deployment/myapp           # follow deployment logs
kubectl logs myapp-pod --tail=100          # last 100 lines

# Execute commands inside a container
kubectl exec -it myapp-pod -- bash
kubectl exec -it myapp-pod -- sh           # if bash not available
kubectl exec myapp-pod -- env              # print env vars
kubectl exec myapp-pod -- cat /config/app.properties

# Port forward to access a service locally
kubectl port-forward pod/myapp-pod 8080:8000
kubectl port-forward service/myapp-svc 8080:80

# Copy files to/from a pod
kubectl cp myapp-pod:/var/log/app.log ./app.log
kubectl cp ./config.yaml myapp-pod:/app/config.yaml

# ── PRODUCTION-GRADE DEBUGGING ───────────────────────────────────────────

# Logs from all pods in a deployment (parallel)
kubectl logs deployment/myapp --all-containers --prefix

# Logs with timestamps — essential for correlating with external events
kubectl logs myapp-pod --timestamps

# Logs from a specific time window
kubectl logs myapp-pod --since=1h
kubectl logs myapp-pod --since-time=2026-04-07T10:00:00Z

# Ephemeral debug container for distroless images (K8s 1.23+)
kubectl debug -it myapp-pod --image=busybox --target=app-container

# Debug a node by creating a privileged debug Pod
kubectl debug node/my-node -it --image=ubuntu

# Copy logs from a crashed pod before it is garbage collected
kubectl logs <pod> --previous > crash.log 2>&1

# Stream logs from multiple pods with a label selector
kubectl logs -l app=payment-service --all-containers --prefix --follow

# Check what environment variables a running pod has
kubectl exec myapp-pod -- printenv | sort

# Test network connectivity from inside a pod
kubectl exec myapp-pod -- curl -s http://other-service.default.svc.cluster.local:8080/health
kubectl exec myapp-pod -- nslookup other-service.default.svc.cluster.local

# Check mounted volumes and config
kubectl exec myapp-pod -- df -h
kubectl exec myapp-pod -- mount | grep config

Managing Deployments

The apply, set image, scale, and rollout commands are how you modify cluster state. These are write operations — they change what is running. Use them with the same care as database writes.

kubectl apply is the declarative entry point. It reads a YAML file, compares it with the live state, and sends a PATCH request to the API server to reconcile the difference. This is idempotent — running apply twice with the same file produces no change.

kubectl rollout undo is the most important safety net. It reverts a Deployment to the previous ReplicaSet revision. This is not a delete-and-recreate — it is a controlled rollback that respects rolling update parameters (maxUnavailable, maxSurge).

kubectl delete is the imperative counterpart to apply. It removes resources. For Deployments, it deletes the Deployment and its ReplicaSets but does not delete the Pods — the Pods become orphaned until garbage collected. Always prefer apply with a modified YAML over delete-then-create.

# Apply config
kubectl apply -f deployment.yaml
kubectl apply -f ./k8s/              # apply all files in directory

# Update image
kubectl set image deployment/myapp api=myapp:2.0

# Scale
kubectl scale deployment myapp --replicas=5

# Rollout management
kubectl rollout status deployment/myapp
kubectl rollout history deployment/myapp
kubectl rollout undo deployment/myapp                    # rollback one step
kubectl rollout undo deployment/myapp --to-revision=2    # rollback to specific revision

# Delete
kubectl delete pod myapp-pod
kubectl delete deployment myapp
kubectl delete -f deployment.yaml   # delete what was applied

# Force delete a stuck pod
kubectl delete pod myapp-pod --force --grace-period=0

# ── PRODUCTION-GRADE DEPLOYMENT MANAGEMENT ───────────────────────────────

# Apply with server-side diff (dry-run) — see what would change
kubectl apply -f deployment.yaml --server-dry-run --diff

# Apply with field validation — reject invalid manifests
kubectl apply -f deployment.yaml --validate=true --server-side

# Restart all pods in a deployment (rolling restart)
kubectl rollout restart deployment/myapp

# Pause a rollout mid-way (canary testing)
kubectl rollout pause deployment/myapp
# ... verify canary pods ...
kubectl rollout resume deployment/myapp

# Check rollout revision details
kubectl rollout history deployment/myapp --revision=3

# Scale with resource-aware scripting
CURRENT=$(kubectl get deployment myapp -o jsonpath='{.spec.replicas}')
DESIRED=$((CURRENT + 2))
kubectl scale deployment myapp --replicas=$DESIRED
echo "Scaled from $CURRENT to $DESIRED replicas"

# Delete with label selector (dangerous — always dry-run first)
kubectl delete pods -l app=temp-worker --dry-run=client
kubectl delete pods -l app=temp-worker

# Apply with pruning — delete resources not in the applied set
kubectl apply -f ./k8s/ --prune -l app=myapp

# Export current state before making changes (safety myapp-backup-$(date +%Y%m%d-%H%M%S).yaml

Kubectl Context and Configuration: The Cluster You Think You're In

You can have five clusters configured, but kubectl only cares about one: the current-context. Forget this and you'll run a production delete against staging at 3 AM. I've seen it. The kubeconfig file lives at ~/.kube/config by default. That file is a list of clusters, users, and contexts — but the 'current-context' field decides which one kubectl touches.

Before any destructive command, run kubectl config current-context. Then kubectl config get-contexts to see all available contexts and their namespaces. If you need to switch, use kubectl config use-context prod-east-1. For quick namespace switching without juggling full context, kubectl config set-context --current --namespace=payments-api saves you from typing -n on every call.

Merging kubeconfig files? Export KUBECONFIG=/path/to/file1:/path/to/file2 and kubectl config view --flatten. This is how you handle multi-cluster setups without a catalog of aliases.

// io.thecodeforge — devops tutorial

// check current context before a rollout
kubectl config current-context
// output: gke_production-us-east1_payments-cluster

// switch to a different cluster
kubectl config use-context gke_staging-us-west1_payments-staging
// output: Switched to context "gke_staging-us-west1_payments-staging".

// set default namespace for current context
kubectl config set-context --current --namespace=billing-service
// output: Context "gke_staging-us-west1_payments-staging" modified.

kubectl apply: Declarative Over Imperative, or Regret It Later

kubectl create and kubectl run are imperative — they work now, but they bypass version control and drift your cluster from source truth. kubectl apply is the declarative hammer. It reads a YAML file and ensures the cluster state matches that file. Run it once, run it in CI/CD, run it from GitOps — it idempotent.

But here's the gotcha: kubectl apply merges patches, it doesn't replace. If someone manually edited a Deployment (and someone will), kubectl apply will merge those changes with your file. That's how you get fields in the live object that exist nowhere in your manifests. Solution: --server-side apply or Kustomize's prune labels.

Always use --dry-run=client first, then --dry-run=server to check validation against the API server. Then drop the flag. And for the love of reproducible builds, never pipe curl into kubectl apply — that's how you deploy a rootkit.

// io.thecodeforge — devops tutorial

// dry-run before apply
kubectl apply -f deployment-payments-v2.yaml --dry-run=client
// output: deployment.apps/payments-api created (dry run)

// actual apply with server-side validation
kubectl apply --server-side -f deployment-payments-v2.yaml
// output: deployment.apps/payments-api serverside-applied

// check what changed
kubectl diff -f deployment-payments-v2.yaml

Deleting Resources: Clean Up Before Your Cluster Bleeds Out

Deleting is not a panic button — it's maintenance. Every dangling Pod, orphaned Service, or stale ConfigMap is debt. Your cluster doesn't get a vacation; it keeps polling, routing, and logging garbage. Delete deliberately.

kubectl delete removes resources by name, label, or file. But watch the cascade: deleting a Deployment kills its ReplicaSet and Pods. Deleting a Namespace nukes everything inside — no undo. Add --cascade=orphan if you want to detach but not destroy. For troubleshooting, kubectl delete pod --force --grace-period=0 is the ripper, but only use it when a pod hangs on Terminating. Prefer graceful deletion: let your containers catch SIGTERM, not SIGKILL.

Production rule: never delete resources you didn't create. Use kubectl diff first, or label your own objects with owner: me. The kubectl delete command expects type/name syntax, not random guesses — kubectl delete pod my-pod, not kubectl delete my-pod.

// io.thecodeforge — devops tutorial

# Delete a specific pod
kubectl delete pod web-app-7f8b9c

# Delete all pods with a label
kubectl delete pods -l app=frontend

# Force delete a stuck pod (last resort)
kubectl delete pod stuck-pod --force --grace-period=0

# Delete a deployment (cascades to ReplicaSet + Pods)
kubectl delete deployment web-app

# Orphan the ReplicaSet but keep it alive
kubectl delete deployment web-app --cascade=orphan

# Delete everything in a namespace (careful!)
kubectl delete namespace staging --wait=false

Describing Resources: Stop Guessing, Start Digging

kubectl describe is your X-ray. kubectl get shows you the label on the box; describe shows you the manufacturing defects. When a pod is crashing, a service has no endpoints, or a node is tainted, describe tells you why in plain text — including events, conditions, and last state logs.

The syntax: kubectl describe <resource>/<name>. For pods, you get the full container state, restart count, and recent events. For nodes, you get capacity, allocatable resources, and taints. For services, you get endpoint lists and selector matching. This is your first debugging step — not Stack Overflow.

Pro tip: run kubectl describe pod with the pod name, then scroll to the Events section. That's where Kubernetes whispers what went wrong — image pull errors, liveness probe failures, OOM kills. If you see BackOff or CrashLoopBackOff, don't restart — read the event. Also, kubectl describe works on Deployments, Services, Nodes, even PersistentVolumeClaims. It's the universal interrogator.

// io.thecodeforge — devops tutorial

# Describe a specific pod (see events, state, conditions)
kubectl describe pod web-app-7f8b9c -n production

# Describe a service to check endpoints
kubectl describe service api-gateway

# Describe a node for capacity and taints
kubectl describe node worker-2

# Describe a deployment (rollout status, selector)
kubectl describe deployment frontend

# Shortcut: describe all resources of a type
kubectl describe pods

# Combine with label filter
kubectl describe pods -l app=backend