The Complete Guide to Kubernetes

Kubernetes has become the de facto standard for container orchestration, powering everything from startup MVPs to planet-scale infrastructure. This guide covers the core concepts, from pods and services to networking and storage, and provides practical guidance on when and how to adopt it.

Why Kubernetes Exists

Before Kubernetes, deploying applications meant manually provisioning servers, configuring software, and hoping that the production environment matched the development environment. Containers (especially Docker) solved the packaging problem: your application and all its dependencies bundled into a single, portable image. But containers created a new problem.

When you have hundreds of containers across dozens of machines, who decides which container runs where? What happens when a container crashes at 3 AM? How do containers find each other? How do you roll out updates without downtime? How do you scale up during a traffic spike and scale down when it passes?

Kubernetes (often abbreviated K8s, because there are 8 letters between the K and the s) answers these questions. It is a container orchestration platform that automates deployment, scaling, and management of containerized applications. Google developed it internally (drawing on over a decade of experience with their internal systems Borg and Omega) and open-sourced it in 2014. Today it is maintained by the Cloud Native Computing Foundation (CNCF) and has become the industry standard for container orchestration.

The name comes from the Greek word for "helmsman" or "pilot," which is why the logo is a ship's wheel. The nautical theme permeates the ecosystem: Helm (the package manager), Istio (the service mesh, from the Greek for "sail"), and Harbor (the container registry).

Architecture Overview

A Kubernetes cluster consists of a control plane and one or more worker nodes.

The control plane manages the cluster. Its components include the API server (the front door for all cluster operations), etcd (a distributed key-value store holding all cluster state), the scheduler (decides which node runs each Pod), and the controller manager (runs control loops that reconcile actual state with desired state).

Worker nodes run your actual applications. Each node runs a kubelet (an agent that communicates with the control plane and manages Pods on the node), a container runtime (containerd or CRI-O, which actually runs containers), and kube-proxy (handles network routing for Services).

The declarative model is central to how Kubernetes works. You declare the desired state ("I want 3 replicas of my web app running"), and the control plane continuously works to make reality match that declaration. If a Pod crashes, the controller notices the discrepancy and creates a replacement. This reconciliation loop runs constantly, making the system self-healing.

Core Concepts

Pods

A Pod is the smallest deployable unit in Kubernetes. It wraps one or more containers that share the same network namespace (they can communicate via localhost) and storage volumes.

In practice, most Pods contain a single container. Multi-container Pods are used for sidecar patterns: a main application container alongside a logging agent, a metrics collector, a proxy (like Envoy), or an init container that runs setup tasks before the main container starts. Init containers are particularly useful for tasks like waiting for a database to be ready, populating a shared volume, or fetching configuration from an external service.

Pods are ephemeral. They can be created, destroyed, and replaced at any time. A Pod that is killed is not restarted; a new Pod is created in its place, potentially on a different node. You should never depend on a specific Pod instance, its IP address, or its local storage persisting. Instead, you use higher-level abstractions that manage Pods for you.

Each Pod gets its own IP address within the cluster network. Containers within a Pod share that IP and can communicate via localhost on different ports. This design simplifies the common pattern of running an application with a local sidecar.

Deployments

A Deployment declares the desired state for your application: which container image to run, how many replicas, resource requirements, and update strategy. The Deployment controller continuously monitors the actual state and takes action to match the desired state.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-api
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web-api
  template:
    metadata:
      labels:
        app: web-api
    spec:
      containers:
      - name: web-api
        image: myregistry/web-api:v2.1.0
        ports:
        - containerPort: 8080
        resources:
          requests:
            memory: "256Mi"
            cpu: "250m"
          limits:
            memory: "512Mi"
            cpu: "500m"
        env:
        - name: DATABASE_URL
          valueFrom:
            secretKeyRef:
              name: db-credentials
              key: url

This tells Kubernetes: "I want 3 instances of my web-api container running at all times, each with at least 256Mi memory and 250m CPU, pulling the database URL from a Secret." If a Pod crashes, the Deployment controller spins up a replacement. If a node goes down, the Pods are rescheduled to healthy nodes. You describe what you want; Kubernetes makes it happen.

Services

Pods get ephemeral IP addresses that change whenever a Pod is replaced. Services provide stable network endpoints that route traffic to the right Pods, regardless of which node they run on or how many replicas exist.

A ClusterIP Service (the default) creates an internal virtual IP address reachable only within the cluster. Other Pods use this to communicate with your application. The Service load-balances across all matching Pods.

A NodePort Service exposes the application on a static port (30000-32767) on every node's IP. External traffic can reach the application via any node's IP and the assigned port. Simple but limited: you expose raw ports, not clean URLs.

A LoadBalancer Service provisions a cloud provider's load balancer (AWS ELB/ALB, GCP Load Balancer, Azure Load Balancer) that routes external traffic to the Service. This is the standard way to expose applications to the internet on managed Kubernetes.

An ExternalName Service maps a Service to a DNS name, acting as a CNAME redirect. Useful for referencing external services (like a managed database) through the Kubernetes DNS system.

Services use label selectors to find their target Pods. Any Pod with matching labels receives traffic. This decoupling means you can update Deployments, scale replicas, or even replace the underlying application, and the Service continues routing correctly.

ConfigMaps and Secrets

ConfigMaps store non-sensitive configuration data as key-value pairs. Applications read them as environment variables or mounted files. Changing a ConfigMap and restarting Pods is cleaner than rebuilding container images for configuration changes.

Secrets store sensitive data (passwords, API keys, TLS certificates). They are base64-encoded (not encrypted) by default. For real security, enable encryption at rest in etcd and use an external secrets manager (Vault, AWS Secrets Manager) via the External Secrets Operator.

Namespaces

Namespaces partition a single cluster into virtual clusters. The default namespace is where resources go if you do not specify one. Common patterns include separate namespaces for different environments (dev, staging, production), different teams, or different applications.

Namespaces provide scope for names (two Deployments can have the same name in different namespaces), resource quotas (limit how much CPU and memory a namespace can consume), and access control via RBAC (restrict who can do what in which namespace).

Networking

Kubernetes networking follows a flat model: every Pod can communicate with every other Pod without NAT. Each Pod gets its own IP address. This simplicity is enforced by the Container Network Interface (CNI) plugin, which handles the actual network implementation.

DNS and Service Discovery

Kubernetes runs an internal DNS server (CoreDNS). Services are automatically registered with DNS names following a predictable pattern. A Service named web-api in namespace production is reachable at:

This DNS-based service discovery eliminates hard-coded IP addresses and enables loose coupling between services. Your application connects to database.production and Kubernetes handles resolving that to the current set of database Pods.

Ingress and Gateway API

Ingress resources manage external HTTP/HTTPS access to Services. An Ingress controller (like nginx-ingress, Traefik, or Contour) watches for Ingress resources and configures routing rules.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web-ingress
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
  tls:
  - hosts:
    - api.example.com
    secretName: api-tls
  rules:
  - host: api.example.com
    http:
      paths:
      - path: /v1
        pathType: Prefix
        backend:
          service:
            name: web-api-v1
            port:
              number: 8080
      - path: /v2
        pathType: Prefix
        backend:
          service:
            name: web-api-v2
            port:
              number: 8080

The Gateway API is the newer, more expressive replacement for Ingress. It separates infrastructure concerns (what load balancer to use) from application concerns (how to route traffic), supports TCP/UDP in addition to HTTP, and provides more granular control over traffic policies.

Network Policies

By default, all Pods can communicate with all other Pods. This is convenient for development but dangerous for production. Network Policies restrict this communication using label selectors.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: api-policy
spec:
  podSelector:
    matchLabels:
      app: web-api
  ingress:
  - from:
    - podSelector:
        matchLabels:
          role: frontend
    ports:
    - port: 8080

This says: only Pods with the label role: frontend can connect to web-api Pods on port 8080. All other ingress traffic is denied. Network Policies require a CNI plugin that supports them (Calico, Cilium, Antrea). The default kubenet plugin does not enforce them.

Storage

Containers have ephemeral filesystems. When a container restarts, all written data is lost. Kubernetes provides persistent storage through Volumes, Persistent Volume Claims, and Storage Classes.

Persistent Volumes and Claims

A Persistent Volume (PV) is a piece of storage provisioned by an administrator or dynamically by a storage class. It exists independently of any Pod and has its own lifecycle.

A Persistent Volume Claim (PVC) is a request for storage by a Pod. Kubernetes binds the PVC to an available PV that meets the requirements (size, access mode, storage class).

Storage Classes define different types of storage. Dynamic provisioning creates PVs automatically when a PVC is created. This is the standard pattern in cloud environments:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: fast-ssd
provisioner: ebs.csi.aws.com
parameters:
  type: gp3
  iops: "5000"
reclaimPolicy: Retain
volumeBindingMode: WaitForFirstConsumer

Access modes control how volumes are mounted: ReadWriteOnce (one node), ReadOnlyMany (many nodes, read-only), and ReadWriteMany (many nodes, read-write). Not all storage backends support all access modes.

StatefulSets

For stateful applications (databases, message queues, distributed caches), Kubernetes provides StatefulSets. Unlike Deployments, StatefulSets provide:

StatefulSets are essential for applications that need to know their identity (like database replicas that need to know if they are the primary or a replica) or that need stable storage (like a database that must not lose its data when rescheduled).

Helm: Package Management

Helm manages Kubernetes applications as packages called "charts." A chart bundles all the YAML manifests for an application with templating and configuration.

helm install my-postgres bitnami/postgresql \
  --set auth.postgresPassword=mypassword \
  --set primary.persistence.size=100Gi \
  --namespace databases \
  --create-namespace

This single command creates a Deployment, Service, ConfigMap, Secret, PVC, and ServiceAccount for a PostgreSQL installation. Helm templates use Go's template language to inject values, making charts reusable across environments.

Values files customize charts for different environments:

# values-production.yaml
replicaCount: 3
resources:
  requests:
    memory: 2Gi
    cpu: "1"
persistence:
  size: 500Gi
  storageClass: fast-ssd

Helm handles versioning (upgrade, rollback to any previous release), dependency management (a chart can depend on other charts), and configuration management. The Artifact Hub hosts thousands of community charts for common applications.

Monitoring and Observability

Metrics with Prometheus

Prometheus is the standard monitoring solution for Kubernetes. It scrapes metrics from Pods (via HTTP endpoints), nodes, and Kubernetes components on a configurable interval. Grafana provides dashboards for visualization. AlertManager handles alert routing and notification.

Key metrics to monitor: Pod CPU and memory usage versus requests and limits, request latency percentiles (p50, p95, p99), error rates (5xx responses), Pod restart counts (frequent restarts indicate application instability), node resource utilization, and persistent volume usage.

The kube-state-metrics service exposes Kubernetes object state (Deployment replica count, Pod phase, Job status) as Prometheus metrics. Combined with node-level metrics from the node-exporter, you get comprehensive cluster observability.

Logging

Kubernetes does not provide built-in log aggregation. Containers should log to stdout/stderr (following the 12-factor app methodology), and a logging infrastructure collects these streams.

Common solutions: the EFK stack (Elasticsearch, Fluentd, Kibana) or Loki with Grafana. Loki is increasingly popular because it indexes only metadata (labels), not the full log text, making it much cheaper to operate at scale.

Health Checks (Probes)

Kubernetes uses probes to monitor container health. Configure all three for production workloads.

Liveness probes: "Is this container alive?" If a liveness probe fails, Kubernetes kills the container and starts a new one. Use for detecting deadlocks, infinite loops, and unrecoverable states. Be conservative: a liveness probe that fails too easily causes unnecessary restarts.

Readiness probes: "Is this container ready to serve traffic?" If a readiness probe fails, the container is removed from Service endpoints but not killed. Use for startup warmup periods, temporary overload conditions, and dependency checks (e.g., the database is not yet reachable).

Startup probes: "Has this container finished starting?" For applications with long startup times (JVM warmup, model loading), a startup probe prevents the liveness probe from killing the container before it is ready. The startup probe runs first; once it succeeds, the liveness and readiness probes take over.

Scaling

Horizontal Pod Autoscaler (HPA)

HPA automatically adjusts the number of Pod replicas based on observed metrics.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-api-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-api
  minReplicas: 2
  maxReplicas: 20
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300

The behavior field controls scaling aggressiveness. The stabilization window prevents flapping: scaling down waits 5 minutes to confirm the load decrease is sustained. You can also limit the rate of scaling (e.g., add at most 4 Pods per minute).

Custom metrics (from Prometheus via the Prometheus Adapter) enable scaling on application-specific signals like request queue depth, in-flight requests, or business metrics.

Cluster Autoscaler

While HPA scales Pods within existing nodes, the Cluster Autoscaler adds or removes nodes. If Pods cannot be scheduled because no node has enough resources, the Cluster Autoscaler provisions new nodes from the cloud provider. If nodes are underutilized (typically below 50% utilization for 10 minutes), it removes them to save costs.

Karpenter (AWS) is a newer alternative that provisions nodes more quickly and efficiently by requesting the right instance types for pending Pods rather than scaling predefined node groups.

Vertical Pod Autoscaler (VPA)

VPA adjusts CPU and memory requests/limits for individual Pods based on observed usage. It monitors actual resource consumption and recommends (or automatically applies) more appropriate settings.

VPA and HPA should not target the same resource metric for the same Deployment. Use HPA for scaling out (more replicas) and VPA for right-sizing (correct resources per replica).

Rolling Updates and Rollbacks

Deployments support rolling updates by default. When you update a container image, Kubernetes gradually replaces old Pods with new ones.

spec:
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 0

maxSurge: How many extra Pods can exist during the update (absolute number or percentage). maxUnavailable: How many Pods can be unavailable during the update. Setting maxUnavailable to 0 ensures zero-downtime deployments at the cost of requiring extra capacity during the rollout.

Kubernetes monitors the rollout and pauses automatically if new Pods fail their readiness probes. This prevents a bad deployment from taking down all replicas.

Roll back to a previous revision:

kubectl rollout undo deployment/web-api
kubectl rollout undo deployment/web-api --to-revision=3

For more sophisticated deployment strategies, consider progressive delivery tools like Argo Rollouts or Flagger, which support canary deployments (route a small percentage of traffic to the new version), blue-green deployments (run two complete versions and switch traffic), and automated rollback based on metrics.

When to Use Kubernetes

Kubernetes adds significant complexity. It is not always the right choice. Be honest about the tradeoffs.

Use Kubernetes when: You run multiple services that need to communicate and scale independently. You need automated scaling and self-healing. Your team has (or can develop) Kubernetes expertise. You deploy across multiple cloud providers or hybrid cloud/on-premises. Your services have different resource requirements and scaling characteristics. You need sophisticated deployment strategies (canary, blue-green). Your scale justifies the operational overhead.

Do not use Kubernetes when: You have a single application with simple scaling needs (use a PaaS like Heroku, Fly.io, Railway, or Render). Your team is small and does not have Kubernetes experience (the learning curve is steep, 3-6 months for basic proficiency). Your workload is primarily serverless or event-driven (use Lambda, Cloud Functions, Cloud Run). The operational overhead exceeds the benefits (a common situation for teams under 10 engineers).

Managed Kubernetes (EKS, GKE, AKS) reduces operational burden by managing the control plane, handling upgrades, and providing integrations with cloud services. But you still need to understand Kubernetes concepts, manage worker nodes (unless using Fargate/Autopilot), and debug networking, storage, and application issues.

Common Mistakes

Not setting resource requests and limits: Without resource constraints, a single misbehaving Pod can starve other Pods on the same node. Always set both requests (guaranteed minimum, used for scheduling) and limits (maximum allowed, enforced by the kernel). Requests that are too low cause scheduling problems; limits that are too low cause OOM kills.

Running as root: Containers should run as non-root users. Use securityContext in your Pod spec to enforce this. Running as root inside a container is a security risk even with container isolation. Set runAsNonRoot: true and readOnlyRootFilesystem: true where possible.

Skipping health checks: Without liveness and readiness probes, Kubernetes cannot detect and recover from application failures. It sees only whether the container process is running, not whether it is healthy. Health checks are not optional for production workloads.

Ignoring resource namespaces: Putting everything in the default namespace creates a flat, unmanageable structure. Use namespaces to organize workloads, enforce resource quotas, and apply RBAC policies.

Treating Pods as pets: Pods are cattle, not pets. They should be replaceable at any time without notice. If your application cannot handle a Pod being killed and replaced (losing in-memory state, open connections, pending work), fix the application, not the infrastructure. Design for graceful shutdown using preStop hooks and SIGTERM handling.

Over-provisioning resources: Requesting more CPU and memory than needed wastes money, especially in cloud environments where you pay for what you allocate. Use VPA recommendations and monitoring data to right-size your resource requests. The gap between requests and actual usage is pure waste.

YAML sprawl: As your cluster grows, managing hundreds of YAML files becomes unwieldy. Use Helm charts for templating, Kustomize for environment-specific overlays, and GitOps tools (ArgoCD, Flux) to keep your cluster state in sync with Git.

Neglecting Pod Disruption Budgets: Without PDBs, cluster maintenance (node upgrades, scaling down) can take down all replicas of a service simultaneously. PDBs guarantee that a minimum number of Pods remain available during voluntary disruptions.

The Kubernetes Ecosystem

Kubernetes is a platform for building platforms. The core system is deliberately minimal, with an ecosystem of extensions filling specific needs.

Service meshes (Istio, Linkerd, Cilium Service Mesh) add observability, traffic management, mTLS encryption, and retry/timeout policies between services. Powerful but complex; adopt only when you need the specific capabilities.

GitOps tools (ArgoCD, Flux) synchronize cluster state from Git repositories. Every change goes through a pull request, providing audit trails and easy rollback. This is the recommended approach for managing production clusters.

Certificate management (cert-manager) automates TLS certificate provisioning and renewal from Let's Encrypt and other issuers.

Policy engines (OPA/Gatekeeper, Kyverno) enforce organizational policies: "all Pods must have resource limits," "no containers may run as root," "images must come from approved registries."

Secrets management (Sealed Secrets, External Secrets Operator) integrates with external secret stores (Vault, AWS Secrets Manager, GCP Secret Manager) to keep sensitive data out of Git while still managing it declaratively.

The ecosystem is vast and can be overwhelming. Start with the basics: Deployments, Services, Ingress, and a monitoring stack. Add tools only when you have specific problems they solve. Premature adoption of ecosystem tools is a common source of unnecessary complexity that teams spend months managing instead of building their actual product.

Kubernetes is not the destination. It is infrastructure that should fade into the background once configured correctly. The goal is reliable, scalable application delivery. Kubernetes is one means to that end, powerful when applied thoughtfully, burdensome when adopted without clear justification.