Kubernetes Requests and Limits: Throttling, Eviction, and the 3x Guess

Requests and limits are the most-edited and least-understood fields in a Pod spec. Teams copy cpu: 500m / memory: 512Mi for three years, then wonder why p99 latency spikes when the node is busy — or why the node fills with idle pods.

Two different mechanisms share one YAML block.

Requests: the scheduler and the quiet neighbor

resources.requests is what kube-scheduler believes the pod will use. A node with 4 CPU and 2 CPU already requested will not take another 3 CPU pod, even if top says the node is idle.

resources:
  requests:
    cpu: "200m"
    memory: "256Mi"
  limits:
    cpu: "1"
    memory: "512Mi"

If you omit requests, the scheduler packs too hard. If you set requests far above real use, you pay for empty nodes. That is the 3× guess: request 3× p95 “to be safe,” then buy 3× the cluster.

CPU limits: throttling, not a hard wall

CPU is compressible. A limit of 1 means cfs_quota will throttle the container when it wants more than 1 CPU in a period. The process is not killed. It just stops getting run.

Symptoms:

• p99 jumps, p50 looks fine
• gRPC deadlines
• “it’s slow only at peak”

cpu.limit == cpu.request (Guaranteed QoS) is a common platform rule. It is also how you guarantee throttle if the number is below burst need. Many latency-sensitive teams set a request and omit the CPU limit, or set the limit several times the request.

Do not cargo-cult “must have CPU limit” from a CIS benchmark without measuring container_cpu_cfs_throttled_seconds_total.

Memory limits: OOMKill

Memory is not compressible the same way. Hit the limit and the container gets SIGKILL. requests.memory also feeds the eviction signal: kubelet evicts BestEffort / Burstable pods when the node is under memory pressure.

request < usage < limit     → usually fine
usage > limit               → OOMKill
usage > request, node full  → eviction of the noisiest Burstable pods

Set memory.request near working set (p95 from metrics) and memory.limit a bit above p99 plus a buffer for GC. A limit of 8Gi with a request of 256Mi on a fat JVM is how you get noisy-neighbor evictions.

QoS classes

Guaranteed is not “faster.” It is “evicted last.” A throttled Guaranteed pod still misses SLOs.

Getting numbers from the cluster

# CPU used, cores
rate(container_cpu_usage_seconds_total{namespace="prod",pod=~"api-.*"}[7d])

# Working set
container_memory_working_set_bytes{namespace="prod",pod=~"api-.*"}

Take p95 over a week, add ~20–30% for request. For memory limit, p99 + headroom. Revisit after a release that changes heap or concurrency.

VPA in Off / recommendation mode writes this down for you. Do not turn on VPA Auto on a latency-sensitive Deployment until you have PDBs and a rollout story.

LimitRange is a seatbelt

apiVersion: v1
kind: LimitRange
metadata:
  name: defaults
  namespace: prod
spec:
  limits:
    - type: Container
      defaultRequest:
        cpu: "100m"
        memory: "128Mi"
      default:
        cpu: "500m"
        memory: "256Mi"

This stops the “forgot resources” pod. It does not stop the 8Gi guess. Kyverno requiring some request is the other half — see Kyverno policies.

Common pitfalls

1. CPU limit = request = 100m on a Go service that needs burst for GC and JSON.
2. No memory request, huge limit — scheduler thinks the pod is tiny.
3. Copying desktop Docker numbers (2 CPU / 4Gi) into every sidecar.
4. Ignoring initContainers. Their requests count at scheduling time.
5. HPA on CPU when the pod is throttled by a limit — HPA scales out, each replica still throttles.

Practical default for a new stateless API

resources:
  requests:
    cpu: "200m"
    memory: "256Mi"
  limits:
    # cpu omitted or 4x request after you look at throttle metrics
    memory: "512Mi"

Then: one week of metrics, then a PR that updates the numbers. Requests are capacity planning. Limits are safety. Do not use the same folklore integer for both.