ZephyrCode teardown · kubernetes · No. 03

The EKS cluster you inherited runs on defaults.

A fresh EKS cluster answers the internet, logs nothing, scales nothing, and treats your workloads as eviction fodder — all by documented default. Here are eight, each with the exact setting, the mechanism, and the fix. This is the format a fixed-price ZephyrCode audit produces, run against the stock configuration instead of your system.

What this is (and isn’t)

Every claim below is about the documented defaults of a fresh Amazon EKS cluster and stock Kubernetes semantics — sourced to the AWS EKS user guide and kubernetes.io at each finding, verified July 2026 — not any team’s private production system. Before/after figures are labelled typical: representative of the failure class, not any client’s numbers. If your platform team built a golden path, some of these are already fixed — this teardown is the checklist for finding out which.

Three things this teardown deliberately does not claim: (1) that the public endpoint means an open cluster — access is authenticated (IAM + RBAC); the finding is about exposure surface, not anonymous access. (2) that EKS is misconfigured out of the box — these defaults optimise for first-contact success, and AWS documents every knob; the failure is inheriting them unread. (3) any claim about managed add-on versions, which change frequently — everything here is either a stable documented default or core Kubernetes semantics.

8 findings · High → Medium
High severityFINDING 01 / 08

The API server is on the internet

Pages security, eventually — and until then, nobody.

AWS’s own docs say it plainly: “By default, this API server endpoint is public to the internet.” Access is still authenticated — IAM plus RBAC — but the front door of your cluster’s control plane answers to the whole internet, which means credential mistakes, token leaks, and future auth CVEs are internet-exploitable instead of VPC-exploitable. Most teams never revisit it, because the cluster worked on day one and the endpoint setting lives in a console tab nobody opens twice.

# a fresh cluster's endpoint access
endpointPublicAccess  = true◂ the control plane answers the internet
endpointPrivateAccess = false
publicAccessCidrs     = 0.0.0.0/0
Before · default
Control-plane auth surface exposed to the internet
After · fixed
API server reachable only from networks you enumerate
Recommended fix

Enable the private endpoint; then either restrict publicAccessCidrs to your offices/VPN or disable public access outright. Do it early — the change is disruptive to bolt on after tooling has assumed a public endpoint.

Tradeoff — Private-only access needs a path into the VPC (VPN, transit gateway, bastion, or CloudShell) — and your CI’s kubectl needs that path too.

High severityFINDING 02 / 08

The control plane is silent

Pages whoever runs the incident where “who deleted that?” has no answer.

“By default, cluster control plane logs aren’t sent to CloudWatch Logs.” All five log types — api, audit, authenticator, controllerManager, scheduler — ship turned off. That means no audit trail of who did what to the cluster, no authenticator record of which IAM identity became which Kubernetes user, and no API-server logs when you’re debugging a webhook that’s rejecting everything. The first time most teams enable audit logging is the day they need last month’s audit log, which is exactly the day it doesn’t exist.

# a fresh cluster's logging config
clusterLogging: api            = off◂ all five types ship OFF
clusterLogging: audit          = off◂ "who deleted it?" — no record
clusterLogging: authenticator  = off
clusterLogging: controllerManager = off
clusterLogging: scheduler      = off
Before · default
No audit trail; incident forensics start from nothing
After · fixed
Every control-plane action attributable, with bounded retention cost
Recommended fix

Enable at least audit + authenticator (compliance and forensics) and api (debugging) on every non-toy cluster, with a CloudWatch retention policy so cost stays bounded.

Tradeoff — CloudWatch ingestion and storage costs — real on chatty clusters, which is why you set retention instead of leaving logs off.

High severityFINDING 03 / 08

Your workloads are eviction fodder

Pages the team whose service dies whenever a neighbour gets busy.

Kubernetes doesn’t require resource requests or limits, so the deployment everyone copies ships without them — and every such pod runs in the BestEffort QoS class. Two consequences, both silent. The scheduler packs blind: with no requests, it can’t know a node is full, so it stacks heavy pods together and lets them fight. And under node memory pressure, BestEffort pods are the first the kubelet evicts — your production service is, by definition, the sacrifice the system makes to save itself. The cluster looks fine until the day one noisy neighbour makes it look haunted.

# the deployment everyone copies
containers:
  - name: api
    resources: {}◂ BestEffort — first against the wall
Before · default
Typical: blind packing + noisy neighbours; BestEffort pods evicted first under pressure
After · fixed
Scheduler places by declared need; eviction order matches your priorities
Recommended fix

Set memory and CPU requests on everything (that’s what the scheduler plans with); set memory limits deliberately; add a LimitRange per namespace so unset pods get defaults instead of BestEffort.

Tradeoff — Right-sizing takes measurement — requests too high waste nodes, too low recreate the problem. Start from observed usage, not guesses.

High severityFINDING 04 / 08

The subnet runs out before the node does

Pages whoever gets the “pods stuck in ContainerCreating” ticket on a half-empty node.

The VPC CNI’s default mode assigns every pod its own secondary IP address from your subnet, one at a time. Two failure shapes ship with that default. In small subnets — the /24s everyone carves — the subnet exhausts long before the nodes do: CPU sits free while pods hang in ContainerCreating because there is literally no IP to give them. And per AWS’s own docs, the default mode “must make more Amazon EC2 API calls to configure network interfaces and IP addresses,” so exactly when a spiky workload scales, pod launches slow down behind EC2 API throttling. Prefix delegation — the fix — exists, is documented, and is off until you turn it on.

# amazon-vpc-cni defaults
ENABLE_PREFIX_DELEGATION = false◂ one EC2 IP per pod, one at a time
WARM_ENI_TARGET          = 1
Before · default
Typical: half-empty nodes, exhausted subnet, pods pending on IPs; slow scale-out under EC2 API throttling
After · fixed
Pod density bounded by compute, not by /24 arithmetic; fastest documented launch path
Recommended fix

Enable prefix delegation (per AWS guidance, on new node groups rather than rolling existing ones) and size subnets for pod density, not just node count.

Tradeoff — Prefix mode allocates /28 blocks — coarser IP consumption per node — and once nodes run prefixes you can’t downgrade the CNI below 1.9.0 without replacing node groups.

Medium severityFINDING 05 / 08

Routine maintenance takes your service down

Pages the on-call during a “non-disruptive” node group upgrade.

Kubernetes ships no PodDisruptionBudgets — they’re opt-in, per workload, and the tutorials skip them. Without one, a voluntary disruption (node drain, managed node-group upgrade, Karpenter consolidation) is allowed to evict every replica of your service at once; three replicas on two draining nodes is an outage the platform considers polite. The cruel part: everything is working as designed, every eviction was “graceful,” and the postmortem finds no failure — just an upgrade that walked through your quorum.

# what protects your 3-replica service during a drain
PodDisruptionBudget: none◂ all replicas evictable simultaneously
Before · default
Node upgrades may evict entire services at once
After · fixed
Drains proceed one replica at a time; upgrades stop being outages
Recommended fix

A PDB for every multi-replica service (maxUnavailable: 1 is a sane start), paired with topology spread so replicas don’t share a node/AZ to begin with.

Tradeoff — Over-strict PDBs (maxUnavailable: 0) wedge node drains and block upgrades — the budget must permit some disruption or operations stop.

Medium severityFINDING 06 / 08

Elastic in the brochure, fixed in the cluster

Pages capacity-planning-by-incident: the Friday traffic doubles.

A fresh EKS cluster includes no metrics-server, no HorizontalPodAutoscaler targets that can work without it, and no node autoscaler — neither Cluster Autoscaler nor Karpenter comes installed. So the “elastic” cluster everyone inherited is a fixed pool of nodes wearing a Kubernetes costume: HPA objects (if anyone created them) sit broken for lack of metrics, and when load exceeds the node group, pods pend until a human buys capacity. Teams discover this precisely once.

# what a fresh cluster ships for scaling
metrics-server        = not installed◂ HPA has no signal
node autoscaler       = not installed◂ pods pend; nobody buys nodes
Before · default
Fixed capacity; load spikes end in pending pods and a human
After · fixed
Pod and node capacity follow demand inside declared bounds
Recommended fix

Install metrics-server; give every scalable service an HPA with sane bounds; run Karpenter (or Cluster Autoscaler) so pending pods buy capacity instead of paging humans.

Tradeoff — Autoscalers are operational surface — bad requests (Finding 3) make them scale wrong, which is why requests come first.

Medium severityFINDING 07 / 08

Every pod can call every pod

Pages security during the audit, and incident response after the breach.

Kubernetes networking is default-allow: with no NetworkPolicies, any pod can reach any pod in any namespace — the payments service answers connections from the marketing microsite’s sidecar. Nothing enforces the architecture diagram’s neat arrows; they’re documentation, not policy. One compromised pod is a flat east-west network away from everything else you run, and the cluster you inherited almost certainly has zero policies, because zero is what ships.

# east-west traffic control, out of the box
NetworkPolicy objects = 0◂ default-allow, cluster-wide
Before · default
Flat network: one compromised pod reaches everything
After · fixed
East-west traffic matches the architecture diagram, enforced
Recommended fix

Enable network-policy enforcement (the VPC CNI supports Kubernetes NetworkPolicies), apply a default-deny per namespace, then allow the flows the architecture actually requires.

Tradeoff — Policy authoring is real work and a wrong deny is an outage — roll out namespace by namespace with observability on denied flows.

Medium severityFINDING 08 / 08

The volume that pins the pod

Pages the team whose stateful pod won’t reschedule after a node dies.

EBS volumes are zonal, and a pod bound to an EBS-backed PersistentVolume can only ever run in that volume’s availability zone. Lose the node — or just the AZ’s spare capacity — and the pod sits Pending with “volume node affinity conflict” while its data waits in an AZ that can’t host it. Multi-AZ node groups make this worse, not better: the scheduler happily created the volume wherever the first pod landed, and that accident is now a permanent placement constraint.

# the constraint nobody declared
EBS volume zone = us-east-1a◂ the pod is now zonal, forever
volumeBindingMode: WaitForFirstConsumer   # use this — binds where the pod schedules
Before · default
Node/AZ loss strands stateful pods on “volume node affinity conflict”
After · fixed
Volumes bind to schedulable zones; AZ risk is a decision, not a surprise
Recommended fix

StorageClasses with volumeBindingMode: WaitForFirstConsumer (so volumes bind where pods actually schedule), per-AZ capacity headroom for stateful workloads, and EFS/replication for data that genuinely must survive an AZ.

Tradeoff — True multi-AZ statefulness costs — EFS latency or application-level replication. The cheap alternative is accepting zonal blast radius consciously.

That’s the stock cluster. Now picture yours.

This is the exact deliverable format of a ZephyrCode audit — findings with the mechanism, the evidence, and the fix, ranked by what pages you. Run against your EKS, your Postgres, your Kafka. Fixed scope, fixed price, two weeks. Not sure it’s worth it? Start with the $1,200 48-hour triage.

ZephyrCode · engineering audits · the price is on the door · previously: Kafka · Postgres