Skip to content

Repository files navigation

image

mnemo-jvm-warden

build coverage

JVM-aware vertical right-sizing for Kubernetes. Warden makes a running JVM give memory back to the OS during predictable low-traffic windows, then safely lowers the Pod's resource footprint — reclaiming idle infrastructure without killing the pod, dropping its warm state, or risking your high-availability posture.


The gap Warden fills

In ~95% of production environments, Horizontal Pod Autoscaling (HPA) is the right answer: scale from 10 replicas to 2 when traffic drops, and you save money. Warden does not compete with that.

But there are three well-known scenarios where removing replicas is not an option, and shrinking the pods you keep is the only lever left:

1. Minimum-replica HA baselines

High-availability architectures often can't scale below a floor — e.g. 3 sub-services × 3 replicas across 3 Availability Zones. If each JVM needs 4 GB at peak, that's 36 GB reserved overnight to serve zero requests. HPA can't touch it; the floor is there for failover, not for load.

Warden keeps all 9 replicas alive for availability but shrinks each one's footprint during the off-peak window — turning a 36 GB idle reservation into ~9 GB, without reducing replica count.

2. Slow-booting monoliths

Many enterprise Java apps (older Spring Boot, WebSphere, large multi-module builds) take 2–5 minutes to boot and pass readiness. If HPA scales you down to 2 pods overnight and a sharp spike hits at 06:00, the new pods sit in ContainerCreating/unready for minutes while the surviving 2 get slammed, spike CPU, and cascade into an outage.

Warden keeps the pods warm — JVM up, classes loaded, connection pools primed. Raising the footprint back to peak is a sub-second resize, not a cold start.

3. Stateful nodes & distributed caches

For an in-memory data grid, a stateful worker queue, or a distributed cache (this is mnemo-cache's home turf), you can't just kill a replica to save money — termination forces data rebalancing, purges local cache, and thrashes surviving nodes with replication overhead.

Warden instead shrinks in place: it flushes expired/evictable entries and idle pages, runs a deep GC, uncommits the freed pages back to the OS, and keeps the node running at a lower price point — with its hot working set intact.


How it actually works (and where the money comes from)

A few load-bearing truths that shape the entire design:

Cost lives in requests, not limits. Lowering a Pod's memory limit saves nothing — you still pay for the node. Kubernetes cost is driven by requests, because that's what the scheduler bin-packs against and what Cluster Autoscaler uses to decide it can drain and delete a node.

Warden's action boundary ends at "lower the request." It resizes an existing pod in place — it never changes the number of pods (that's HPA) or the number of nodes (that's the autoscaler). Everything past the request change is emergent cluster behaviour that Warden enables but does not perform, and does not guarantee:

Two zones separated by a hand-off. WARDEN does: release JVM memory (GC + uncommit), verify RSS actually dropped (safety gate), lower the pod's memory request in place — Warden's responsibility ends at the request change. KUBERNETES does (requires a Cluster Autoscaler, not Warden's action): scheduler bin-packs the freed reservation, autoscaler drains a node, actual money saved. With no autoscaler, capacity is freed but nothing is billed-saved.

The is a hand-off, not a step Warden takes. On a fixed node pool with no autoscaler, Warden's lower-request still frees schedulable capacity but zero billed capacity — so the savings story is only as real as your consolidation setup.

You can't change -Xmx on a running JVM. Max heap is fixed at launch. What is movable at runtime is the soft ceiling and committed memory:

  • ZGCSoftMaxHeapSize, settable live via jcmd VM.set_flag, plus automatic uncommit of unused heap.
  • Shenandoah — aggressive uncommit via -XX:ShenandoahUncommitDelay (the gold standard for returning memory to the OS).
  • G1-XX:G1PeriodicGCInterval / -XX:G1PeriodicGCSystemLoadThreshold already trigger idle GC + uncommit.

The JVM already does half of this. Warden's job is the orchestration — it drives these knobs from a traffic schedule and coordinates them with a Kubernetes in-place Pod resize, which the JVM has no way to do on its own.


The safety handshake

The shrink and grow sequences are not symmetric. Get the order wrong and you OOMKilled the pod. This ordering is the core correctness guarantee of the tool:

sequenceDiagram
    participant Sched as Traffic Schedule
    participant W as Warden Agent
    participant JVM
    participant K8s as Kubelet (cgroup)

    Note over Sched,K8s: SHRINK — off-peak window begins
    Sched->>W: low-traffic window predicted
    W->>JVM: lower SoftMaxHeapSize / flush evictable cache
    W->>JVM: force deep GC + uncommit pages
    JVM-->>W: RSS dropped, pages returned to OS
    W->>W: verify actual RSS < target limit  ✅ gate
    W->>K8s: in-place resize DOWN (request + limit)
    Note right of W: cgroup lowered ONLY after RSS confirmed

    Note over Sched,K8s: GROW — traffic returning
    Sched->>W: peak window predicted
    W->>K8s: in-place resize UP (request + limit) FIRST
    K8s-->>W: headroom granted
    W->>JVM: raise SoftMaxHeapSize
    W->>JVM: (optional) pre-warm cache ahead of load
Loading
  • Shrinking: shrink the JVM first, verify RSS actually fell, then lower the cgroup. Lowering the cgroup before the JVM has released memory = instant OOMKill.
  • Growing: raise the cgroup first, then raise the JVM's soft max. Give the headroom before the JVM tries to use it.
  • Cache re-warming happens ahead of the predicted traffic return — a flushed cache is itself a cold start, so Warden shrinks early and re-warms early rather than reactively.

Warden vs. VPA

Warden's real neighbor isn't HPA — it's the Vertical Pod Autoscaler, which now also supports in-place updates. The difference is that VPA is JVM-blind:

VPA mnemo-jvm-warden
Resizes the Pod in place
Can cause the JVM to release memory ❌ observes RSS, can't move it ✅ drives soft-max, GC, uncommit
Coordinates GC/uncommit with the resize ✅ ordered handshake
Understands cache / working-set state ✅ flushes evictable, keeps hot set
Driven by predicted traffic curves ❌ reactive to observed usage ✅ proactive, schedule-aware

VPA resizes the box and hopes the JVM cooperates. Warden makes the JVM cooperate first, then resizes.


Deploying with Helm

Two charts, two different jobs. Both are published to a Helm repo hosted on GitHub Pages, so no local checkout is needed:

helm repo add warden https://baokhang83.github.io/mnemo-jvm-warden/
helm repo update

(You can still install straight from a local checkout — helm install warden charts/warden — if you're working on the charts themselves.)

charts/warden — installs the controller

helm install warden warden/warden

This is the one-shot, cluster-wide install: it applies the WardenPolicy CRD (once — Helm's crds/ directory is never touched again by helm upgrade, see charts/warden/crds/), a ClusterRole/ClusterRoleBinding scoped to exactly what the controller reads and writes (watch WardenPolicy, patch its status, read Deployment/StatefulSet, read/patch Pod annotations), and the controller Deployment itself (hardcoded to replicas: 1 — there's no leader election yet, so a second replica would double-reconcile every policy).

Values worth knowing (charts/warden/values.yaml):

key what it controls
controller.image.repository / .tag which controller image to run
controller.prometheusUrl where Prometheus lives, for guardrail metric evaluation (W-401) — leave empty if no policy uses a guardrail
controller.resources the controller Pod's own request/limit (defaults to 256Mi request / 512Mi limit — the controller is a JVM with a heavy dependency graph, so its non-heap memory needs real headroom; 256Mi was too tight and OOMKilled it under load)

charts/warden-sidecar — the reusable sidecar template

There is no admission webhook. "Injecting" the sidecar means an operator's own app chart includes a Helm named template that renders the same native-sidecar shape as deploy/example-sidecar.yaml, parameterized instead of hand-copied. It's a Helm library chart (type: library) on purpose: depending on it can never deploy a second controller alongside your app, because a library chart is structurally forbidden from rendering any resources of its own.

In your own app chart's Chart.yaml:

dependencies:
  - name: warden-sidecar
    version: "0.1.4"   # match the published chart version (see `helm search repo warden`)
    repository: "https://baokhang83.github.io/mnemo-jvm-warden/"

In your own values.yaml, a warden: block shaped like charts/warden/values.yaml's sidecar: section (targetContainerName and resources are required, with no default — see that file's comments for why):

warden:
  enabled: true
  image:
    repository: ghcr.io/baokhang83/mnemo-jvm-warden
    tag: latest
  targetContainerName: app   # the sibling container Warden resizes
  resources:
    requests: { cpu: 25m, memory: 64Mi }
    limits: { memory: 256Mi }

And in your own Pod template:

spec:
  shareProcessNamespace: true   # lets the sidecar see the target JVM's PID (W-102)
  volumes:
    {{- include "warden.sidecar.volumes" . | nindent 4 }}
  initContainers:
    {{- if .Values.warden.enabled }}
    {{- include "warden.sidecar" (dict "cfg" .Values.warden) | nindent 4 }}
    {{- end }}
  containers:
    - name: app   # must match warden.targetContainerName above
      # ... your app container, with the JMX flags deploy/example-sidecar.yaml documents

warden.sidecar renders only the initContainer entry; shareProcessNamespace and the host-cgroup volume are set by your own chart rather than injected, so your chart stays in full control of its own Pod spec end to end.

Configuring a policy

Neither chart above makes Warden do anything on its own — it acts only once a WardenPolicy exists for the target workload:

apiVersion: warden.mnemo.io/v1alpha1
kind: WardenPolicy
metadata:
  name: my-app-policy
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: my-app
  timezone: Europe/Paris
  profiles:
    off-peak: { request: 512Mi, limit: 768Mi }
    peak: { request: 2Gi, limit: 3Gi }
  schedule:
    - { cron: "0 22 * * *", profile: off-peak }
    - { cron: "0 7 * * *", profile: peak }

Apply it with kubectl apply -f, same as any other custom resource. For every field (leadTime, blackout, guardrail), the sidecar/controller environment variables, and the full set of Helm values, see docs/configuration.md — it also has an operator runbook for the failure modes you'll actually hit (read-only mode on an unsupported collector, a shrink aborted by the RSS verification gate, cgroup mount issues, and more).


Status

🧪 Beta. The controller, agent, CRD, and both Helm charts are built, released, and installable (published container images on GHCR, charts on the Helm repo above, libraries on Maven Central), and the full schedule → shrink/grow → verify flow works end to end against a real cluster. It has not yet been battle-tested under production load, and APIs may still change — issues and feedback welcome.

Requirements

  • Kubernetes 1.35+ (in-place Pod resize is GA; on earlier versions it needs the InPlacePodVerticalScaling feature gate)
  • A JVM with runtime-tunable heap commit: ZGC, Shenandoah, or G1 with periodic GC enabled
  • Cluster Autoscaler (or equivalent node consolidation) to realize cost savings from reduced requests

Security

Warden runs with real privileges inside your cluster (RBAC, JMX, a host cgroup mount). See SECURITY.md for how to report a vulnerability, docs/threat-model.md for the full trust-boundary analysis, and docs/hardening.md for an operator checklist before going to production.

Press

Automatically Shrink and Grow JVM Memory in Kubernetes

License

Apache License 2.0

About

💰 Cut K8s idle infrastructure costs. Safely shrink memory request during low-traffic windows with no cold-start penalties. ☸️

Topics

Resources

Security policy

Stars

8 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages