Skip to content

fix(core): increase ephemeral storage overhead to 200M#2549

Open
LopatinDmitr wants to merge 1 commit into
mainfrom
fix/vm/up-ephemeral-request
Open

fix(core): increase ephemeral storage overhead to 200M#2549
LopatinDmitr wants to merge 1 commit into
mainfrom
fix/vm/up-ephemeral-request

Conversation

@LopatinDmitr

@LopatinDmitr LopatinDmitr commented Jun 27, 2026

Copy link
Copy Markdown
Contributor

Description

Increase ephemeralStorageOverheadSize from 50M to 200M in the 3p-kubevirt dependency.

This is a temporary hotfix. The root cause — unbounded growth of virt-launcher stdout/stderr logs — is addressed in a separate task.

Related: deckhouse/3p-kubevirt#134

Why do we need it, and what problem does it solve?

When a node runs low on ephemeral-storage, the kubelet evicts pods ordered by excess usage (actual usage − request). The d8v-compute container accumulates stdout/stderr logs over time; once log size exceeds the 50M request, the pod becomes a prime eviction candidate — which causes the VM running inside it to be killed.

Observed in production:

  • Eviction event: d8v-compute used ~63M against a 50M request → VM killed.
  • Long-running pods show log sizes up to 105M, well above the 50M threshold.

Raising the request to 200M gives significantly more headroom before a virt-launcher pod is selected as an eviction victim, reducing unintended VM downtime caused by log accumulation.

What is the expected result?

  1. Deploy updated virt-launcher with new ephemeral-storage request (200M).
  2. Confirm d8v-compute container has resources.requests.ephemeral-storage: 200M.
  3. Verify virt-launcher pods are no longer evicted under moderate node disk pressure.

Checklist

  • The code is covered by unit tests.
  • e2e tests passed.
  • Documentation updated according to the changes.
  • Changes were tested in the Kubernetes cluster manually.

Changelog entries

section: core
type: fix
summary: "Fixed unexpected VM termination under node disk pressure — VMs are now resilient to moderate ephemeral storage consumption."

Signed-off-by: Dmitry Lopatin <dmitry.lopatin@flant.com>
@LopatinDmitr LopatinDmitr requested a review from Isteb4k as a code owner June 27, 2026 20:04
@LopatinDmitr LopatinDmitr self-assigned this Jun 27, 2026
@LopatinDmitr LopatinDmitr added this to the v1.9.2 milestone Jun 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant