Skip to content

Interactive mode: num_latent_chunk override uses sum() of the cumulative schedule instead of max() — requests below the schedule explode quadratically, then OOM #136

Description

@smcshahid

Summary

In helios/diffusers_version/pipeline_helios_diffusers.py:1162-1165:

if use_interpolate_prompt:
    if num_latent_chunk < max(interpolate_cumulative_list):
        num_latent_chunk = sum(interpolate_cumulative_list)

interpolate_cumulative_list is already the cumulative schedule (e.g. 6 prompts ×
interpolate_time=7[7,14,21,28,35,42]). When the requested chunk count is below the
schedule's end, the intended clamp is clearly max(...) (= 42 chunks, enough for all
intervals). Using sum(...) sets 147 chunks = 4,851 frames (~3.4 min of video) — a
quadratic blow-up in prompt count: Σ k·t = t·n(n+1)/2.

Environment

Helios @ 8f2a2fa, H100 80 GB, torch 2.10.0+cu128, diffusers 0.39.0, Helios-Distilled,
official interactive example CSV (example/prompt_interactive_helios.csv, id 6 = 6 prompts).

Reproduction

Any interactive request below the schedule length, e.g. the example CSV with
--num_frames 693 (21 chunks < 42):

Update num_latent_chunk to: 147

then ~9.5 min of generation and finally, on an 80 GB H100:

torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 13.32 GiB … 78.87 GiB in use
  (at video_processor.postprocess_video, denormalize of the 4,851-frame accumulated tensor)

Aggravator: when driven through infer_helios.py's interactive branch, the per-item
except Exception: continue (line 505) swallows the OOM entirely — the process exits 0 with
zero output files after ~10 minutes of GPU time.

Expected

num_frames=693 renders 21 chunks; if the schedule needs more, clamp to max(cumulative) =
42 chunks (1,386 frames) — the schedule's actual length.

Actual

147 chunks / 4,851 frames attempted → postprocess OOM (invisible under the CLI).

Confirmation of the boundary: requesting exactly ≥1,386 frames (e.g. the official script's
1,452) skips the override and works — we generated the full 6-prompt interactive video
(1,449 frames, 165 s on H100) with no other changes. The shipped example script avoids the
bug only because it happens to request 1,452.

Suggested fix

-                num_latent_chunk = sum(interpolate_cumulative_list)
+                num_latent_chunk = max(interpolate_cumulative_list)

(One line; PR available.) Optionally also log the override value with its frame equivalent so
users see the video length actually being produced.

Impact

Interactive mode is unusable below 1,386 frames with the shipped example (and generally below
33·t·n(n+1)/(2·…) thresholds); combined with the silent exception swallowing, users see
"exit 0, no output" after minutes of GPU spend — indistinguishable from success at the
process level. The feature is labeled experimental, but this failure is a plain arithmetic
slip with a safe one-line fix.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions