bug: VLM captioning runs sequentially (one image at a time), unlike the OCR path

[Goldziher]

Summary

The VLM captioning enrichment processor captions embedded images sequentially, awaiting one VLM call per image in a plain loop. The OCR path, by contrast, processes images with bounded concurrency derived from the thread budget. On image-heavy documents the caption pass therefore dominates wall-clock time, since each extract_region_with_vlm round-trip blocks the next.

Where

crates/xberg/src/plugins/processor/builtin/captioning.rs:81

for image in images.iter_mut() {
    if !image_is_caption_candidate(image, min_area) {
        continue;
    }
    let mime = mime_for_format(image.format.as_ref());
    match extract_region_with_vlm_usage(
        image.data.as_ref(), mime, RegionKind::Caption,
        &caption_config.llm, prompt,
    ).await { /* ... */ }
}

Each iteration .awaits a full VLM request before starting the next, so N images = N serial network round-trips.

Contrast: OCR already does this right

crates/xberg/src/extraction/image_ocr.rs bounds image-OCR tasks with a concurrency limit from core::config::concurrency::resolve_thread_budget(config.concurrency.as_ref()) "to prevent resource exhaustion when documents contain many embedded images." Captioning should follow the same pattern.

Impact

• On a document with, say, 10-20 images, captioning latency scales linearly and dominates total extraction time.
• The &image.data mutation is why the current code uses iter_mut(); a concurrent version needs to caption against borrowed data first, then apply results back by index.

Suggested fix

Process caption candidates with bounded concurrency mirroring the OCR path:

• Collect the candidate indices + borrowed image data.
• Drive the VLM calls through a bounded stream (e.g. futures::stream::iter(..).buffer_unordered(max_tasks) where max_tasks = resolve_thread_budget(config.concurrency.as_ref())).
• Reapply (index -> caption/usage) back onto images after the join.

This keeps the per-provider request pressure bounded (avoids hammering the VLM endpoint) while removing the serial stall.

Origin

Surfaced by a user on r/OpenWebUI asking whether found images are queued and captioned in parallel; on inspection they are not. Filing so the honest answer becomes "yes."

[Goldziher]

Fixed in 10523e12bf — VLM captioning now captions images with bounded concurrency instead of one at a time, matching the OCR path.

Verified on current main: crates/xberg/src/plugins/processor/builtin/captioning.rs takes the image vec out of the result for a concurrent drain loop and bounds it via resolve_thread_budget(config.concurrency), so the parallelism respects the configured thread budget rather than being unbounded. Shipped in v1.0.14.