Skip to content

Complete GUI localization coverage and add headless contracts - #1527

Open
francescofugazzi wants to merge 52 commits into
MrNeRF:masterfrom
francescofugazzi:feature/localization
Open

Complete GUI localization coverage and add headless contracts#1527
francescofugazzi wants to merge 52 commits into
MrNeRF:masterfrom
francescofugazzi:feature/localization

Conversation

@francescofugazzi

@francescofugazzi francescofugazzi commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Summary

This pull request completes the GUI localization pass and strengthens localization handling across native C++, RML, Python panels, asynchronous operations, MCP/UI bridges, and shipped locale resources.

It also introduces headless localization contracts and hardcoded-UI audits to prevent localization regressions from being reintroduced.

The English locale currently contains 2042 localization keys, with matching key sets and placeholder contracts across all shipped locales.

Localization coverage

The changes move remaining user-facing GUI text into the localization resources, including:

  • viewport and rendering controls;
  • viewport and depth controls;
  • LOD controls;
  • video extractor controls and dropdown options;
  • sequencer labels and status text;
  • asset manager messages;
  • export and import panels;
  • plugin marketplace status messages;
  • operator labels and cancellation messages;
  • asynchronous task progress and completion states;
  • MCP task and export status;
  • COLMAP export dialogs;
  • startup and overlay UI;
  • tooltips and sidebar labels;
  • RML attributes and option labels.

All shipped locales are kept structurally aligned with en.json.

Runtime language switching

Runtime language switching now updates:

  • cached native panels;
  • cached Python panels;
  • live RML documents;
  • RML text translation directives;
  • RML title and placeholder attributes;
  • translated <option> elements;
  • cached selected values in RML select controls;
  • startup language selection;
  • menus, toolbars, HUD labels, and overlays.

Dynamic application-owned content is preserved during localization refreshes. Values such as paths, progress values, frame counts, FPS values, and other runtime-generated content are not overwritten by stale localized text.

The refresh path also handles the initial language generation correctly, including startup when the generation value is zero.

Python panel localization

Python panels now use explicit localization mappings for their labels.

The Rendering Panel includes dedicated keys for:

  • environment map;
  • environment exposure;
  • environment rotation;
  • rendering and viewport controls;
  • LOD controls;
  • mesh and post-processing controls.

The localization contracts inspect literal Python LOCALE_KEY dictionaries so indirect mapping references cannot bypass validation.

Python formatting is protected with a safe formatter that supports both positional and named placeholders and gracefully handles malformed translator-controlled format strings.

Stable task outcomes

Task state now separates stable machine-readable outcomes from localized display stages.

Export and dataset operations expose stable outcomes such as:

  • idle;
  • running;
  • completed;
  • failed;
  • cancelled.

Localized stage text remains presentation-only and is not used for program logic.

The export outcome is propagated consistently through:

  • native asynchronous task state;
  • Python store bindings;
  • Python overlay state;
  • Python-facing UI bridges.

COLMAP export handling

COLMAP overwrite dialogs now detect the actual source sparse format.

Binary sources report:

  • cameras.bin;
  • images.bin;
  • points3D.bin.

Text sources report:

  • cameras.txt;
  • images.txt;
  • points3D.txt.

The message uses a shared positional localization placeholder and is translated in every shipped locale.

Localization validation tools

The localization validation suite now checks:

  • English and shipped locale key parity;
  • placeholder parity;
  • plural-form compatibility;
  • malformed format strings;
  • one JSON key per physical line;
  • JSON key indentation;
  • direct C++ localization lookups;
  • direct Python translation calls;
  • Python LOCALE_KEY dictionaries;
  • RML translation directives;
  • RML translated attributes;
  • runtime RML refresh wiring;
  • dynamic-content preservation logic;
  • translated select option handling;
  • cached native panel refresh behavior;
  • cached Python panel refresh behavior;
  • operator label localization;
  • stable task outcome usage;
  • hardcoded UI candidates;
  • declared UI fallback exceptions;
  • watch-directory message formatting;
  • language generation propagation.

The hardcoded UI audit supports CI enforcement through --fail-on-candidates.

CI and headless contracts

The repository includes a headless localization contract target that can be run without Torch-dependent tests.

The contracts are also suitable for CI execution and fail when localization regressions are detected.

Example commands:

python tests/python/test_localization_contracts.py
python tools/check_locale_completeness.py
python tools/check_ui_hardcoded.py
ctest --test-dir build/tests --output-on-failure -R LocalizationContracts

Documentation

Localization conventions, validation tools, hardcoded UI audit rules, allowlist policy, formatting rules, runtime refresh behavior, and language-specific grammar handling are documented in:

  • CONTRIBUTING.md;
  • docs/building_and_distribution.md;
  • the localization testing documentation.

Validation performed

The following checks pass locally:

  • localization contract tests;
  • locale completeness validation;
  • hardcoded UI audit;
  • indirect Python localization mapping validation;
  • placeholder validation;
  • plural-form validation;
  • one-key-per-line validation;
  • Python syntax checks;
  • clang-format validation;
  • git diff --check.

A full application build and runtime verification should be performed before merge, with particular attention to:

  • runtime language switching while multiple panels are open;
  • Video Extractor dropdown refreshes;
  • Sequencer and Asset Manager refreshes;
  • COLMAP binary and text overwrite dialogs;
  • dynamic RML content preservation;
  • startup localization warnings.

Replace hardcoded user-facing labels across the viewport overlays, rendering and sequencer panels, window controls, camera toolbar, editor context menus, console, marketplace, and downloader with localization lookups. Add the corresponding English keys and language-specific values for all shipped locales while preserving technical identifiers and format names where appropriate. Extend the locale completeness checker to reject multiple JSON keys on one physical line, then normalize all reported locale files. Validation: locale completeness check, JSON parsing, and git diff check passed.
Move dynamic configuration, export, conversion, checkpoint, RAD LOD, scene history, CLI logging, filtering, clipboard, and log export messages into localization keys. Preserve formatting placeholders across all shipped locales and add language-specific translations for the Scene and runtime feedback strings. Apply clang-format to the modified C++ files and validate locale completeness, JSON parsing, placeholder consistency, and repository whitespace.
Move fixed import, export, video export, mesh-to-splat, and splat simplification progress states out of async task manager literals and into the runtime localization catalog.

Add typed runtime keys and provide language-specific translations for initialization, start, completion, failure, cancellation, and apply states across all supported locale files.
Move asynchronous task progress states, export and video status messages, COLMAP prerequisites, rendering failures, import failures, and training fallback errors into the runtime localization catalog.

Add translated entries for every supported locale and keep the typed string key registry aligned with the runtime catalog.
Move remaining asynchronous task, CUDA compatibility, modal, and Python console feedback through the localization catalog with translations for every shipped locale.

Add a recursive UI hardcoded-text audit with exact and regex allowlist support, plus a reviewed baseline for backend diagnostics and canonical asset names.
Move Python Console, scene graph, sequencer, rendering, image preview, video export, and HUD strings to locale keys across every supported language.

Improve the UI hardcoding audit to preserve UTF-8 symbols, ignore encoded graphical literals, and retain reviewed internal diagnostics in the allowlist. The audit now reports no likely hardcoded UI strings.
Keep the sequencer distance delta in meters with the standard m symbol instead of translating the unit name.
Localize remaining viewport, depth, export, marketplace, asset manager, sequencer, rendering and event messages across every shipped locale.

Refresh RML resources after a language change at a safe UI boundary and make cached toolbar, selection, transform, GT comparison, import and video overlay state depend on language_generation so labels and tooltips no longer require an interaction to update.

Replace unsafe localized std::format call sites with runtime formatLocalized support, fix Python marketplace interpolation and downloader startup handling, and add missing rendering section labels including translated LOD headings.

Extend the hardcoded UI scanner to cover Python UI bindings while excluding reviewed dynamic formatting patterns; retain the remaining import watcher strings as explicit candidates for the next localization pass.
Document the canonical locale completeness check, including its one-key-per-line JSON formatting requirement.

Add contributor guidance for the hardcoded UI scanner, reviewed allowlist exceptions, fail-on-candidates enforcement, and language-generation cache invalidation.
Add a CMake/CTest entry point for deterministic localization validation without building test binaries.\n\nCover locale key and placeholder parity, JSON formatting, literal localization references, RML directives, hardcoded UI audit output, localized scan-message formatting, and language-generation cache invalidation.\n\nLocalize remaining import-panel scan statuses and Asset Manager URL errors across shipped locales, reuse existing console status keys, tighten the hardcoded UI scanner, and document the target in the build guide.

@MrNeRF MrNeRF left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this huge effort — this is a big and genuinely useful pass. The locale data itself is in really good shape: all 10 files have exact key parity with en.json, zero placeholder mismatches, correct terminology in ja/ko/zh, and the deferred-refresh idea is the right approach. Dutch and Italian are particularly strong.

A few things need fixing before merge (details in the inline comments):

Must fix

  • Task stage strings are now translated, but src/app/mcp_runtime_tools.cpp and export_panel.py still compare against English "Complete" / "Failed" / "Cancelled" — in a non-English session finished jobs report as idle and exports never register in the Asset Manager. Stage should stay a stable token, translated only for display.
  • LocalizationManager has no locking but is now called from worker threads (exports, video encode, folder scans) — a language switch during an export can crash.
  • LOCF is a bare std::vformat: one broken placeholder in a locale file (or via lf.ui.loc_set) throws, and two call sites sit inside catch blocks on worker threads, which terminates the app.
  • The new contract test spawns python and fails on python3-only machines (sys.executable).
  • selection_controls.py tracks language_generation but never dirties its labels — the stale-language bug stays in that panel.
  • The startup overlay language picker doesn't trigger the new deferred refresh — first-run users keep an English UI until restart.
  • Neither new check runs in CI yet — wiring check_ui_hardcoded and LocalizationContracts into the workflows would let them actually guard.

Also worth a pass

  • de/fr/es still have 25 runtime.* values in plain English (it has them all translated — nice reference). A few German strings lost umlauts: "Schluesselbild", "Alle loeschen""Keyframe", "Alle löschen".
  • Small pluralization losses ("Showing 1 assets", "plugins"), and en.json disk_space_dialog.checkpoint_save_failed = "Checkpoint Save Failed (Iteration" is a fragment — "… (Iteration {})" would make it translatable.
  • The scanner's clean result is mostly it being too forgiving (single words always skipped) — see inline notes; still valuable as a review queue.
  • Not from this PR: the Ubuntu CI failures are SDL3/vcpkg infrastructure — a re-run should clear them.

With the stage-token, threading and format-fallback issues sorted, this will be a great step for the app. Thanks again!

{
const std::lock_guard lock(export_state_.mutex);
export_state_.stage = "Complete";
export_state_.stage = LOC(lichtfeld::Strings::Runtime::TASK_COMPLETE);

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These stage values are also read as plain English elsewhere: mcp_runtime_tools.cpp checks stage == "Complete" / "Failed" / "Cancelled", and export_panel.py:758 checks "Complete" before registering the export. Once translated (de: "Abgeschlossen"), non-English sessions never see a job as finished. Suggest keeping stage as a fixed token and translating only at display time.

}
} catch (const std::exception& e) {
error_msg = std::string("COLMAP export crashed with exception: ") + e.what();
error_msg = LOCF(lichtfeld::Strings::Runtime::TASK_FAILED_DETAIL, e.what());

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two risks here: this runs in a catch on a worker thread, and LOCF is a bare std::vformat that throws on a broken placeholder — an uncaught throw here ends the process. Also LocalizationManager has no locking, so a language switch during an export races with these lookups.

#define LOC(key) lfs::event::LocalizationManager::getInstance().get(key)
template <typename... Args>
[[nodiscard]] inline std::string formatLocalized(const std::string_view key, Args&&... args) {
return std::vformat(LocalizationManager::getInstance().get(key), std::make_format_args(args...));

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

std::vformat throws std::format_error if a translation has a bad placeholder (and plugins can set arbitrary strings via lf.ui.loc_set). A try/catch falling back to the English text would make bad locale data harmless instead of fatal. Same applies to the fmt::runtime(LOC(...)) site in rml_sequencer_overlay.cpp.



def test_hardcoded_ui_audit_has_no_candidates():
result = subprocess.run(["python", str(ROOT / "tools" / "check_ui_hardcoded.py")],

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"python" doesn't exist on most Linux boxes (only python3), so this test fails with FileNotFoundError. sys.executable fixes it.



def _fields(text):
return {name.split(".", 1)[0].split("[", 1)[0] for _, name, _, _ in string.Formatter().parse(text) if name}

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Filtering on if name drops auto-numbered {} fields, so "Failed: {}" vs "Fehler: {} und {}" pass this check. check_locale_completeness.py keeps them — same approach would work here.

"environment_mode": "main_panel.environment",
"environment_map_path": "main_panel.preset",
"environment_exposure": "main_panel.ppisp_exposure",
"environment_rotation_degrees": "transform.rotation",

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Chinese this key is 旋转: (fullwidth colon) and _entry_label only strips ASCII :, so the row renders as 旋转::.

const std::string export_button = "Overwrite";
lfs::core::ModalRequest request;
request.title = "Export COLMAP sparse";
request.title = LOC(lichtfeld::Strings::Window::EXPORT);

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The title became generic ("Export" instead of which export), while "Overwrite", "Cancel" and "This writes" in the same modal stay English — worth finishing this dialog in one go.

Comment thread src/python/lfs_plugins/import_panels.py Outdated
with self._scan_state_lock:
while len(self._scan_log) <= index:
self._scan_log.append({"status": "Queued", "path": ""})
self._scan_log.append({"status": _tr("watch_dirs.scan_log_queued"), "path": ""})

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_tr() is called per padded row while holding _scan_state_lock — hoisting it above the loop avoids repeated FFI calls under the lock.

model.bind_func("depth_view_disable_label", lambda: "Disable Depth Map")
model.bind_func(
"depth_view_disable_label",
lambda: _ui_label("toolbar.depth_mode_disable", "Disable Depth Map"),

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This borrows toolbar.depth_mode_disable ("Disable Depth Mode") for the Depth Map tool — a small mismatch, and it couples two unrelated buttons to one key.

status = f"Downloading... {int(percent * 100)}% ({_format_bytes(downloaded)} / {_format_bytes(total_size)}) {speed_str}"
status = f"{lf.ui.tr('asset_manager.import_button_downloading')} {int(percent * 100)}% ({_format_bytes(downloaded)} / {_format_bytes(total_size)}) {speed_str}"
if eta_str:
status += f" ETA: {eta_str}"

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ETA: ... suffix (and the extraction messages below: "Extracting... {i}/{total} files", "Extraction complete", "(size unknown)") are still English right next to the newly translated part.

Use the active Python interpreter for nested contract commands and preserve automatic std::format placeholders during locale comparison.

Make Selection Controls rebuild localized bindings when language_generation changes, allow localization contracts to coexist with regular and Unicode test configurations, and run the scanner plus headless contracts in the locale workflow.

Expand the hardcoded UI audit to cover single-word UI text, f-strings, status/progress sinks, class labels, and RML title/placeholder attributes. Add focused audit fixtures and fix relative --root handling so newly exposed UI text cannot be hidden by permissive heuristics.
Separate machine-readable terminal outcomes from localized progress labels for scene export, dataset import, video export, and mesh-to-splat conversion. MCP task payloads now expose stable outcome tokens instead of inferring status from translated stage text, and the export panel uses the stable completion outcome.

Synchronize localization reads, language switches, and overrides. Return thread-local translation copies so locale map replacement cannot invalidate strings in use, and make formatted localization fall back safely for malformed format strings.

Localize the remaining VRAM HUD labels, GT normal mode, and URL download/extraction progress in every shipped locale. Add contracts preventing localized stage comparisons from returning to MCP task status handling.
…copy

Teach the hardcoded UI scanner to recognize fallback labels supplied to _ui_label, _tr, and _trf alongside a localization key. These fallbacks preserve compatibility when a translation is unavailable and are not independent user-facing bypasses.

Extend the scanner contract fixture so it verifies that a real hardcoded label is still reported while the equivalent declared localized fallback is excluded.
Move ui.normal into the color and depth option group in every locale, restore canonical four-space indentation, and localize viewport export status messages with formatted keys across all shipped languages.

Extend locale validation to reject odd JSON key indentation and validate RML translation directives against every shipped locale, including non-empty translated values.
Route plugin installer progress updates through a safe localized formatter and add named-placeholder messages for downloads, environment creation, dependency synchronization, clone operations, and updates in every shipped locale.

Teach the hardcoded UI audit to ignore only static panel class labels when the same file demonstrably refreshes that panel through a localized binding or set_panel_label call. This removes already-localized metadata without suppressing real UI strings.
Add optional localization keys to tool, submode, and pivot definitions. Their labels are resolved only when serializing metadata for the UI, so builtin tools retain stable identifiers and update with the active language instead of translating during module import.

Localize all builtin tool names and the Python Scripts panel title in every shipped locale. Strengthen the UI hardcode audit with AST-based docstring exclusion and recognition of label/key declaration pairs, retaining reports for labels without an associated localization key.
Record the remaining scanner exclusions as narrow, commented exceptions for Python outline formatting, language-server implementation identifiers, canonical shortcut and URL syntax, renderer backend names, and a CSS class token.

The hardcoded UI audit now completes with zero candidates while retaining coverage for user-facing literals.
…strings

Replace the sequencer overlay's direct fmt::runtime(LOC(...)) calls with the guarded LOCF helper so malformed localized format strings fall back safely instead of propagating formatting exceptions.

Add a localization contract that rejects direct fmt::runtime(LOC(...)) usage anywhere in C++ sources.
Re-read language_generation after forwarding startup overlay input, because selecting a language mutates localization state during that input phase. The splash now updates localized text and refreshes its cache in the same frame.

Add a headless contract requiring the post-input language refresh path.
Track language_generation in the VRAM HUD and refresh cached localized iteration and empty-state text when the active locale changes. Menu and tooltip paths already refresh through generation or dynamic resolution.
Add a shared plural-form helper for localized count messages, including Polish one/few/other forms. Update marketplace registry and asset result summaries to select the correct localized phrase, and repair the remaining Python tr() count formatting call. Extend the localization contracts to validate the supported plural categories and catalog keys.
Map every PluginState displayed by marketplace cards to an explicit localized label, covering installing, loading, active, unloaded, error, and disabled states across all shipped locales.
Add a dedicated localized Python console error for a second run request instead of reusing the running-state label. Register the key in the native key catalog and every shipped locale.
Replace hardcoded undo, redo, total, and GPU history fragments with complete locale-specific summary messages in every shipped language.
Localize grouped and untitled history fallbacks and pass translated undo or redo labels into history stack rows instead of hardcoded English tokens.
Use the shared interaction-safety guard for pending localization refreshes so modal dialogs and open menus retain the request until a safe frame. Add a localization contract for the full guard.
Translate Python operator labels that are localization keys before registering their native property metadata, preventing raw Sequencer and action keys from appearing in operator consumers.
Replace duplicated hardcoded COLMAP export confirmation labels in the Python export panel and native scene graph with shared localized keys, including a dedicated overwrite action in every shipped locale.
Treat fullwidth colons as existing label punctuation in the rendering panel and move Watch Directories localization lookups outside scan-state locks. Add a contract for the fullwidth-colon behavior.
Align the German locale with maintainer review feedback by using the established Keyframe terminology instead of the ASCII transliteration Schluesselbild, and restore umlauts in sequencer actions and tooltips.

Correct the remaining Alle loeschen label to Alle löschen while preserving one-key-per-line locale formatting.
Restore the missing string-concatenation operator before the localized COLMAP sparse-output warning. This fixes the scene graph element compilation failure while preserving escaped localized text in the modal body.
Keep localization keys intact while Python operators are registered, instead of resolving them before the UI is rendered. This avoids startup lookups for sequencer labels and preserves dynamic localization at the rendering boundary.

Add a localization contract that prevents property registration from resolving label keys prematurely.
Make RmlPythonPanelAdapter request an animation frame whenever the active locale differs from the document locale. Cached dirty-policy panels now reach prepareForRender without requiring pointer movement or another input event.

The frame reloads the panel document through the existing language-aware lifecycle, refreshing RML translation directives, bound localized labels, placeholders, and tooltips. This fixes the Asset Manager refresh path and applies the same guarantee to every Python-backed RML panel.

Add a headless localization contract that verifies cached Python panels cannot remain idle across a language-generation change.
Track the language used to load each RmlPanelHost document and reload the document whenever the active language changes.

Apply the synchronization before normal panel drawing, direct rendering, layout preparation, and cached direct rendering. This prevents cached native panels, including Video Export, from continuing to display the previous locale until pointer input or another invalidation occurs.

Request an animation frame while a native RML panel has an outdated language so the refresh is scheduled immediately even when no pointer event is received.

Add a localization contract that protects both the cached and uncached native RML rendering paths.
…wing

Revert the per-panel language synchronization introduced in 8660115.

Reloading an RML document from draw, direct-draw, layout, and cached-composite paths can invalidate RmlUI elements while the current frame still holds references to them. The crash dump recorded after switching language with several panels open is an access violation and makes that approach unsafe.

Keep the existing centralized localization refresh path, which performs panel resource reloads at the coordinated GUI-manager boundary, and retain the separate cached Python-panel frame invalidation from 7b56e46.
Keep the full interaction-safety guard for runtime localization reloads, but release focus retained by completed RML button and select interactions before evaluating the pending refresh.

Editable text fields remain focused and continue to defer the reload. This avoids invalidating a document during active text editing, menus, or modal interactions while preventing inactive focus from blocking static @tr: content indefinitely.

Request the centralized refresh after a language change from the startup picker as well as from the Python language API. This lets Video Extractor and Sequencer reload their static RML labels instead of updating only their dynamic labels.

Add contracts for focus cleanup, the full safety guard, and the startup language-switch refresh path.
StartupOverlay is initialized with RmlUIManager, which deliberately does not own the GuiManager localization-refresh API. Remove the invalid call that prevented startup_overlay.cpp from compiling.

Keep the focus-cleanup localization fix and adjust its contract to verify the implemented non-text-focus behavior rather than an unavailable startup callback.
Remove the non-text focus cleanup added by 1e8f9d8.

The reported access violation persists when a language switch clears RML focus and then reloads resources in the same GUI cycle with multiple panels open. Restore the established full interaction guard without mutating focus state.

This deliberately leaves the Video Extractor and Sequencer static-label issue unresolved until it can be fixed through a refresh path that is separated from live RmlUI document and frame lifetimes.
…oads

Preserve the localization keys behind parse-time @tr: directives as data attributes while loading RML documents. Cover translated element text as well as title and placeholder attributes.

Refresh those translations directly on every live RmlUI document when the application language generation changes. This keeps document instances, event listeners, focused controls, select state, and cached component references alive instead of unloading resources during an active GUI lifetime.

Replace the runtime localization reload path with the in-place document refresh while retaining full resource reloads for development RML and locale hot reloads.

Detect language-generation changes centrally so switches initiated from either the startup picker or Python API reach static RML content in Video Extractor, Sequencer, and the other RML surfaces.

Extend localization contracts to require the in-place path, reject destructive resource reloads from runtime language changes, and verify that every shipped @tr: directive has a supported text or attribute shape.
Include rml_document_utils.hpp in rmlui_manager.cpp so the runtime document refresh can resolve the rml_documents namespace and refreshLocalizedContent declaration under MSVC.
…e changes

Make Video Extractor request an animation frame whenever the active language differs from the language used by its last synchronized render.

Expose the same localization-frame demand from the Sequencer panel and include it in SequencerUIManager animation scheduling.

This ensures the in-place DOM translations are composited immediately instead of waiting for mouse movement, playback, or another unrelated panel invalidation.

Add contracts covering both native cached-panel scheduling paths.
Reuse the existing UI layout-settle frame budget after an in-place localization refresh.

Keeping three consecutive GUI frames active lets translated DOM updates propagate through measurement, layout, cached rendering, and final composition without falling back to the idle event-loop timeout between stages.

Extend the runtime localization contract to require the settle-frame scheduling.
Synchronize each RmlUI select control after its translated option elements are updated so the closed dropdown displays the active locale immediately.

Apply the behavior in the shared live-document refresh path, covering Video Extractor values such as Frame Interval, PNG (lossless), and Original as well as other RML selects without reloading documents or dispatching selection changes.

Extend the localization contracts to require selected-label synchronization and verify formatting, locale completeness, hardcoded UI scanning, and all localization contracts.
Refresh RmlUI selectvalue content directly after translated options change so closed Video Extractor dropdowns update immediately during runtime language switches.

Complete the remaining review follow-ups by translating the residual German, French, and Spanish runtime messages, preserving checkpoint placeholders, localizing log entry wording, adding the dedicated Depth Map disable label, and keeping operator metadata stable for non-GUI consumers.

Extend the localization contract for the selectvalue refresh and document the intentional Play/Pause operator exception in the hardcoded UI allowlist.
Update every cached Video Extractor select option when the runtime language changes, including the visible selectvalue for the currently selected option. Cover extraction mode, output format, resolution, sharpness controls, and window candidate presets so open dialogs no longer retain English labels after switching locale. Add a headless localization contract for the cached select refresh and representative Video Extractor keys.
Document the count-sensitive locale-key convention, the current Polish one/few/other rule, and the contract-driven process for adding future grammar rules. Clarify that headless localization contracts cover plural-form behavior.
Fall back to the English locale text when a localized format string is invalid, while retaining thread-local result storage for worker-thread callers. Cache navigation toolbar labels by language generation and cache the HUD iteration label instead of resolving it on every update. Remove trailing colons from the compact Iteration label in every shipped locale and add contracts for all three regression paths.
@francescofugazzi

Copy link
Copy Markdown
Contributor Author

Thanks for the detailed review. I went through every inline thread and added follow-up commits covering the remaining functional, safety, validation, runtime-refresh, and documentation points.

Review follow-ups addressed

Stable task state and worker-thread safety

  • Async task stages now remain stable machine-readable outcomes. Localized text is applied only at display boundaries, so MCP/export/import completion checks no longer compare translated values such as Complete, Failed, or Cancelled.
  • LocalizationManager now synchronizes access for worker-thread callers and returns copied thread-local buffers, so a language switch cannot invalidate a string used by export, encoding, or scan workers.
  • Localized formatting now catches invalid locale or runtime-override format strings and retries with the English fallback. If the fallback itself is malformed, the raw fallback text is returned safely instead of propagating std::format_error.

Contracts, CMake, and CI

  • The contract runner uses sys.executable, avoiding the Linux python versus python3 issue.
  • Placeholder validation preserves anonymous {} fields rather than dropping them.
  • Contracts now cover real invalidation wiring for cached Python UI state, startup language selection, live RML refreshes, cached native/Python panels, translated select values, toolbar labels, HUD labels, plural forms, and safe formatting fallback.
  • BUILD_LOCALIZATION_TESTS registers the headless CTest entry without disabling the regular test suite when other test options are enabled.
  • CI now runs locale completeness, the hardcoded UI audit, and the headless localization contracts on pull requests and pushes to master.

Runtime language refreshes

  • Cached Python controls now dirty their localized state when language_generation changes.
  • The startup language picker publishes the language generation and refreshes the shell immediately.
  • Live RML documents are translated in place instead of being reloaded during rendering or interaction.
  • RML title and placeholder directives are preserved as metadata and refreshed at runtime.
  • Native select controls now refresh both their translated options and RmlUI's separate closed selectvalue cache.
  • Video Extractor explicitly refreshes all cached select labels: extraction mode, output format, resolution, sharpness options, and window candidates.
  • Navigation toolbar labels are cached by language generation rather than looked up every frame.
  • The VRAM HUD caches its localized Iteration label by language generation; compact iteration labels no longer retain trailing ASCII or full-width colons.

Remaining localization and terminology

  • Restored Marketplace state visibility for unloaded, installing, loading, active, error, and disabled plugins.
  • Added count-sensitive Asset Manager and Marketplace messages.
  • Added Polish plural selection through src/python/lfs_plugins/localization.py:
    • one for 1;
    • few for values ending in 2 through 4, excluding 12 through 14;
    • other otherwise.
  • Other shipped locales retain .one, .few, and .other entries for key parity while currently using the applicable subset of those forms.
  • Completed the runtime translations requested for German, French, and Spanish, including the German terminology corrections.
  • Corrected locale-specific punctuation handling, including Chinese full-width colons.
  • Completed the COLMAP export dialog, console labels, log-entry wording, downloader/extraction feedback, and dedicated Depth Map disable wording.

Hardcoded UI audit and documentation

  • Tightened the hardcoded UI scanner to inspect additional native, RML, Python, status/progress, f-string, class-label, and title-attribute sinks.
  • Added regression coverage for common scanner bypasses and fixed relative-root handling.
  • Kept reviewed technical/non-UI exceptions in the documented allowlist rather than hiding ordinary UI text.
  • Updated CONTRIBUTING.md with locale conventions, one-key-per-line formatting, allowlist policy, cached-language invalidation, count-sensitive message conventions, the Polish rule, and the process for future language-specific grammar.
  • Updated the build documentation with the headless contract scope and CMake/CTest entry point.

Validation

The following checks pass:

python tests/python/test_localization_contracts.py
python tools/check_locale_completeness.py
python tools/check_ui_hardcoded.py --fail-on-candidates
clang-format --dry-run --Werror <all modified C/C++ files>
git diff --check

The locale checker validates 2028 English keys across all shipped locales, and the hardcoded UI audit reports no likely user-facing hardcoded strings.

Manual validation

Runtime language switching was manually checked with open cached panels, including Video Extractor dropdowns, toolbar tooltips, Asset Manager, Sequencer, and Python panels. No stale labels, delayed refreshes, or crashes were observed.

@MrNeRF

MrNeRF commented Aug 3, 2026

Copy link
Copy Markdown
Owner

Thanks for the thorough follow-up — most of the round-1 list is properly closed. The LocalizationManager locking, the std::format fallback, sys.executable, the selection_controls dirty map, the marketplace per-state keys, the de/fr/es runtime.* translations (umlauts restored), the tests/CMakeLists.txt return(), the full-width colon, and the HUD/toolbar caching all check out. The Polish rule matches CLDR and the pl strings use the right genitive forms. Locale data re-verified clean: 2028 keys across all 10 files, no key diffs, no placeholder mismatches, no empty values, and all 923 string_keys.hpp constants resolve.

A few things still block merge, mostly fallout from the stage/outcome split and the new RML refresh.

Must fix

1. Completed exports no longer register with the Asset Manager — now in every language, including English.
export_panel.py:758 reads state.get("outcome"), but the field was only added to the C++ ExportProgressState. Neither dict that reaches Python publishes it: py_store.cpp:200 (export_progress_state_to_dict) and py_ui.cpp:4453 (get_export_state) are unchanged and emit active/progress/stage/format/error/path only. So .get("outcome") is always None. The old bug only hit non-English sessions; this one hits everyone.

2. MCP/CLI dataset imports never report finished.
async_task_manager.cpp:735-787 — the DatasetLoadStarted/DatasetLoadCompleted handlers set stage but never outcome, while mcp_runtime_tools.cpp:477-480 now keys off outcome. That path is live via mcp_gui_tools.cpp:1993 and visualizer_impl.cpp:2194SceneManager::loadDataset. Also, outcome isn't reset on start, so a previous failure leaves "failed" and the next successful load reports failed.

3. common.cut / copy / copy_all / paste / select_all still exist in no locale file.
7 sites in python_editor.cpp:2402-2418 and python_console_panel.cpp:461-475; en.json's common block has none of them, so both context menus render the raw keys. These call LocalizationManager::getInstance().get("…") directly, which test_literal_localization_calls_resolve doesn't scan (it only matches LOC(/LOCF(/lf.ui.tr(/_tr() — worth widening that pattern.

4. @tr: attributes show the raw key until the user changes language.
gui_manager.cpp:4559 only refreshes when language_generation differs from localized_rml_language_generation_, and both start at 0 — nothing bumps the generation at startup (the only publishers are the startup picker's change handler, lf.ui.set_language, and dev locale hot-reload). A default session shows @tr:ui.minimize, @tr:common.close, @tr:ui.independent_split_view, @tr:ui.hide_ui, @tr:ui.maximize_window as menubar tooltips, plus @tr:ui.search_annotations and @tr:common.filter as the VRAM-HUD placeholders (viewport_overlay.rml:1047,1067) — those two are visible text that used to read fine in English. Priming the generation once during startup would cover it.

Also worth a pass

  • The in-place refresh overwrites dynamic content and it doesn't heal. rml_document_utils.cpp:290 unconditionally re-applies the key to every data-lfs-i18n element. Elements that start as @tr: but get written programmatically snap back to their placeholder — and since the writers cache the last value in a data-lfs-* attribute (video_extractor_dialog.cpp:60-77), the next sync short-circuits and the wrong text sticks. Video extractor #video-value, #output-value, #progress-text, #interval-value, #sharpness-mode-desc are all in this shape. Skipping elements whose current content differs from the previous translation would fix it.
  • RML <option> labels don't live-refresh. RmlUi parents options under the non-DOM selectbox, and refreshLocalizedContent recurses with DOM-only GetNumChildren() — which is why the video extractor needed setCachedSelectLabels. dataset_import_panel.rml:90-92, export_panel.rml:70-71, plugin_marketplace.rml:18-27 and the new rendering.rml options stay in the old language. Recursing with GetNumChildren(true) would make the generic path cover them.
  • get() lifetime changed silently. It now returns a pointer into an 8-slot thread-local ring; it used to be stable until setLanguage. Nothing violates it today (worst case is two live results at error_event_bridge.cpp:164), but there's no comment or assert marking the trap. Returning std::string would remove it outright.
  • runtime.logs_copied still interpolates English entry/entries (scene_panel_native.cpp:1036) — de renders "5 entries in die Zwischenablage kopiert.". The same file already does it right at :312 with runtime.log_entry_word / log_entries_word.
  • Two operator labels are raw keys with no resolver (sequencer_ops.py:22,33): mcp_operator_tools.cpp:299 returns descriptor.label verbatim, and :231 composes "sequencer.add_keyframe_here could not be performed". This contradicts the comment kept on PlayPauseOperator.
  • Dead code from the COLMAP dialog rework: colmapSparseOutputFormat / colmapSparseOutputFileNames / colmapSparseOutputFileList (scene_graph_element.cpp:133-177, anonymous namespace → -Wunused-function) and export_panel.py:399. Both dialogs also lost the format-dependent file list — the new static "Writes cameras.bin, images.bin, and points3D.bin" is wrong for a text-format source.

Minor

  • The CI "Audit hardcoded UI text" step runs without --fail-on-candidates, so it can never fail; the real gate is test_localization_contracts.py:89.
  • string_keys.hpp ALL = "all" points at no key.
  • preserveTranslationDirectives runs two std::regex passes over every RML document at load — that's on the startup path.
  • Key reuse in rendering_panel.py:194-197: environment_map_pathmain_panel.preset, environment_exposuremain_panel.ppisp_exposure, environment_rotation_degreestransform.rotation.
  • status.iteration lost its colon for the HUD; training_panel.py:1132 shares the key and now reads "Iteration 5,000 (…)".
  • localized_count, _tr_format and installer._localized_progress call str.format on translator output unguarded — a plugin loc_set override with a stray brace raises, where the C++ side now degrades gracefully.
  • common.filter drops the old "…"; download_progress_known formatted with speed="" leaves a trailing space.
  • Unrelated churn: rml_panel_host.cpp drops 4 blank lines; image_preview.rml:125 and rendering.rml:518-519 picked up odd indentation.
  • gui_manager.cpp:155,273 localizes two LOD strings and leaves their siblings ("Waiting for frame", "Tree loaded, off", "leaf-only reference") English.

Items 1–4 are the merge blockers; the rest can follow. Thanks again for keeping at this — the locale data itself is in great shape.

Complete the four merge-critical localization fixes identified by the upstream maintainer. Publish stable export outcomes through both Python state bridges, reset dataset import outcomes on each load, and record completed or failed dataset results for MCP and CLI consumers.

Add the missing common editing actions to every shipped locale and extend literal localization contracts to cover direct LocalizationManager lookups. Prime the initial RML refresh generation, traverse non-DOM RmlUi children, and protect dynamically written content from being overwritten during language changes.

Harden Python translation formatting against malformed plugin overrides, localize MCP operator labels and copied-log counts, restore missing LOD status strings and HUD punctuation, enforce the hardcoded UI audit in CI, remove obsolete COLMAP helpers, and clean up the unused string key.

Validation: localization contracts, locale completeness, hardcoded UI audit, Python compilation checks, clang-format dry-run with Werror, and git diff check.
Complete the export state contract used by Python panels by adding the stable outcome field to OverlayExportState and populating it from AsyncTaskManager. This keeps the Python UI path consistent with the AppStore state bridge and allows completed exports to be registered by the Asset Manager.

Validation: localization contracts, clang-format dry-run with Werror, and git diff check. Build was not run.
Resolve the remaining upstream localization review findings across native and Python UI paths.

- Report the actual export outcome through Python overlay state and task bridges.
- Format COLMAP overwrite warnings for the detected binary or text sparse model.
- Harden RML translation refreshes against dynamic content and translated select options.
- Consolidate translation directive preprocessing into one regex traversal.
- Preserve direct localization lookup lifetime documentation and operator label resolution.
- Add regression contracts for dynamic RML content, directive handling, and COLMAP formats.
- Keep all shipped locales synchronized with the new positional formatting placeholder.
- Remove unrelated whitespace churn and normalize touched RML and C++ formatting.

Validation performed: localization contracts, locale completeness, hardcoded UI audit, Python syntax checks, clang-format, and git diff --check.
Add the missing environment map, exposure, and rotation labels to every shipped locale.

Extend localization contracts to inspect literal LOCALE_KEY dictionaries in Python panels, preventing indirect mapping references from bypassing key validation.

Validation performed: localization contracts, locale completeness, hardcoded UI audit, Python syntax checks, and git diff --check.
@francescofugazzi

Copy link
Copy Markdown
Contributor Author

I went through all the listed points and hopefully addressed the remaining issues in the latest local commits.

The implemented fixes include:

  • Export outcomes are now propagated consistently through the native task state, Python overlay state, and both Python-facing bridges.
  • Dataset loading explicitly resets and updates the stable outcome field instead of relying on localized task stage text.
  • The missing common editing commands were added to every shipped locale:
    • common.cut
    • common.copy
    • common.copy_all
    • common.paste
    • common.select_all
  • Localization contracts now detect direct LocalizationManager::get(...) calls.
  • Initial RML localization refreshes now run correctly even when the initial language generation is zero.
  • RML refreshes preserve application-owned dynamic content instead of overwriting it with stale localized text.
  • Translated RML <option> elements are refreshed recursively, including non-DOM children, and the cached selected label is updated.
  • Attribute and text translation directives are processed through a single regex traversal.
  • The lifetime behavior of LocalizationManager::get() is documented explicitly.
  • Operator labels and operator cancellation/error messages now resolve through localization keys.
  • COLMAP overwrite warnings now report the actual sparse format:
    • cameras.bin, images.bin, and points3D.bin for binary sources;
    • cameras.txt, images.txt, and points3D.txt for text sources.
  • Python localized formatting now supports both positional and named placeholders while failing safely on malformed translator-controlled strings.
  • The CI hardcoded-UI check now runs with --fail-on-candidates.
  • The unused localization key constant was removed.
  • The iteration label was separated from the value-only status key to avoid punctuation conflicts across languages.
  • The remaining LOD and common UI labels were added to all shipped locales.
  • The Rendering Panel environment map, exposure, and rotation labels are now explicitly defined in every locale.
  • The localization contract scanner now inspects literal Python LOCALE_KEY dictionaries, preventing indirect mappings from bypassing key validation.
  • Unrelated whitespace churn was removed from the affected RML and C++ files.

The English locale currently contains 2042 localization keys. All shipped locales are checked against the English key set, placeholder signatures, plural forms, and one-key-per-line formatting.

Validation completed locally:

  • localization contract tests;
  • locale completeness checks;
  • hardcoded UI audit;
  • indirect Python localization mapping checks;
  • one-key-per-line locale formatting check;
  • placeholder and plural-form validation;
  • Python syntax checks;
  • clang-format validation;
  • git diff validation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants