feat: v1/static/info process-static introspection endpoint - #721
Draft
lilith wants to merge 10 commits into
Draft
Conversation
…ncoder,Decoder}Name, build_killbits consts, ExecutionSecurity extensions
Introduce the three-layer killbits system at the types level.
- `ImageFormat` enum — the canonical list of codec-eligible formats.
- `FormatKillbits` / `FormatPermissions` / `FormatGrid` — per-format
decode/encode gating with intersection semantics.
- `CodecKillbits` + `NamedEncoderName` / `NamedDecoderName` — finer-
grained per-codec gating (e.g. allow zen_jpeg_encoder but not mozjpeg)
layered on top of the format-level grid.
- `build_killbits` — compile-time `COMPILE_DENY_DECODE` /
`COMPILE_DENY_ENCODE` const arrays populated from feature-gate
enablement, used later by the core to compute the compile ceiling.
- `ExecutionSecurity::{formats, codecs}` — new optional boxed fields
carrying the per-format / per-codec killbits. Boxed to keep the
struct small (~120 bytes of inline list capacity stays on the heap).
- `SetPolicyRequest` wire type for `v1/context/set_policy`.
- Validation helpers (`KillbitsValidationError`, `CodecKillbitsJobLevelError`,
etc.) surface mutual-exclusion errors and job-level "deny-only"
violations as structured error values.
Pure types crate; no core wiring yet.
Previously `Build001`/`Execute001` inline `security` blocks mutated `Context.security` in place, leaking per-job intent across subsequent jobs on the same Context. Fix the root cause: * Rename `Context.security` to `default_job_security` (Context-scoped defaults). Never mutated by job-level `security` JSON — job-level intent lives in `active_job_security` for the lifetime of the job. * Add `Context.active_job_security: Option<Box<ExecutionSecurity>>` — per-job effective value, set for the duration of a single `build`/`execute` call and cleared when the job finishes. Boxed to keep the idle-Context footprint small. * Add pure `Context::effective_security(&self, inline)` returning `default_job_security ∩ inline` — no mutation. * Add `JobSecuritySnapshot::install` / `restore` — scope guard that swaps `active_job_security` for the job body and restores afterward. * `build_inner` / `execute_inner` validate the inline block, compute the effective value, install it via `JobSecuritySnapshot`, run the job, then restore. * `Context::current_security()` is the single read point for per-node limit checks (`max_decode_size`, `max_frame_size`, `max_encode_size`, `max_input_file_bytes`, `max_total_file_pixels`) — returns the active value when mid-job, the default otherwise. * Codec decoders (gif, png, webp, zen) and encode-path limit checks (`execution_engine`, `codecs_and_pointer`) route through `current_security()` instead of reading `ctx.security` directly. * `configure_security` is removed; the per-job mutation path is gone. `max_json_bytes` is still read pre-parse from `default_job_security` (the parse has to know the cap before it can see the job's own security block). That's the one field that stays Context-scoped. The trusted-policy layer (layer 2) is introduced in a follow-up commit; this commit strictly fixes the job-scoping bug and exposes the hooks the trusted layer will plug into.
…endpoints
Layer 2 of the three-layer killbits system: a Context-scoped trusted
policy set once via a new JSON endpoint, intersected with the
per-job effective security on every build/execute.
Context:
- `Context.trusted_policy: Option<Box<ExecutionSecurity>>` — set at
most once per Context (from `v1/context/set_policy`). Boxed to
keep the idle-Context footprint small.
- `Context::set_trusted_policy(policy, require_unlocked)` — installs
the policy. Narrow-only on re-lock; rejects `allow_*` entries
naming formats denied at build time.
- `Context::effective_security` now intersects `trusted_policy ∩
default_job_security ∩ inline` (layer 1 ∩ layer 2 ∩ layer 3).
- `Context::net_support` / `Context::codec_support` — return the
effective format and codec grids.
New `imageflow_core::killbits` module:
- `feature_compiled_in(format, op)` — runtime feature-flag map for
the c-codecs / zen-codecs gates.
- `parse_format_name` / `from_output_format` — string/enum to
`ImageFormat`.
- `compute_net_support` / `compute_net_support_with_codecs` /
`build_ceiling_grid` — grid math across layers 1..3.
- `enforce(grid, op, format)` — produces structured
`decode_not_available` / `encode_not_available` errors when denied.
- `intersect_security` / `ensure_narrowing` / `validate_trusted_*`
helpers for policy installation and per-job merging.
- `NetSupport` / `CompileCeiling` / `LockedPolicyReport` /
`CodecSupportGrid` / `FormatGridView` wire types.
`NamedDecoders::wire_name` / `image_format` and
`NamedEncoders::wire_name` expose the symmetric mappings the killbits
grid uses to enumerate live backends.
Endpoints (via the existing `imageflow_context_send_json` entry
point; no new C ABI):
- `v1/context/set_policy` — request body
`{ policy: ExecutionSecurity, require_unlocked?: bool }`. Sets the
trusted policy once; later calls are accepted only when narrowing.
Response echoes the resulting `net_support` grid.
- `v1/context/get_net_support` — no request body. Returns the current
grid, whether a trusted policy is set, and a `compile_ceiling`
summary (denied decode, denied encode, and formats with no
compiled-in backend).
The OpenAPI schema regeneration lands in a follow-up commit.
Gate every live decode/encode dispatch on the net_support grid (layer 1 ∩ layer 2 ∩ layer 3) and the codec-level kill lists. Decode dispatch (`EnabledCodecs::create_decoder_for_magic_bytes`): - Compute the effective grid once per request; call `killbits::enforce(grid, Op::Decode, format)` for each matching decoder. If the format is denied, return a structured `decode_not_available` error. - Codec-level kill: when a matching decoder's wire name is denied by trusted/active codec killbits, skip it and try the next matching decoder (mozjpeg killed → fall through to image-rs JPEG). Only error `codec_not_available` when every matching decoder is dead. - Distinguish "no decoder handled magic bytes" from "every matching decoder was killed" so operators can diagnose configuration vs. unsupported-input errors. Encode dispatch (`codecs/auto.rs`): - `auto::create_encoder` and `EncoderPreset` parsing call `killbits::enforce(grid, Op::Encode, format)` with the output format derived from the preset. Denied formats produce `encode_not_available` errors before any encoder is instantiated. - When a format's preferred encoder is killed, fall back to the highest-priority live encoder for that format; return `codec_not_available` only when all encoders for the format are denied. Every denial path carries a structured JSON payload (`error`, `format`, `reasons`) that the ABI layer passes through unchanged.
Broad test surface for the three-layer killbits system: Types-level unit tests (in `imageflow_types::killbits`): killbits `validate`, `validate_job_level`, `intersect`, `apply_to` coverage for both `FormatKillbits` and `CodecKillbits`. Verifies mutual exclusion (allow_* xor deny_* xor formats table), job-level "deny-only" rejection, and that intersection is the union of the two layers' denies. Core-level unit tests (in `imageflow_core::killbits`): `build_ceiling_grid` reflects feature gates, `intersect_security` narrows scalar limits and combines killbits, `compute_net_support` folds the three layers, `codec_not_available_error` surfaces structured JSON. Integration tests (`imageflow_core/tests/integration/killbits.rs`): - End-to-end: `Context::set_trusted_policy` rejects `allow_decode` naming format denied at build time; accepts narrowing re-locks; errors on widening re-locks. - Dispatch: `create_decoder_for_magic_bytes` returns `decode_not_available` when the format is killed, and `codec_not_available` when every matching decoder is killed. - Encode: `create_encoder` returns `encode_not_available` / falls back to a live encoder when the preferred one is killed. - Scoping regression (`inline_max_decode_size_is_not_persisted_across_jobs`, `inline_killbits_is_not_persisted_across_jobs`, `trusted_policy_persists_but_inline_does_not`): confirms that `default_job_security` is untouched by inline job security, and that trusted-policy-installed scalar limits persist while inline denies do not. - Cache: `net_support_cache_hands_out_same_arc_when_policy_unchanged`, `net_support_cache_is_invalidated_by_set_policy`, `net_support_per_job_inline_security_bypasses_cache`, `compile_ceiling_is_process_static` pin the caching contract.
Pick up the new `v1/context/set_policy` and `v1/context/get_net_support` endpoints plus the `ExecutionSecurity.formats` / `ExecutionSecurity.codecs` fields and the `ImageFormat` / `FormatPermissions` / `FormatKillbits` / `CodecKillbits` / `NamedEncoderName` / `NamedDecoderName` schemas in `openapi_schema_v1.json`. Updated hash in `openapi_schema_v1.json.hash` so the `hash_files_relevant_to_schema_and_compare` drift test passes. Generated by `cargo test -p imageflow_core --features schema-export,json-schema` (the test rewrites both files in place).
Adds a new module `imageflow_types::static_info` with the serializable types that back the upcoming process-wide `v1/static/info` endpoint: - `StaticInfoResponse` (top-level) - `BuildInfo`, `CapsSummary`, `FormatAvailability`, `CodecAvailability` - `RiapiSchema` + `RiapiKeyInfo` with `RiapiCategory` + `RiapiValueKind` - `ServerRecommendations` (Accept→RIAPI translation + cache-key hints) - `CodecRole` (Encode / Decode) for the codecs table `CapsSummary::union_in_place` OR-merges flags and widens ranges so the core aggregator can collapse the per-codec `EncodeCapabilities` / `DecodeCapabilities` into a single per-format summary. All types derive `Serialize + Deserialize` (plus `JsonSchema` / `ToSchema` behind the existing feature flags). Types only — the endpoint, aggregation, and caching land in a follow-up commit that wires `imageflow_core` to zencodec's `ImageFormatRegistry` and the per-codec capability descriptors.
Adds the `v1/static/info` JSON endpoint — a process-wide, read-only introspection surface that never depends on `Context` state. Distinct from `v1/context/get_net_support` (which is scoped to trusted policy + per-job narrowing); safe for clients to cache forever since the response only changes when the binary changes. Content is aggregated from three sources: - zencodec's `ImageFormatDefinition` via `ImageFormat::definition()` — display name, MIME types, extensions, format-level capability flags, magic-byte recommendation. Falls back to a hand-maintained metadata table when `zen-codecs` is off or the format lacks a zencodec definition. - `EnabledCodecs::default()` — the set of encoders/decoders this build exposes at runtime. Each is walked once, per-codec caps populate `codecs` and get OR-merged / range-widened into the per-format `encode_union` / `decode_union` summary. - `imageflow_riapi::ir4::get_query_string_keys()` — the same source the existing `v1/schema/riapi/latest/list_keys` endpoint reads. Each key is annotated with category, cache-relevance, value kind, enum values (for `format`, `mode`, `anchor`, `flip`), and — for `accept.*` — the HTTP media type the edge should translate from. Caching layers: - `STATIC_INFO: OnceLock<Arc<StaticInfoResponse>>` — built once per process. Repeat calls are an `Arc::clone`. - `STATIC_INFO_JSON: OnceLock<String>` — cached inner payload JSON. - `STATIC_INFO_WIRE_BYTES: OnceLock<Vec<u8>>` inside `json::endpoints::v1` — the fully-wrapped `JsonAnswer` envelope serialized. The hot path returns `Cow::Borrowed(&'static [u8])`; no serde work after the first call. For zen-backed codecs this invokes each codec config's `capabilities()` associated method (e.g. `<zenjpeg::JpegEncoderConfig as zc::encode::EncoderConfig>::capabilities()`); for C codecs (mozjpeg, libpng, libwebp, pngquant, lodepng, gif-rs) we hand-author a `CapsSummary` matching what each codec actually supports. No zencodec trait gap was exposed — all zen crates already expose their capabilities through the associated trait method. The endpoint is registered in `try_invoke_static` (no `Context` needed), wired into `list_schema_endpoints`, and documented in the OpenAPI schema alongside existing introspection endpoints. Nine in-module tests cover: identity of cached `&'static str` / `Arc<StaticInfoResponse>` pointers across calls; every registered format present in `formats_available`; every enabled encoder/decoder present in `codecs`; the JPEG encode union covers [0, 100] quality; `accept.webp` → `image/webp`; PNG encode+decode always on. Final JSON wire size: 26086 bytes with both `c-codecs` and `zen-codecs` enabled.
Exercises the endpoint through the public `Context::message()` front door and asserts: - Response status 200 with the expected top-level shape (imageflow_version, build, formats_available, codecs, riapi, server_recommendations). - PNG encode+decode availability (always-on baseline codecs). - RIAPI schema exposes `accept.webp` with `accept_header_origin = image/webp`. - `server_recommendations.accept_header_translation` covers webp/avif/jxl. - Two fresh `Context`s produce byte-identical response bodies — the endpoint does not depend on Context state. - `/v1/static/info` is listed by `v1/schema/list-schema-endpoints`. Complements the 9 in-module unit tests that cover the aggregator internals + cache pointer equality.
Adds `/v1/static/info` path entry and the `StaticInfoResponse` + `FormatAvailability` + `CodecAvailability` + `RiapiSchema` + `ServerRecommendations` component schemas. Regenerated via `cargo test --features schema-export,json-schema hash_files_relevant_to_schema_and_compare`.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #720 (
feat/killbits-three-layer). Merge #720 first; thenrebase this branch onto
mainbefore merging.Summary
Adds a new JSON endpoint
POST v1/static/infothat returns aprocess-wide, read-only introspection blob: build info (features +
compile killbits), per-format availability + metadata + capability
union, per-codec capability rows, annotated RIAPI schema, and
opinionated server/CDN deployment recommendations.
The response content is compile-time static (it changes only when the
binary changes), so clients can cache it forever —
imageflow_versionmakes a natural ETag.
Distinct from
v1/context/get_net_support, which isContext-scopedand depends on trusted policy + per-job narrowing.
v1/static/infonever touches
Contextstate; two freshContexts producebyte-identical response bodies.
Content sources
ImageFormatDefinitionviaImageFormat::definition()— display name, preferred MIME +extension, all MIME types + extensions,
supports_alpha,supports_animation,supports_lossless,supports_lossy,magic_bytes_needed. Hand-maintained fallback whenzen-codecsisoff.
capabilities()associated method on
EncoderConfig/DecoderConfig. C codecs(mozjpeg, libpng, libwebp, pngquant, lodepng, gif-rs) get
hand-authored
CapsSummaryrows matching actual behaviour.every codec that backs the format in the current build.
Nonewhenno codec backs the role.
imageflow_riapi::ir4::get_query_string_keys()—the same source as the existing
v1/schema/riapi/latest/list_keysendpoint. Each key is annotated with
category,cache_relevant,accepts(kind),enum_values(forformat,mode,anchor,flip), and — foraccept.*— the HTTP media type the edge shouldtranslate from.
translation at the edge, cache-key prefix inclusion rules, and
Varyheader guidance.Caching
Three layers of
OnceLockso the hot path has zero serde work afterthe first call:
STATIC_INFO: OnceLock<Arc<StaticInfoResponse>>— parsed form,pointer-equal across calls.
STATIC_INFO_JSON: OnceLock<String>— inner payload JSON.STATIC_INFO_WIRE_BYTES: OnceLock<Vec<u8>>— fully-wrappedJsonAnswerenvelope. The endpoint returnsCow::Borrowed(&'static [u8]).Final wire size: 26086 bytes with
c-codecs,zen-codecs.Commits
e523f8cdfeat(types):StaticInfoResponsetypes for v1/static/infoad4105e4feat(core): v1/static/info endpoint with OnceLock wire-bytes cache57372913test: v1/static/info end-to-end integration coverage1be0a371chore: regenerate OpenAPI schema hash for v1/static/infoEach commit compiles cleanly;
cargo test --workspace --no-runpasses.
Test plan
cargo test -p imageflow_types --lib— 47 passed (3 newstatic_info::tests::*)cargo test -p imageflow_core --lib --features c-codecs,zen-codecs— 113 passed (9 newstatic_info::tests::*covering cache pointer equality, format coverage, codec table
coverage, JPEG quality-range union, PNG always-available,
accept.*header origin, server recommendations)cargo test -p imageflow_core --test integration -- static_info— 3 new end-to-end tests pass
cargo test -p imageflow_core --test integration -- killbits::— 27 killbits tests still pass (no regression from the shared
JsonAnswerwiring)cargo test --workspace --no-run— full workspace compilescargo test --features schema-export,json-schema hash_files_relevant_to_schema_and_compareConstraints honored
v1/context/get_net_support.already exposes its capabilities through the associated
EncoderConfig::capabilities()/DecoderConfig::capabilities()method.
Draft.