Skip to content

Standardize submission attribution: authors vs model, and canonical model names #451

Description

@MathysRennela

Summary

Agents (and humans) regularly get confused about how to attribute a submitted code. Two recurring failure modes:

  1. Author vs model confusion — the producing AI tool/agent gets put in provenance.authors instead of provenance.model, or the author handle is written without the @ prefix / with wrong casing.
  2. Model names are not standardized — the same underlying model appears under several string aliases, so the board's MODEL column and the by-model chart fragment.

The convention ("humans in authors, version-specific model in model") exists only implicitly, scattered across commit messages and post-hoc fix PRs. It is never stated in one authoritative place an agent reads before submitting, and nothing enforces it.

Evidence (from codes/ and git history)

Model-name fragmentation — 14 distinct strings for ~10 real models:

  • DeepSeek V4 Flash 0731 vs Deepseek V4 Flash 0731 (case)
  • Xiaomi MiMo-V2.5 vs Xiaomi Mimo-V2.5 (case)
  • GPT 5.6 Sol vs OpenAI 5.6 Sol vs OpenAI GPT-5.6 Luna (same vendor, three spellings)
  • Ensemble strings with inconsistent separators/member names, e.g. Claude Opus 4.6 + GPT-5.2 + Gemini 3 Pro Preview vs Claude Opus 4.6 + GPT-5.3-Codex + Gemini 3.1 Pro Preview

Attribution fixes already needed in history:

  • f779e80 — AI tool (Antigravity) placed in authors instead of model
  • 51c19bc"Claude-autoresearch" placed in authors
  • 1e7c93a — handle written seunomonije without @, and model never set
  • @mathysrennela vs @MathysRennela (case) both present in the data

Root causes

  1. The convention is nowhere authoritative. CONTRIBUTING.md, research/AUTORESEARCH.md, the qldpc-submit-code skill, and schema/SCHEMA.md each mention pieces, but none states the full rule. Agents default to listing whatever produced the code (often the model) as an author.

  2. The autoresearch path has no way to set the model. research/kit/submit.py's make_submission() takes authors but no model parameter, so candidates staged through the kit (the primary unattended path) silently drop the model — hence the many "set model later" fixes. The CLI (cli/qldpc.py) does have --model, so the two paths disagree.

  3. The normalization script is untracked and not wired in. research/normalize_models.py has a NORMALIZE map and a PROVIDER taxonomy, but it is not committed and nothing calls it. It is a one-time migration, not a guard. The verifier's model check only requires "a digit somewhere", so new variants land freely.

  4. No canonical vocabulary the verifier can enforce. The schema's model field is free text (max 80 chars) with no enum, no case normalization, no provider taxonomy. research/kit/fix_provenance.py (also untracked) even hardcodes the wrong-case "Deepseek V4 Flash 0731", showing the inconsistency propagates through the tooling itself.

  5. The authorship gate is case-insensitive but the data isn't. verify/check_authorship.py lowercases handles for the CI check, so @MathysRennela passes CI even though it renders as a different (broken) link on the site.

Proposed fixes (in rough priority)

  • Document the convention once, in CONTRIBUTING.md (and point the qldpc-submit-code skill at it): humans (with @handle) go in authors; the producing model (version-specific) goes in provenance.model; never put a tool/agent/model name in authors.
  • Add a model parameter to make_submission() so the autoresearch kit path carries the model like the CLI does.
  • Make normalization a guard, not a migration: commit normalize_models.py (or fold its NORMALIZE/PROVIDER map into verify/) and have the verifier flag or reject unknown/case-variant model strings.
  • Add a canonical model vocabulary to the schema (enum or pattern) so the gate enforces standardization rather than just "has a digit".

Notes for the maintainer

  • Changing verify/ is deliberate per AGENTS.md (re-pin with --update + review).
  • This issue is scoped to attribution/metadata only; it does not touch distance claims, witnesses, or the verification stack's trust model.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions