Skip to content

Backfill page_or_section on the existing research evidence rows #38

Description

@matentzn

The medic-research-curation skill extracts page hints from the .citations.md files and writes
page_or_section on every evidence row it produces. Every research file we have predates that
change: 0 of the 35 kb/research/<MONDO>.yaml files contain the field, across all 164 evidence
references.

page_or_section is what turns a bare PMID into a locatable claim — it is the difference between
"this paper supports the association" and "this paper supports it here". Without it the research
axis is the only evidence surface in MeDIC where a reader cannot get from the record to the
supporting text without re-reading the whole reference. Going forward the skill fills it; the
existing rows will stay empty unless something backfills them or every disease file is re-curated.

The open question is which. A non-LLM backfill script would scan the cached .citations.md files,
match each existing evidence row by reference_title / reference, and lift the page hint where one
exists — cheap, deterministic, and it touches nothing else in the record. The alternative is to leave
the rows alone and let coverage fill in as diseases are re-curated, which costs nothing now but
leaves the field permanently uneven.

Coverage will be uneven either way: Falcon citations carry page hints reliably, Perplexity
citations rarely do, and the bulk of the current 35 files are Perplexity-sourced. A backfill run on
scripts/backfill_research_pages.py in 2026-05 returned 0 hits for exactly that reason. If the
Falcon share of the corpus is still small, the script is not worth writing until after the next
research batch lands.

Examples

$ ls kb/research/*.yaml | wc -l
35
$ grep -l "page_or_section" kb/research/*.yaml | wc -l
0

Related

  • Expand the research axis with ClinicalTrials.gov and systematic PubMed #32 — expands the research axis with ClinicalTrials.gov and systematic PubMed. Trial records carry
    their own locators, so anything ingested there arrives with the field populated; this issue is only
    about the deep-research backlog.
  • .claude/skills/medic-research-curation/SKILL.md — the emitter for new rows.
  • scripts/backfill_research_pages.py — the existing backfill script, previously a no-op.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions