[FEATURE]: let config-driven models infer comparators for fields the author did not specify

[sromoam]

Motivation

The GenAI IDP Accelerator drives stickler from JSON config. A realistic extraction schema has twenty-plus fields, and an evaluation author cares about the comparison semantics of three or four of them: the invoice ID must match exactly, the total needs numeric tolerance, a date needs date semantics. For the rest they want something reasonable without writing it out.

Today that is either impossible or wrong, depending on which entry point they use.

The problem: four different answers to "no comparator specified"

Measured on dev (d5776b2), same logical model each time:

entry point	comparator	threshold
hand-written StructuredModel, bare annotation	LevenshteinComparator, type-blind	0.7
model_from_json()	raises ValueError	n/a
from_json_schema()	type-based, stops at type	flat 0.5
stickler.evaluate() / auto	type + format + enum + name	tuned per type

model_from_json() rejects omission outright

StructuredModel.model_from_json({"fields": {"total": {"type": "float"}}})
# ValueError: Field 'total' with primitive type 'float' requires a 'comparator'.
#            Primitive fields need comparators to define how they should be compared.

So on the accelerator's own path this is not "choose a better default" -- omission is not currently expressible at all. That is the feature.

The hand-written default is actively wrong

class Invoice(StructuredModel):
    invoice_id: str = ComparableField(comparator=ExactComparator(), threshold=1.0)
    total: float        # unconfigured
    paid: bool          # unconfigured
    issued: date        # unconfigured

field	today	what auto infers
total: float	Levenshtein @ 0.7	NumericComparator @ 0.95
paid: bool	Levenshtein @ 0.7	ExactComparator @ 1.0
issued: date	Levenshtein @ 0.7	DateComparator @ 0.95
vendor: str	Levenshtein @ 0.7	Levenshtein @ 0.85

A float compared by string edit distance means 1000.00 vs 1000.0 scores on character overlap rather than numeric equality. auto/README.md already documents this as the reason that package exists:

Unannotated StructuredModel fields fall through configuration_helper.py's type-blind default and get compared with LevenshteinComparator, which is wrong for float, bool, date.

The gap is that auto fixed it for plain BaseModel and never back-filled the config-driven paths.

from_json_schema() already infers, just shallowly

It gets number -> Numeric and boolean -> Exact, so the precedent for inferring is established. What it misses:

property	from_json_schema()	auto
{"type": "string", "format": "date"}	Levenshtein	DateComparator
{"type": "string", "format": "email"}	Levenshtein	(name/format aware)
{"type": "string", "enum": [...]}	Levenshtein	ExactComparator
threshold, all types	flat 0.5	tuned per type

Proposal

One opt-in setting, honoured by both config paths, routing unspecified fields through stickler.auto.inference instead of the current defaults.

{
  "model_name": "Invoice",
  "infer_unspecified_fields": true,
  "fields": {
    "invoice_id": {"type": "str", "comparator": "ExactComparator", "threshold": 1.0},
    "total": {"type": "float"},
    "paid": {"type": "bool"}
  }
}

Schema-path equivalent: x-aws-stickler-infer-unspecified: true at the object level.

Off by default. Nothing existing moves: model_from_json() keeps raising unless opted in, from_json_schema() keeps its current shallow inference unless opted in. That matters because turning inference on changes reported metrics for any model with an unconfigured non-string field, and a silent metric change is the failure mode to avoid.

Explicitly not proposing "default to Levenshtein @ 0.7 when unspecified". That value is the worst of the four and only exists as an accident of getattr(cls, "match_threshold", 0.5) finding the class default -- see #237. The honest options are inference or an error.

Decisions to make

1. Model-level flag or per-field "comparator": "auto"?

A model-level flag suits the motivating case ("I configured three fields, infer the rest"). A per-field sentinel is more surgical and self-documenting in a config file someone reads later. They are not exclusive; the field-level form could win over the model-level one.

2. Partial config -- is fallback per-parameter?

If a field specifies threshold but not comparator, does it get the inferred comparator with the author's threshold, or the default comparator with their threshold? Per-parameter is more useful, but today the presence of any config means "fully configured", so this is a real behaviour change rather than a new code path.

3. Should the hand-written path change too?

ConfigurationHelper.get_comparison_info's type-blind fallback has the same defect and is not reachable by a JSON flag. Fixing it would help every user, but it is a silent metric change with no opt-in surface, so it likely wants its own decision and its own issue.

Implementation notes

infer_field_config(field_name, field_info) only reads field_info.annotation (inference.py:387), so the config paths need either a synthesized FieldInfo or a small type-based entry point alongside it. No deeper refactor of auto looks necessary.

InferredSpec already carries provenance (e.g. ["type:float -> NumericComparator@0.95"]), which should surface wherever the built model is introspected so an author can see why a field got what it got. That is the part that makes inference debuggable rather than magic.

Acceptance criteria

• A config with infer_unspecified_fields builds a model where unspecified primitive fields get auto-inferred comparators and thresholds
• Without the flag, model_from_json() still raises and from_json_schema() still behaves exactly as today
• Explicit per-field config always wins over inference
• The chosen answer to decision 2 is tested either way
• Inference provenance is retrievable for a built model
• format: date, enum, and numeric types are covered by tests, since those are where the current schema path is weakest
• CHANGELOG notes that enabling the flag moves metrics relative to the previous defaults

Notes

Requested for the GenAI IDP Accelerator integration, where evaluation config is JSON-driven and authors configure a small subset of fields. Overlaps #237, which is the match_threshold-as-field-threshold leak responsible for the 0.7 in the table above.