TRAPI standard for overriding implicit filtering defaults (statistical significance cutoff, negated edges)

[SkyeAv]

Context

Following the Translator engineering call, the cross-team discussion about the new statistical significance qualifier (biolink/biolink-model#1766) concluded that implicit filtering belongs in Gandalf, with defaults overridable via TRAPI — the decisions are recorded in BioPack-team/shepherd#136. Two defaults came out of that discussion:

• Statistical significance qualifier: a default significance cutoff (value TBD by engineering); standard TRAPI qualifier constraints override it.
• Negated edges: excluded from responses by default; clients can query to allow them (cf. Filter negated evidence from Aragorn creative results shepherd#127).

This issue coordinates the open TODO from that discussion — the exact standard for communicating these overrides in TRAPI — with @tokebe, plus the Retriever-side pass-through.

What needs deciding

• The override mechanism. How does a TRAPI query express "use a different significance cutoff" or "allow negated edges"? Candidate: the toplevel Parameters object (precedent: TRAPI parameter to disable Implicit Subclass Reasoning #146 — TRAPI parameter to disable Implicit Subclass Reasoning), query-graph qualifier constraints, or another standard mechanism.
• The default cutoff value for the significance qualifier (engineering TBD).
• Pass-through. Retriever must carry the qualifier itself (feat: implement pass through logic for the new biolink:statistical_significance_qualifier #197) and relay overrides to Gandalf so ARAs can honor them (Shepherd ranking: feat: ranking for biolink:statistical_significance_qualifier shepherd#135; ARAX: feat: rank and filter logic for biolink:statistical_significance_qualifier RTXteam/RTX#2859). Until feat: implement pass through logic for the new biolink:statistical_significance_qualifier #197 ships, no qualifier-bearing edges flow at all.

Related

• Discussion: statistical significance qualifier filtering — filtering belongs in Gandalf, Shepherd-side filter removal shepherd#136 — decision record: filtering belongs in Gandalf; Shepherd-side filter removed
• feat: implement pass through logic for the new biolink:statistical_significance_qualifier #197 — Retriever pass-through for the statistical significance qualifier
• TRAPI parameter to disable Implicit Subclass Reasoning #146 — precedent for query-level TRAPI parameters
• feat: ranking & filtering support for biolink:statistical_significance_qualifier shepherd#134 / feat: ranking for biolink:statistical_significance_qualifier shepherd#135 — Shepherd ranking support (filtering removed)
• Filter negated evidence from Aragorn creative results shepherd#127 — filter negated evidence from Aragorn creative results
• feat: rank and filter logic for biolink:statistical_significance_qualifier RTXteam/RTX#2859 — ARAX ranking for the qualifier
• feat: statistical significance qualifier on associations biolink/biolink-model#1766 — upstream qualifier definition

[tokebe]

Note: Retriever's Tier 1 transpiler will have to be updated to apply these filters to the Tier 1 ES backend queries as well

[SkyeAv]

@tokebe is that something you want me to prototype and open a PR for?

[tokebe]

If you want to dig into the retriever code, but it might be worth waiting until we have an agreed upon standard for overriding it in TRAPI, so we can get it all done in one shot.

Typically that implementation level would go to @shuchenliu, and is within our team's scope of expected feature updates to write, so you can leave it to us if you prefer.