Skip to content

feat: add Hy3 EvalForge MCP server#146

Open
soso23412 wants to merge 1 commit into
Tencent-Hunyuan:rhinobird2026from
soso23412:feature/hy3-evalforge
Open

feat: add Hy3 EvalForge MCP server#146
soso23412 wants to merge 1 commit into
Tencent-Hunyuan:rhinobird2026from
soso23412:feature/hy3-evalforge

Conversation

@soso23412

Copy link
Copy Markdown

Summary

  • Add Hy3 EvalForge, a local stdio MCP server for evidence-backed AI-output evaluation.
  • Provide specification design, case generation, run scoring, and blinded baseline/candidate comparison tools.
  • Enforce artifact path containment, prompt-injection boundaries, secret redaction, deterministic hard checks, evidence validation, and model-call budgets.
  • Add public synthetic calibration pairs, real Hy3 validation artifacts, and a terminal demo GIF.

Validation

  • 35 passed, 1 skipped
  • ruff check .
  • ruff format --check .
  • python -m build --no-isolation
  • python -m twine check dist/*
  • stdio MCP tool discovery smoke test

Live validation

  • Synthetic baseline/candidate comparison completed with Hy3 and returned BLOCKED for one newly introduced critical disclosure.
  • Public 30-pair calibration completed with 90.0% agreement with human reference labels.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant