Skip to content

ci(test): authenticate HF model pulls in Linux CLI integration to stop 429 flakes#1379

Merged
kovtcharov-amd merged 1 commit into
mainfrom
ci/linux-cli-hf-token
Jun 3, 2026
Merged

ci(test): authenticate HF model pulls in Linux CLI integration to stop 429 flakes#1379
kovtcharov-amd merged 1 commit into
mainfrom
ci/linux-cli-hf-token

Conversation

@kovtcharov-amd
Copy link
Copy Markdown
Collaborator

The Linux "Full Integration" CI job has been flaking on every release/PR — including the v0.20.0 release PR (#1334) and #1377. Root cause: it pulls Qwen3-0.6B-GGUF from HuggingFace unauthenticated, so shared GitHub runners hit 429 Too Many Requests, Lemonade fails to start, and the whole job (summarizer/RAG/lemonade-client integration) fails. The Windows CLI, SD, and Agent SDK jobs already pass HF_TOKEN and never flake — this just brings the Linux job to parity.

After this, the Linux integration job authenticates its model pulls and stops 429-flaking.

Test plan

  • CI on this PR: GAIA CLI Tests (Linux) → Test GAIA CLI on Linux (Full Integration) reaches the model pull without a 429 and passes
  • Confirm no secret leakage in logs (GitHub masks secrets.* automatically)

The Linux "Full Integration" job pulled Qwen3-0.6B-GGUF from HuggingFace
unauthenticated, so shared CI runners got 429-throttled — killing
Lemonade startup and failing the whole job intermittently (seen on
#1377 and #1334). The Windows CLI, SD, and Agent SDK jobs already pass
HF_TOKEN and don't flake. Wire the same secret into the Linux job's
Lemonade step so its pulls are authenticated too.
@github-actions github-actions Bot added the devops DevOps/infrastructure changes label Jun 3, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 3, 2026

Summary

Clean, well-scoped CI fix that brings the Linux integration job to parity with Windows. The change adds HUGGINGFACE_ACCESS_TOKEN / HF_TOKEN to the one step that pulls models from HuggingFace, mirroring the existing pattern in test_gaia_cli_windows.yml:76-77 exactly. The diagnosis (unauthenticated pulls → 429 on shared runners → Lemonade fails to start) is plausible and well-documented, and the inline comment captures the why in two lines without rotting. Most importantly, both model-pull operations are covered: the env block is on the Start Lemonade Server and Test Core Commands step (lines 79–228), which contains both the background lemonade-server-dev run (line 93) and the explicit lemonade-server-dev pull (line 140) — nothing is left unauthenticated.

Issues Found

None blocking.

🟢 Minor — later steps reuse the model but not the token (informational) Subsequent steps like Test evaluation and utility commands (line 230) operate against the already-running server and the model pulled in the prior step, so they don't need the token. Worth keeping in mind only if a future step adds a new model pull — it would need its own env block (or the token promoted to a job-level env:). No action needed for this PR.

Strengths

  • Exact pattern parity with the Windows job (test_gaia_cli_windows.yml:76-77) rather than inventing a new approach — both HF_TOKEN and HUGGINGFACE_ACCESS_TOKEN are set since different layers of the stack read different names.
  • Step-level scoping keeps the secret exposed only where it's needed, not job-wide.
  • Good comment hygiene — one concise why (429 throttling on shared runners) with no narration of mechanics, matching CLAUDE.md's comment guidance.

Verdict

Approve — No blocking issues. The change is minimal, correct, covers both pull sites, and matches the established Windows pattern. Safe to merge once CI confirms the model pull reaches completion without a 429. The unchecked test-plan boxes are the right gate before merge.

@kovtcharov-amd kovtcharov-amd enabled auto-merge June 3, 2026 18:17
@kovtcharov-amd kovtcharov-amd disabled auto-merge June 3, 2026 18:17
@kovtcharov-amd kovtcharov-amd enabled auto-merge June 3, 2026 18:22
@kovtcharov-amd kovtcharov-amd added this pull request to the merge queue Jun 3, 2026
Merged via the queue into main with commit 0bc65ee Jun 3, 2026
22 checks passed
@kovtcharov-amd kovtcharov-amd deleted the ci/linux-cli-hf-token branch June 3, 2026 18:23
kovtcharov-amd pushed a commit that referenced this pull request Jun 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

devops DevOps/infrastructure changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants