Fix Claude Code / Opus 4.8 compatibility and web_search reliability - #235
Open
coderhisham wants to merge 6 commits into
Open
Fix Claude Code / Opus 4.8 compatibility and web_search reliability#235coderhisham wants to merge 6 commits into
coderhisham wants to merge 6 commits into
Conversation
…xt windows (jwadow#208) Add claude-opus-4.8 to the fallback model list with a 1M maxInputTokens so it is recognized on the runtime.kiro.dev endpoint (which has no /ListAvailableModels). Make context-window accounting configurable: DEFAULT_MAX_INPUT_TOKENS is now an env var, and MODEL_CONTEXT_WINDOWS lets users declare per-model windows (e.g. 1M) that take precedence over discovered/default limits. This only affects token-usage estimation; requests remain pass-through. Adds tests for the override resolution chain, the env parser, and the opus-4.8 fallback entry.
When the auth provider rejects the refresh token (e.g. AWS SSO OIDC 400 invalid_grant), the gateway previously surfaced it as a generic HTTP 500 'Internal error'. Add InvalidRefreshTokenError, raise it from get_access_token when the provider response indicates an invalid/expired token, and map it to a clear 401 (authentication_error) with re-auth guidance in both API routes. Transient/server errors (e.g. 500) still propagate unchanged. Adds tests for the detector and for get_access_token raising the typed error vs propagating others.
…ow#231, jwadow#173) Two root causes made web_search fail and surface as 'No such tool available' in clients: 1. The /mcp call sent only a bare Authorization header. It now sends the full Kiro client-identity headers (User-Agent/x-amz-user-agent/x-amzn-kiro-agent-mode) adjusted for JSON-RPC, and includes top-level profileArn for Enterprise accounts. Error logging now includes the response body. 2. On MCP failure the gateway fell through and emitted the raw web_search tool_use/tool_call to the client. All paths now degrade gracefully: - Anthropic streaming + non-streaming emit a native web_search_tool_result (error) instead of leaking a tool_use. - OpenAI streaming (and non-streaming via reuse) emit an in-band unavailable note instead of a tool_call. Adds shared outcome helpers and tests across both APIs and both modes.
…blocks (jwadow#190, jwadow#219) Claude Code sends two shapes the strict models rejected with 422: 1. Inline messages with role 'system' in the messages array. AnthropicMessage.role is now a free-form string (mirroring OpenAI ChatMessage); inline system messages are hoisted into the top-level system prompt (separate_inline_system_messages), matching the OpenAI adapter and avoiding spurious user turns. 2. Follow-up turns echoing back the server_tool_use / web_search_tool_result blocks the gateway emitted. These block types are now part of the ContentBlock union, and convert_anthropic_content_to_text folds an echoed web_search_tool_result into text so search grounding survives conversion. Adds model-validation, converter, and route tests for both shapes.
Stopping a stuck/orphaned gateway previously meant manually running lsof + kill. Add cross-platform port utilities (kiro/port_utils.py) and two CLI options: - --stop: stop any process listening on the resolved port, then exit. - --force: if the port is in use at startup, stop the occupant and start anyway. Also adds a preflight check so an occupied port produces a clear, actionable message (with the exact --stop/--force commands) instead of a raw 'address already in use' traceback. Only processes LISTENING on the exact port are targeted, and the current process is always excluded. Adds test_port_utils.py covering detection, lsof/netstat parsing, self-exclusion, graceful/forceful termination, and free_port orchestration.
On Ctrl+C/SIGTERM, uvicorn waits for in-flight connections to drain. With long-lived streaming (STREAMING_READ_TIMEOUT=300), an open SSE connection kept the process alive and the port bound for minutes - which is why a stopped gateway sometimes lingered on the port. Add SHUTDOWN_TIMEOUT (env, default 10s) and pass it to uvicorn as timeout_graceful_shutdown, so shutdown is bounded and the port is freed promptly. A second Ctrl+C still forces an immediate exit. Adds CLI-parsing tests for --stop/--force and config tests for SHUTDOWN_TIMEOUT.
|
Thanks for the PR! 🎉 Before merge, we need a one-time CLA confirmation. Full CLA text: Please reply once with: You need to write once, all further messages from me can be ignored. |
|
@coderhisham could you give a try to this gateway: https://github.com/ankitcharolia/kiro-gateway It works quite well with All AI harness and actively being developed. The most important thing is that it is ACP compliant |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR bundles several fixes that make the gateway work reliably with the latest Claude Code (Opus 4.8), plus quality-of-life improvements for running the server. All changes keep the gateway's pass-through/transparent philosophy, are consistent across both the OpenAI and Anthropic surfaces (and streaming + non-streaming), and ship with tests. Full suite passes (only 2 environment-specific
.envdefault-host/port tests fail locally, unrelated to these changes).What's included
1.
feat(models): Claude Opus 4.8 (1M context) + configurable context windows — closes #208claude-opus-4.8toFALLBACK_MODELSwith a 1MmaxInputTokens, so it's recognized on theruntime.kiro.devendpoint (which has no/ListAvailableModels).DEFAULT_MAX_INPUT_TOKENSis now an env var, and a newMODEL_CONTEXT_WINDOWSJSON env lets users declare per-model windows that take precedence over discovered/default limits. Affects token-usage estimation only; requests stay pass-through.2.
fix(anthropic): accept inlinesystemrole + server-side web_search blocks — closes #190, #219 (also #211, #226)role: "system"messages, which the strictLiteral["user","assistant"]rejected with 422.AnthropicMessage.roleis now a free-form string (mirroring the OpenAI model); inline system messages are hoisted into the top-level system prompt, matching the OpenAI adapter and avoiding spurious user turns.server_tool_use/web_search_tool_resultblocks. These are now part of theContentBlockunion, and their content is folded into text so search grounding survives conversion.3.
fix(mcp): reliable web_search, never leak the tool call — closes #231, #173/mcpcall now sends the full Kiro client-identity headers (it previously sent onlyAuthorization, causing 403) and includes top-levelprofileArnfor Enterprise/Builder accounts.web_searchtool_use/tool_call to the client (which surfaced as "No such tool available: web_search"). It now degrades gracefully — Anthropic emits a nativeweb_search_tool_resulterror; OpenAI emits an in-band "unavailable" note — across streaming and non-streaming.4.
fix(auth): actionable 401 on invalid/expired refresh tokeninvalid_grant) previously surfaced as a generic 500. It now raises a typed error mapped to a clear 401 with re-auth guidance in both routes. Transient/server errors still propagate unchanged.5.
feat(cli):--stopand--forceto free the server port--stopstops any process listening on the resolved port and exits;--forcestops the occupant and starts anyway. Adds a preflight check so an occupied port gives an actionable message instead of a raw "address already in use" traceback. Cross-platform (lsof/netstat), only targets processes listening on the exact port, never the current process.6.
fix(server): bounded graceful shutdown so Ctrl+C frees the port promptlySHUTDOWN_TIMEOUT(env, default 10s) passed astimeout_graceful_shutdown. A second Ctrl+C still forces immediate exit.Testing
Notes
.env.example:DEFAULT_MAX_INPUT_TOKENS,MODEL_CONTEXT_WINDOWS,SHUTDOWN_TIMEOUT.