feat(proxy): add opt-in Fetch Relay transport - #975
Conversation
|
Added a deployable Cloudflare Worker example in 8dfe1b4 and completed a live edge smoke test using the built FreeLLMAPI proxyFetch(): JSON POST/Authorization/body round-tripped exactly; SSE chunks arrived at 115 ms and 1,614 ms around a deliberate 1,500 ms upstream delay; 302 remained manual; wrong secret path returned 404; disallowed host returned 403. The test caught and fixed the Workers-to-Workers routing edge case via global_fetch_strictly_public. Both temporary Workers were deleted afterward. Full evidence is now in the PR description. |
|
Follow-up verification with a real provider and the user-facing Dashboard is complete (commit End-to-end path
Production observabilityThe reference Worker now emits structured events for request acceptance, upstream headers, stream completion, policy rejection, configuration errors, and upstream failures. Fields are limited to request ID, method, target hostname, colo/country, status, content type, duration, byte count, and fixed error categories. Workers Logs is enabled in The Relay secret is compared using SHA-256 plus Additional checks after this change:
|
Reference implementationThese are commit-pinned links to the exact Worker source used for the live provider and Playground verification above. |
Closes #974
Problem
Standard proxy transports cannot use URL-based Fetch Relays such as Cloudflare Workers. FreeLLMAPI currently interprets every HTTPS proxy URL as a CONNECT-capable forward proxy.
Summary
This PR adds an opt-in global
fetch-relaytransport beneath the existingproxyFetch()abstraction:forward | fetch-relaymode, settings API persistence,PROXY_MODE, Dashboard selector, and draft Test supportX-FreeLLMAPI-Target-URL; supports an encoded{url}template for existing URL-query relaysDefault behavior is unchanged
proxyMode->forwardPROXY_URL->forwardunlessPROXY_MODE=fetch-relayis explicitALL_PROXY,HTTPS_PROXY,HTTP_PROXY,NO_PROXY, HTTP/HTTPS/SOCKS, and per-key proxies keep their existing behaviorRelay protocol
Preferred form:
Compatibility form:
The header form keeps provider query parameters out of edge URL logs. Relay responses are returned as the original
Response, without callingtext(),json(), orarrayBuffer().Security
redirect: manualTests
Added 16 regression tests, including a real local path through a fake relay and mock upstream.
{url}encoding and Header-first dispatchnpm run buildLive Cloudflare smoke test
The exact example Worker in this PR was temporarily deployed together with a separate mock-upstream Worker and exercised through FreeLLMAPI's built
proxyFetch():Authorizationdata: firstafter 115 ms anddata: secondafter 1,614 ms (the upstream deliberately waited 1,500 ms), proving incremental deliveryBoth temporary Workers were deleted after the smoke test. The test also caught the need for Cloudflare's
global_fetch_strictly_publicflag when an upstream is itself hosted on Workers; the checked-in Wrangler configuration now includes it.Local Windows-only test limitations, unrelated to this diff:
npm teststops in the bootstrap fixture because its spawned Windows PowerShell does not exposeGet-FileHashThis lowers the infrastructure barrier for developers, educators, nonprofits, and researchers who cannot operate an additional proxy server.