Skip to content

fix(anthropic): make Anthropic/Claude provider usable out-of-box (three bugs)#2972

Open
he-yufeng wants to merge 1 commit intoHKUDS:devfrom
he-yufeng:fix/anthropic-provider-three-bugs
Open

fix(anthropic): make Anthropic/Claude provider usable out-of-box (three bugs)#2972
he-yufeng wants to merge 1 commit intoHKUDS:devfrom
he-yufeng:fix/anthropic-provider-three-bugs

Conversation

@he-yufeng
Copy link
Copy Markdown
Contributor

Fixes #2955.

The Anthropic/Claude provider in `lightrag/llm/anthropic.py` had three independent defects that each blocked `anthropic_complete` end-to-end under the current Anthropic SDK and LightRAG's own KG extraction pipeline. All three are addressed here in one file.

1. `voyageai` imported unconditionally at module load

`pipmaster`'s auto-install silently no-ops inside a uv-managed venv (or anywhere without site-packages write permission), so the subsequent `import voyageai` at module top level raised `ModuleNotFoundError` for users who never touched Voyage embeddings at all. Moved the `install + import voyageai` pair into `anthropic_embed` so it's only required when embeddings are actually used.

2. `max_tokens` missing

`anthropic.messages.create()` requires `max_tokens`, but the wrapper never set a default and never surfaced it as a documented kwarg. Every call failed with:

```
TypeError: Missing required arguments; Expected either ('max_tokens', 'messages' and 'model')
or ('max_tokens', 'messages', 'model' and 'stream') arguments to be given
```

Added `kwargs.setdefault("max_tokens", 8192)` — matches the other providers' generous default and callers can still override via `**kwargs`.

3. `stream=True` hardcoded

The wrapper always built `create_params` with `"stream": True` and returned an `AsyncIterator[str]`, so LightRAG's KG extraction (which calls the LLM expecting a `str`) crashed with:

```
TypeError: expected string or bytes-like object, got 'async_generator'
```

A caller that overrode with `stream=False` via `**kwargs` landed on the reverse failure — the SDK returned a `Message` and the code's `async for event in response` blew up with `'async for' requires an object with aiter method, got Message`.

Made streaming opt-in: pop `stream` out of `kwargs`, thread it through explicitly, and return concatenated `content[*].text` when `stream=False` (matching `openai_complete`). The existing async-generator path stays intact for `stream=True`.

Test plan

  • Repro from the issue (`rag.ainsert(...)` with `anthropic_complete`) now completes; previously raised at step 2 (`max_tokens` missing) and step 3 (iterator vs string).
  • `stream=True` still yields chunks in the pre-existing shape.
  • `anthropic_embed` still installs/imports voyageai on first use; LLM-only callers no longer hit `ModuleNotFoundError`.
  • `ruff check .` clean on the touched file.

@danielaskdd
Copy link
Copy Markdown
Collaborator

@codex review

@chatgpt-codex-connector
Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Bravo.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@danielaskdd
Copy link
Copy Markdown
Collaborator

The dev branch has undergone significant changes related to LLM invocation. Please rebase your current modifications onto dev, conduct thorough debugging and testing, and then submit a pull request targeting the dev branch.

Fixes HKUDS#2955. The Anthropic/Claude provider had three independent
defects that each blocked `anthropic_complete` under the current
Anthropic SDK and LightRAG's own pipeline.

1. `voyageai` was imported unconditionally at module load. The
   `pipmaster` auto-install silently no-ops in a uv-managed venv, so
   the subsequent `import voyageai` raised ModuleNotFoundError even
   for users who only want the LLM side. Move the install+import
   inside `anthropic_embed` where it's actually needed.

2. `messages.create()` requires `max_tokens` but the wrapper never
   set it. Every call failed with
   `TypeError: Missing required arguments; Expected ('max_tokens',
   'messages' and 'model') ...`. Default to 8192, overridable via
   **kwargs, matching the other providers.

3. `stream=True` was hardcoded, so `anthropic_complete` always
   returned an AsyncIterator[str]. LightRAG's KG extraction expects
   a plain str and crashed with
   `TypeError: expected string or bytes-like object, got 'async_generator'`.
   If the caller overrode with `stream=False` via **kwargs, the SDK
   returned a Message object and `async for` blew up instead. Make
   streaming opt-in: pop `stream` out of kwargs, pass it through
   explicitly, and return the concatenated text content when False;
   keep the existing iterator shape when True.
@he-yufeng he-yufeng force-pushed the fix/anthropic-provider-three-bugs branch from 21717a5 to 52d4cb5 Compare May 6, 2026 14:39
@he-yufeng he-yufeng changed the base branch from main to dev May 6, 2026 14:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Anthropic/Claude provider: 3 bugs prevent out-of-box usage (v1.4.14)

2 participants