feat(llm): add LiteLLM AI gateway as a backend for LLMProvider

[RheagalFire]

Description

This PR adds LiteLLM as an embedded AI gateway backend in LLMProvider, opt-in via provider="litellm". It is not a separate proxy server. The framework imports the litellm SDK directly and routes
every completion through litellm.completion / litellm.acompletion, so a single LLMProvider instance can talk to OpenAI, Anthropic, Vertex AI, Bedrock, Azure, Cohere, Mistral, Groq, Ollama, and 90+ other
hosted backends without configuring each SDK individually.

What it does:

• Adds 4 new functions in core/llm.py: call_llm_litellm, call_llm_with_tools_litellm, plus async variants. Same call-site contract as the existing call_llm / call_llm_with_tools, but routes through
  LiteLLM.
• Adds a provider parameter to LLMProvider.__init__ (default "heurist", new option "litellm"). When "litellm", the existing call() method dispatches to the new functions.
• Adds litellm>=1.60,<1.85 as a dependency.

Why:

• One backend, many upstream providers. Users can run agents against Anthropic Sonnet, Bedrock Claude, Vertex AI Gemini, Azure GPT, Groq Llama, etc. by changing only large_model_id and the relevant env var.
• Credentials are resolved per-provider from env vars (ANTHROPIC_API_KEY, OPENAI_API_KEY, AWS_ACCESS_KEY_ID, AZURE_API_KEY, ...) by default, with optional api_key / base_url for OpenAI-compatible
  custom endpoints (private LiteLLM proxies, Azure Foundry deployments, etc.).
• drop_params=True is on by default so kwargs that some providers reject (presence_penalty / frequency_penalty on Anthropic, Gemini, Bedrock; response_format on Bedrock; etc.) are silently dropped
  instead of raising UnsupportedParamsError.

Backward compatibility: the default backend stays "heurist". All existing LLMProvider(...) call sites in agents/, mesh/, interfaces/, and core/examples/ are unchanged.

Usage:

from core.components.llm_provider import LLMProvider

# Default Heurist gateway path, unchanged
heurist = LLMProvider(                                                                                                                                                                                             
    base_url=HEURIST_BASE_URL,
    api_key=HEURIST_API_KEY,                                                                                                                                                                                       
    large_model_id=LARGE_MODEL_ID,
    tool_manager=tools,                                                                                                                                                                                            
)               
                                                                                                                                                                                                                   
# New LiteLLM backend; model uses LiteLLM's provider-prefixed name
litellm_provider = LLMProvider(                                                                                                                                                                                    
    provider="litellm",
    large_model_id="anthropic/claude-3-5-sonnet-20241022",                                                                                                                                                         
    tool_manager=tools,                                                                                                                                                                                            
)
                                                                                                                                                                                                                   
text, image_url, tool_back = await litellm_provider.call(
    system_prompt="You are a helpful assistant.",                                                                                                                                                                  
    user_prompt="What is 2+2?",                                                                                                                                                                                    
    temperature=0.0,                                                                                                                                                                                               
)                                                                                                                                                                                                                  
                

##How Has This Been Tested?

Unit tests (17 / 17 pass)

Added tests/test_litellm_provider.py covering:

• _litellm_kwargs builder (drop_params default, credential propagation, user-kwargs override).
• call_llm_litellm returns content dict, propagates tool calls, raises LLMError on failure, forwards temperature / max_tokens / litellm_kwargs.
• call_llm_litellm_async and call_llm_with_tools_litellm_async variants.
• LLMProvider init: default backend, litellm backend skips Heurist env, accepts explicit credentials, rejects unknown backend.
• LLMProvider._dispatch picks the correct call functions for each backend.
• LLMProvider.call() with provider="litellm" actually routes through litellm.completion.

  $ ruff check --select=I --line-length=120 core/llm.py core/components/llm_provider.py tests/test_litellm_provider.py                                                                                               
  All checks passed!                                                                                                                                                                                                 
                                                                                                                                                                                                                     
  $ ruff format --check --line-length=120 core/llm.py core/components/llm_provider.py tests/test_litellm_provider.py                                                                                                 
  3 files already formatted
                                                                                                                                                                                                                     
  $ pytest tests/test_litellm_provider.py
  ======================= 17 passed in 0.66s =======================                                                                                                                                                 
                  
  Live E2E (Anthropic Claude Sonnet 4-6 via Azure AI Foundry)                                                                                                                                                        
   
  Three scenarios verified end-to-end against a real provider:                                                                                                                                                       
                  
  === call() :: anthropic/claude-sonnet-4-6 ===                                                                                                                                                                      
    text       : '4'                                                                                                                                                                                                 
    image_url  : None                                                                                                                                                                                                
    tool_back  : None                                                                                                                                                                                                
    PASS                                                                                                                                                                                                             
                  
  === drop_params=True default :: anthropic/claude-sonnet-4-6 (presence_penalty=0.5) ===                                                                                                                             
    text       : '4'
    PASS  (Anthropic accepted; presence_penalty silently dropped)                                                                                                                                                    
                                                                                                                                                                                                                     
  === tools=[get_weather] :: anthropic/claude-sonnet-4-6 ===                                                                                                                                                         
    text       : '\nIt is sunny and 22C in Tokyo'                                                                                                                                                                    
    PASS  (LiteLLM parsed Anthropic tool call, dispatched through tool_manager.execute_tool, result merged into the text response)                                                                                   

The tool-calling test instantiates a FakeToolManager whose execute_tool returns a string, then verifies the full chain: Anthropic returns tool_calls, LiteLLM normalizes the response shape, LLMProvider.call()
extracts the call, dispatches to tool_manager.execute_tool, and merges the result back into the text.

Reproduce

  uv sync                                                                                                                                                                                                            
  export ANTHROPIC_API_KEY=sk-ant-...
  uv run pytest tests/test_litellm_provider.py

  import asyncio
  from core.components.llm_provider import LLMProvider                                                                                                                                                               
                                                                                                                                                                                                                     
  p = LLMProvider(
      provider='litellm',                                                                                                                                                                                            
      large_model_id='anthropic/claude-3-5-sonnet-20241022',
  )                                                                                                                                                                                                                  
  text, _, _ = asyncio.run(p.call(
      system_prompt='Reply with just the number.',                                                                                                                                                                   
      user_prompt='2+2?',                                                                                                                                                                                            
  ))
  print(text)                                                                                                                                                                                                         

Checklist

• Documentation: I have updated or added documentation where needed (in code, README, or other docs).
• Tests:
  • Added or updated tests to cover my changes. All tests pass locally on my machine.
  • I have added a test script in mesh/tests/ that instantiates my mesh agent and calls its handle_message with example input.
  • No tests needed for this PR.
• Metadata (for Heurist Mesh Agents): I have filled out or updated the agent's metadata (name, description, author, inputs/outputs, etc.) if relevant.
• No Duplicates: I have checked that no other PR is addressing the same issue or feature.
• No Breaking Changes: The changes in this PR do not break existing functionality. If they do, I have clearly explained the impact below.

Additional Notes

[RheagalFire]

cc @wjw12

[RheagalFire]

@rexdotsh do you have any update on this PR?