Feat/add litellm provider

[RheagalFire]

Summary

Adds LiteLLM as a first-class inference provider, enabling access to 100+ LLM providers (OpenAI, Anthropic, Google Gemini, AWS Bedrock, Azure OpenAI, Mistral, Cohere, and more) through automaton's existing inference routing system.

Changes

• src/conway/inference.ts - Added LiteLLM as a recognized InferenceBackend, routed through chatViaOpenAiCompatible with provider-specific error messages (401/429/404), streaming opt-in via opts.stream
• src/index.ts - LiteLLM provider startup: reads LITELLM_BASE_URL/LITELLM_API_KEY env vars, auto-discovers available models from proxy via GET /v1/models, populates the model registry
• src/types.ts - Added "litellm" to ModelProvider union type, added litellmBaseUrl/litellmApiKey to config
• src/inference/registry.ts - Registered litellm in the provider registry
• src/__tests__/inference-router.test.ts - 4 unit tests covering model registry, auto-disable skip, model discovery, fallback routing

Tests

Unit tests (68/68 passing):
All existing tests plus 4 new LiteLLM-specific tests pass.

Risk / Compatibility

• Additive only. No existing providers modified.
• LiteLLM backend is opt-in via LITELLM_BASE_URL env var.
• Auto-discovery from /v1/models populates available models at startup.
• Streaming supported via opts.stream (same as other backends).

Example usage

1. Configure the LiteLLM proxy connection:

export LITELLM_BASE_URL=http://localhost:4000
export LITELLM_API_KEY=sk-your-litellm-key

Or in your automaton config:

{
  "litellmBaseUrl": "http://localhost:4000",
  "litellmApiKey": "sk-your-litellm-key"
}

**2. Any model served by your LiteLLM proxy is registered with provider: "litellm" and available for inference routing. The startup log shows:

LiteLLM: discovered 12 models from http://localhost:4000/v1/models
LiteLLM backend: http://localhost:4000

3. Use any LiteLLM-routed model in inference calls:

const inference = createInferenceClient({
  apiUrl: config.conwayApiUrl,
  apiKey: config.conwayApiKey,
  litellmBaseUrl: "http://localhost:4000",
  litellmApiKey: "sk-your-litellm-key",
  defaultModel: "openai/gpt-4o",
  maxTokens: 4096,
});

// Non-streaming
const response = await inference.chat([
  { role: "user", content: "What is 2+2?" },
], { model: "anthropic/claude-sonnet-4-20250514" });

console.log(response.content);

// Streaming
const stream = await inference.chat([
  { role: "user", content: "Tell me a joke" },
], { model: "openai/gpt-4o", stream: true });

// Tool calling
const toolResponse = await inference.chat([
  { role: "user", content: "What's the weather in Tokyo?" },
], {
  model: "openai/gpt-4o",
  tools: [{
    type: "function",
    function: {
      name: "get_weather",
      description: "Get current weather",
      parameters: { type: "object", properties: { city: { type: "string" } } },
    },
  }],
});

4. Model routing is automatic. Once registered, models route to the LiteLLM backend based on the model registry lookup - no manual backend selection needed. The inference router picks the right backend (conway, openai, anthropic, ollama, or litellm) based on the model's registered provider.