feat: add BaseRT as first-class LLM, embedder, and agent provider

[IsuraManchanayake]

Pull Request Type

• ✨ feat (New feature)
• 🐛 fix (Bug fix)
• ♻️ refactor (Code refactoring without changing behavior)
• 💄 style (UI style changes)
• 🔨 chore (Build, CI, maintenance)
• 📝 docs (Documentation updates)

Relevant Issues

None

Description

Adds BaseRT, a local LLM inference
runtime for Apple Silicon as a first-class provider for LLM chat, embeddings,
and agents.

Why BaseRT?

BaseRT is an open (Apache-2.0 CLI, format, and bindings) inference runtime
built directly on Apple's Metal API with no intermediate framework, developed
by Base Compute and described in a peer-reviewable
paper (arXiv:2607.00501). Its published
benchmarks report the highest LLM inference throughput measured on Apple
Silicon to date: up to 1.56x the decode throughput of llama.cpp and 1.35x of
MLX on M3/M4-class hardware, across Qwen3, Llama 3.2, and Gemma 4 families at
Q4/Q8 quantization.

On the new M5 generation the gap widens considerably. A follow-up technical
report (BaseRT on M5) shows BaseRT
driving the M5's per-core GPU Neural Accelerators through hand-written Metal 4
tensor kernels, reaching up to 6.4× the prompt-processing (prefill)
throughput of llama.cpp and 3.9x of MLX on an M5 Pro across fifteen model
configurations (sub-1B to 35B, dense and MoE), while holding a smaller decode
lead (up to 1.75x / 1.33x) since decode is memory-bandwidth-bound for every
runtime.

The prefill numbers are what make this directly relevant to AnythingLLM:
RAG-style chats send long prompts (system prompt + retrieved context chunks +
history), so time-to-first-token is dominated by prompt processing, exactly
the phase where BaseRT's advantage is largest. For users running AnythingLLM
fully locally on a Mac, that translates into noticeably lower first-token
latency on document-heavy workspaces compared to the existing llama.cpp- and
MLX-based options. It also fits AnythingLLM's local-first lineup (Ollama,
LM Studio, oMLX, LocalAI) with the same privacy properties: models and chats
never leave the machine.

Practically, BaseRT ships a single CLI (basert pull / serve) that pulls
models from HuggingFace and serves an OpenAI-compatible API with native tool
calling, embeddings, and multi-model serving -- everything this integration
consumes.

What this PR adds

• LLM provider (server/utils/AiProviders/baseRT) - chat + SSE streaming
  with exact token metrics (BaseRT emits usage in the final stream chunk);
  per-model context windows auto-detected from meta.n_ctx on GET /v1/models
  and cached at boot (same pattern as oMLX/LM Studio), with optional user
  override and a 16,384-token default cap; vision capability read from the
  model's architecture.input_modalities.
• Embedding engine (server/utils/EmbeddingEngines/baseRT) - sequential
  chunk embedding via /v1/embeddings; explicitly requests
  encoding_format: "float" since BaseRT rejects the OpenAI SDK's base64
  default.
• Agent provider (server/utils/agents/aibitat/providers/basert.js) -
  native OpenAI tool calling by default (verified working), with the standard
  UnTooled fallback via PROVIDER_DISABLE_NATIVE_TOOL_CALLING.
• Frontend - provider entries in LLM Preference, Embedding Preference,
  onboarding, agent LLM selection, and the privacy table; endpoint
  auto-discovery on port 8080 (BaseRT's default); live model dropdown from
  /v1/models; optional Bearer API key for servers started with --api-key.

New ENV vars (documented in server/.env.example): BASERT_LLM_BASE_PATH,
BASERT_LLM_MODEL_PREF, BASERT_LLM_TOKEN_LIMIT, BASERT_LLM_API_KEY.

Visuals (if applicable)

Additional Information

Tested end-to-end against a live basert serve instance (two models loaded):
model listing, context-window auto-detection (40960 reported, capped to 16384
by default), non-streaming and streaming chat with usage metrics, native tool
calling through the agent provider, embeddings, and the settings UI
(auto-detect + model dropdown) in the running app.

Implementation follows the existing oMLX provider pattern throughout, so all
touchpoints (env validation, custom model listing, eager context-window
caching, agent wiring) mirror established conventions.

Developer Validations

• I ran yarn lint from the root of the repo & committed changes
• Relevant documentation has been updated (if applicable)
• I have tested my code functionality
• Docker build succeeds locally