A local-first studio for testing and comparing LLM prompts — free, runs on your machine.
You change a prompt or swap a model and behavior shifts. The usual way to check is running it a few times and eyeballing the output. bodhirax makes "did this get better or worse?" a repeatable, visual answer, without sending your prompts or API keys to anyone.
| Code-first (Promptfoo, DeepEval) | SaaS (Confident AI, Braintrust) | bodhirax | |
|---|---|---|---|
| Visual UI | No | Yes | Yes |
| Runs locally | Partial | No | Yes |
| Free | Yes | No | Yes |
| Data stays on your machine | Yes | No | Yes |
| Works with local models | Partial | Partial | Yes |
Linux / macOS
git clone https://github.com/Kunal-Somani/bodhirax
cd bodhirax/backend
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
uvicorn app.main:appIn a second terminal:
cd bodhirax/frontend
npm install && npm run devOpen http://localhost:3000
Windows (WSL)
Same as Linux / macOS — run inside a WSL terminal.
Docker
git clone https://github.com/Kunal-Somani/bodhirax
cd bodhirax
docker compose upWith local Ollama models: docker compose --profile ollama up
Open http://localhost:3000
Verified: Linux. macOS expected to work. Windows via WSL or Docker.
graph LR
U[You] --> UI[bodhirax UI]
UI --> API[Local server]
API --> ENG[Eval orchestrator]
ENG --> LOCAL[Ollama · local models]
ENG --> CLOUD[OpenAI / Anthropic / Groq / HF]
ENG --> M[Metric engine]
API --> DB[(SQLite · on your machine)]
Full internal architecture, data model, and route list: docs/architecture.md.
- exact — does the output match the expected string, after normalizing case and whitespace.
- fuzzy — Levenshtein similarity, tolerates small wording differences.
- embedding — local sentence-embedding cosine similarity, for "means the same thing."
- llm_judge — a model scores the output 1–5 against a rubric. The only metric that costs inference.
Full methodology and the LLM-judge bias notes: docs/metrics.md.
Ollama (free, local), OpenAI, Anthropic, Groq. Keys are stored locally and never logged or returned by the API.
Hugging Face (experimental — classic Inference API; verify model availability)
Setup for each: docs/providers.md.
CONTRIBUTING.md covers dev setup, code rules, and how to add a provider or metric — both are deliberately one file each.
Local-first by design: the backend binds to 127.0.0.1, and nothing leaves your machine except a request to the specific provider you configured. Details and how to report a vulnerability: SECURITY.md.