Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -87,3 +87,7 @@ CLAUDE.md

# Google Jules
.jules/

# Playwright local artifacts
/lightrag_webui/test-results/
/lightrag_webui/playwright-report/
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Changelog

## Unreleased

- Added Little Bull premium control-plane contracts and PostgreSQL schema.
- Added local tri-database pilot path for Postgres, Neo4j, and Qdrant.
- Added non-destructive Phase 3 data-plane pilot harness and validation tests.
2 changes: 2 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ RUN --mount=type=cache,target=/root/.local/share/uv \

# Copy project sources after dependency layer
COPY lightrag/ ./lightrag/
COPY lightrag_enterprise/ ./lightrag_enterprise/

# Include pre-built frontend assets from the previous stage
COPY --from=frontend-builder /app/lightrag/api/webui ./lightrag/api/webui
Expand Down Expand Up @@ -79,6 +80,7 @@ ENV UV_SYSTEM_PYTHON=1
COPY --from=builder /root/.local /root/.local
COPY --from=builder /app/.venv /app/.venv
COPY --from=builder /app/lightrag ./lightrag
COPY --from=builder /app/lightrag_enterprise ./lightrag_enterprise
COPY pyproject.toml .
COPY setup.py .
COPY uv.lock .
Expand Down
18 changes: 18 additions & 0 deletions docs/cli-reference.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# CLI Reference

## Project Commands

- `python -m lightrag_enterprise.system.migrate`
- `uv run ruff check lightrag_enterprise tests_enterprise lightrag/api/lightrag_server.py`
- `./scripts/test.sh tests_enterprise -q`
- `node /Users/joao_tourinho/Documents/specops-tooling-os/packages/cli/dist/index.js validate`

## Pilot Commands

- `scripts/little_bull_phase3_pilot.py` runs opt-in data-plane pilots.
- `scripts/little_bull_phase3_inventory.py` lists pilot artifacts without deletion.

## Acceptance Criteria

- CLI commands do not echo secrets.
- Destructive commands are not run without explicit confirmation.
16 changes: 16 additions & 0 deletions docs/github-integration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# GitHub Integration

## Policy

Pull requests should target `HKUDS/LightRAG:main` when publishing upstream work, from a dedicated branch.

## Checks

- Include summary, operational impact, linked issues, and validation evidence.
- Use authenticated GitHub tooling only when credentials are valid.
- Do not expose secrets in PR text, logs, screenshots, or artifacts.

## Acceptance Criteria

- PRs include test evidence for touched surfaces.
- Upstream attribution and fork policy are preserved.
17 changes: 17 additions & 0 deletions docs/installation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Installation

## Local Setup

Use the repository setup flow and keep secrets in local environment files that are not committed.

## Commands

- `python -m venv .venv && source .venv/bin/activate`
- `pip install -e .`
- `pip install -e .[api]`
- `cd lightrag_webui && bun install`

## Acceptance Criteria

- Dependencies install without committing generated environments.
- `.env` values remain local and are never printed in shared logs.
106 changes: 106 additions & 0 deletions docs/little-bull-diagnostic-artifacts-inventory.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
# Little Bull Diagnostic Artifacts Inventory

Status date: 2026-05-01

This inventory is read-only documentation for possible future cleanup. No cleanup has been executed.
It contains no secrets, tokens, passwords, `.env` values or connection strings.

## Rule

Do not delete, drop, truncate, reset, prune, or remove any item in this file without explicit confirmation that names the exact targets.

## Local Services Observed

- `trag-phase2-postgres-1`: healthy, loopback Postgres service.
- `trag-phase2-neo4j-1`: healthy, loopback Neo4j service.
- `trag-phase2-qdrant-1`: healthy, loopback Qdrant service.

These containers are diagnostic. Stopping them without volume deletion is lower risk than removing containers or volumes, but still should be coordinated if active tests depend on them.

## Filesystem Artifacts

Candidate directories under `rag_storage/`:

- `rag_storage/phase21_smoke_18311564788c`: empty directory observed.
- `rag_storage/phase21_smoke_820a9f72e3b4`: 10 files observed.
- `rag_storage/reindex_smoke_933a0228`: empty directory observed.
- `rag_storage/reindex_smoke_ac241841`: empty directory observed.

Candidate directories under `inputs/`:

- `inputs/phase21_smoke_18311564788c`
- `inputs/phase21_smoke_820a9f72e3b4`
- `inputs/phase21_smoke_820a9f72e3b4/__enqueued__`
- `inputs/phase21_tribank_44657da3ed86`
- `inputs/phase21_tribank_44657da3ed86/__enqueued__`
- `inputs/reindex_smoke_933a0228`
- `inputs/reindex_smoke_ac241841`

Known document in `rag_storage/phase21_smoke_820a9f72e3b4`:

- `doc-1588f7b35aed62f7a69230c53c9b711a`

Visual smoke screenshots are temporary and may or may not exist depending on `/tmp` retention:

- `/tmp/trag-little-bull-premium-mobile.png`
- `/tmp/trag-little-bull-premium-tablet.png`
- `/tmp/trag-little-bull-premium-desktop.png`

## Qdrant Artifacts

Collections observed on the local diagnostic Qdrant service:

- `lightrag_vdb_chunks_openai_text_embedding_3_small_1536d`
- `lightrag_vdb_entities_openai_text_embedding_3_small_1536d`
- `lightrag_vdb_relationships_openai_text_embedding_3_small_1536d`
- `lightrag_vdb_chunks_phase3_fake_local_phase3_pilot_20260430130751_16d`
- `lightrag_vdb_entities_phase3_fake_local_phase3_pilot_20260430130751_16d`
- `lightrag_vdb_relationships_phase3_fake_local_phase3_pilot_20260430130751_16d`
- `lightrag_vdb_chunks_phase3_fake_local_phase3_pilot_20260430130805_16d`
- `lightrag_vdb_entities_phase3_fake_local_phase3_pilot_20260430130805_16d`
- `lightrag_vdb_relationships_phase3_fake_local_phase3_pilot_20260430130805_16d`
- `lightrag_vdb_chunks_phase3_fake_local_phase3_pilot_20260430130847_16d`
- `lightrag_vdb_entities_phase3_fake_local_phase3_pilot_20260430130847_16d`
- `lightrag_vdb_relationships_phase3_fake_local_phase3_pilot_20260430130847_16d`

Workspace-specific counts observed for `phase21_tribank_44657da3ed86`:

- `lightrag_vdb_chunks_openai_text_embedding_3_small_1536d`: 1 point with `workspace_id=phase21_tribank_44657da3ed86`.
- `lightrag_vdb_entities_openai_text_embedding_3_small_1536d`: 5 points with `workspace_id=phase21_tribank_44657da3ed86`.
- `lightrag_vdb_relationships_openai_text_embedding_3_small_1536d`: 0 points with `workspace_id=phase21_tribank_44657da3ed86`.

Do not delete whole shared OpenAI collections just to remove one workspace. A future cleanup should use workspace-filtered point deletion only after explicit approval.

## Neo4j Artifacts

Indexes observed in the local diagnostic Neo4j service:

- `entity_id_fulltext_idx_phase21_smoke_18311564788c`
- `entity_id_fulltext_idx_phase21_smoke_820a9f72e3b4`
- `entity_id_fulltext_idx_phase21_tribank_44657da3ed86`

Workspace-property counts for `phase21_tribank_44657da3ed86` returned 0 nodes and 0 relationships. The index remains a cleanup candidate.

## PostgreSQL / Control Plane Artifacts

Known diagnostic workspaces/documents created during live smokes:

- workspace `phase21_smoke_820a9f72e3b4`
- document `doc-1588f7b35aed62f7a69230c53c9b711a`
- workspace `phase21_tribank_44657da3ed86`
- document `doc-974a377549c90fa3d2434ec663c8b1f4`

PostgreSQL cleanup must be planned from table metadata and foreign-key order before any `DELETE`, `DROP`, or `TRUNCATE`.
Prefer a read-only SQL inventory first, then a dry-run transaction script that rolls back, then an approved cleanup transaction.

## Safe Next Step For Cleanup

If cleanup is approved later, first generate a read-only cleanup packet containing:

- exact filesystem paths;
- exact Qdrant collection names and workspace-filter predicates;
- exact Neo4j index names and any node/relationship predicates;
- exact PostgreSQL tables and row counts by workspace/document id;
- rollback notes or rebuild path for each plane.

No destructive command should be run before that packet is reviewed and approved.
150 changes: 150 additions & 0 deletions docs/little-bull-erros-operacionais.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,150 @@
# Little Bull/TRAG - erros operacionais que nao podem se repetir

Data: 2026-05-02
Escopo: Docker local, autenticacao, UI Little Bull Premium e validacao visual.
Status: ativo. Este documento e bloqueante para qualquer declaracao de READY.

## Regra de ouro

Nao declarar "pronto", "READY", "no ar funcional" ou "entregue" so porque Docker subiu, build passou, endpoint respondeu 200 ou contrato backend existe.

Para declarar entregue, precisa haver evidencia minima:

- login visual feito como usuario real;
- smoke visual da pagina no navegador;
- console e network sem erro critico;
- fluxo principal executado, nao apenas tela renderizada;
- screenshot ou relatorio objetivo;
- riscos residuais tratados ou com plano claro de correcao.

## Erros cometidos

1. Declarei o sistema "no ar" antes de validar a experiencia humana no navegador.
2. Confundi Docker ativo e HTTP 200 com produto utilizavel.
3. Nao detectei cedo que a tela de login podia ficar branca por token antigo/stale session.
4. Demorei a entregar acesso local usavel e a ajustar autenticacao para modo enterprise.
5. Tratei backend, schema e contratos como se bastassem para validar produto premium.
6. Nao auditei cada pagina premium antes de sugerir que o sistema estava pronto.
7. Deixei varias paginas distintas renderizarem o mesmo componente generico `PremiumModulePage`.
8. Aceitei "Notas", "Inbox", "Daily Notes", "Canvas", "MOCs", "Trilhas" e "Agent Builder" como paginas quando eram shells genericos.
9. Nao exigi API client frontend para todos os endpoints backend ja existentes.
10. Nao capturei antes os erros HTTP 500 em rotas administrativas.
11. Nao diagnostiquei cedo o erro de excesso de conexoes PostgreSQL no Docker local.
12. Nao usei subagentes cedo o suficiente, apesar da solicitacao explicita.
13. Nao mantive um ledger simples de "erros a nao repetir" durante a execucao longa.
14. Nao tratei riscos residuais com energia suficiente antes de tentar avancar fase.
15. Confundi comunicacao de "ambiente local subiu" com "produto esta pronto para uso".

## Evidencias confirmadas por subagentes

Auditoria visual read-only:

- funcional de ponta a ponta: 0 paginas;
- parcial: 7 paginas;
- placeholder: 17 paginas;
- quebrada visual: 0 paginas, mas com erros 500;
- causa relevante dos 500: `asyncpg.exceptions.TooManyConnectionsError`.

Auditoria frontend/read-only:

- `grupos`, `subgrupos`, `notas`, `inbox`, `daily`, `canvas`, `mocs`, `trilhas`, `agent-builder`, `modelos`, `custos`, `jobs`, `juridico`, `relatorios`, `auditoria` e `aprovacoes` caem em `PremiumModulePage`;
- `PremiumModulePage` nao implementa editor Markdown, wikilinks, backlinks, inbox real, daily note, canvas visual, MOCs, trilhas ou builder por chat;
- `lightrag_webui/src/api/lightrag.ts` nao expoe clients para grande parte das rotas Obsidian-like ja existentes no backend.

Auditoria backend/contratos:

- backend tem endpoints reais para notas, tags, backlinks, provenance, inbox, daily notes, canvas, content maps, trilhas e agent builder;
- o problema principal desta frente e a ponte frontend/API client/UI, nao ausencia total de contrato backend.

## Status real das paginas

| Pagina | Status real | Problema principal |
|---|---|---|
| Dashboard | parcial | metricas e cards podem falhar por 500 |
| Workspaces | parcial | sem prova de criacao/edicao real |
| Grupos | placeholder | sem CRUD/workflow proprio |
| Subgrupos | placeholder | sem CRUD/workflow proprio |
| Documentos | parcial | upload/listagem nao provados ponta a ponta |
| Notas | placeholder | sem editor Markdown, wikilinks, tags ou backlinks |
| Inbox | placeholder | sem triagem, status, prioridade ou conversao |
| Daily Notes | placeholder | sem daily note automatica ou resumo do dia |
| Canvas | placeholder | sem board visual, nodes, edges ou drag/drop |
| MOCs | placeholder | sem mapa de conteudo navegavel |
| Trilhas | placeholder | sem construtor de trilha e passos |
| Grafo | parcial | dados invalidos podem quebrar confianca |
| Chat | parcial | query real e contexto operacional incompletos |
| Agent Builder | placeholder | sem builder por chat, sessoes ou publish review |
| Assistentes | parcial | sem validacao completa de criacao/uso |
| Modelos | placeholder | 500 em admin/models |
| Custos | placeholder | sem ledger navegavel enterprise |
| Jobs | placeholder | sem job manager real |
| Juridico | placeholder | sem revisao humana e fluxo processual real |
| Relatorios | placeholder | sem builder/export governado |
| Atividade | parcial | timeline simples |
| Auditoria | placeholder | sem filtros enterprise |
| Aprovacoes | placeholder | sem fluxo real de aprovacao |
| Admin | parcial | admin parcialmente usavel, com 500 em endpoints |

## Correcoes ja aplicadas

- Dockerfile passou a incluir `lightrag_enterprise` na imagem.
- Docker local foi colocado em modo de autenticacao enterprise.
- Usuario MASTER local foi bootstrapado sem registrar segredo no git.
- Login passou a limpar token stale ao abrir `/login`, evitando loop/tela branca.
- Build frontend, testes Bun, TypeScript e lint passaram apos a correcao de login.

## Bloqueios atuais

1. Paginas premium ainda nao sao enterprise interativas.
2. `PremiumModulePage` mascara modulos diferentes como se fossem entregas reais.
3. Faltam clients TypeScript para notas, inbox, daily notes, canvas, MOCs, trilhas e agent builder.
4. Docker local ainda precisa correcao de pool/conexoes para eliminar `TooManyConnectionsError`.
5. Faltam testes visuais que falhem quando paginas diferentes renderizam o mesmo shell generico.
6. Faltam fluxos de ponta a ponta com evidencia para cada modulo premium.

## Checklist obrigatorio antes de falar "pronto"

- [ ] `git status --short` revisado.
- [ ] Sem secrets, `.env`, tokens ou senhas em logs/docs/commits.
- [ ] Docker local sobe sem erro critico nos logs.
- [ ] Login testado visualmente no navegador.
- [ ] Cada pagina do menu foi aberta no navegador.
- [ ] Cada pagina tem componente proprio ou justificativa explicita para compartilhar componente.
- [ ] Nenhuma pagina premium critica e apenas placeholder.
- [ ] Console do navegador sem erro bloqueante.
- [ ] Network sem HTTP 500 em fluxo normal.
- [ ] API client frontend existe para os endpoints usados pela pagina.
- [ ] Fluxo principal da pagina foi executado, nao apenas renderizado.
- [ ] Teste automatizado cobre o bug corrigido.
- [ ] Teste visual ou Playwright cobre as paginas premium principais.
- [ ] Riscos residuais foram corrigidos ou viraram plano datado e pequeno.
- [ ] Nao declarar READY se algum gate acima falhar.

## Proxima menor fatia correta

1. Corrigir `TooManyConnectionsError` no backend/Postgres Docker local.
2. Criar API clients TypeScript para:
- notas;
- inbox;
- daily notes;
- canvas;
- MOCs/content maps;
- trilhas;
- agent builder sessions.
3. Substituir `PremiumModulePage` por telas reais, nesta ordem:
- `NotesPage`;
- `InboxPage`;
- `DailyNotesPage`;
- `CanvasPage`;
- `MocsPage`;
- `TrailsPage`;
- `AgentBuilderPage`.
4. Adicionar teste que falha se essas paginas renderizarem o mesmo shell generico.
5. Revalidar Docker local com smoke visual e network sem 500.

## Regra permanente de comunicacao

Quando algo estiver apenas parcial, dizer "parcial".
Quando for placeholder, dizer "placeholder".
Quando estiver no ar mas nao pronto, dizer "local subiu, produto ainda nao esta pronto".
Nunca maquiar contrato backend como experiencia premium entregue.
Loading