MVP end-to-end validation

[lucaosti]

Context

The MVP has never been validated end to end as a product. There is good automated coverage (~388 backend test functions plus frontend and QVAC suites), but automated tests do not tell us whether a student can actually complete a study session on a real course with real materials.

This issue covers manual end-to-end validation of the MVP. Automated coverage gaps are tracked separately.

Scope

Run against the integrated MVP (after the mvp-testing integration), on a clean environment, using real course material — not fixtures.

Test plan

Setup

• Clean checkout, follow the README Docker quick start exactly; note every place the instructions are wrong or incomplete
• Repeat with the manual (non-Docker) path
• Verify first-run model download completes and the service recovers if interrupted
• Verify behaviour with QVAC_LLM_ENABLED=false (retrieval-only mode)

Account and access

• Signup, login, logout, session expiry, refresh
• Account lockout after repeated failed logins
• A student cannot reach another user's courses

Ingestion

• Upload a real lecture PDF, a real PPTX deck, a DOCX
• Multi-file upload: progress, per-file status, one failure does not block the others
• Retry a failed document; delete a document
• Large file (100+ pages): does it complete, how long, does the UI stay honest
• Corrupt/encrypted PDF: is the error actionable
• Verify parsed output and chunks in the debug view are faithful to the source

Study actions

For each of explain, summarize, retrieve, open_questions, quiz, oral, derive, compare:

• Output is grounded in the uploaded material
• Citations resolve to the correct page/slide
• Streaming renders progressively and completes
• Behaviour on a question the material does not answer — does it say so, or invent
• Behaviour with no documents in the course

Course builder

• Generate an outline from uploaded documents; is the topic grouping sensible
• Generate lesson content; check the groundedness judge flags genuinely unfaithful content
• Review, edit and publish flow
• Generated quizzes are answerable from the lesson content

Assessment

• Take a lesson quiz and a chapter test; scoring and passing threshold correct
• Progress and badges update; certificate issued on completion and verifiable by code

Robustness

• Kill the QVAC service mid-request; kill Redis; stop the ARQ worker mid-ingestion
• Two courses do not leak content into each other's retrieval
• Mobile viewport

Deliverable

A findings document with one row per defect: what was done, what happened, what should have happened, severity. Every P0/P1 gets its own issue linked back here.

Acceptance criteria

• Every checklist item executed and recorded, including passes
• All P0 defects fixed or explicitly accepted before release
• README corrected wherever setup did not work as written

[lucaosti]

This is now the gate on #170.

The integration pull request is deliberately a draft waiting on this issue: CI proves the code builds, type-checks and passes 1,136 automated tests, but none of that answers whether a student can actually complete a study session end to end.

Specifically untested by CI, and worth prioritising in the checklist above:

• Upload → outline → lesson generation → study → assessment → certificate, on real course material rather than fixtures
• The groundedness judge actually rejecting unfaithful lesson content
• Streaming study actions rendering progressively through Caddy, not just directly
• First-run behaviour: model download (~3 GB), and QVAC_LLM_ENABLED=false retrieval-only mode
• The container stack starting from a clean clone — the compose and Dockerfile fixes are verified by CI builds only, since there is no Docker on the development machine

Findings from this pass should land as their own issues linked here, and any P0 blocks #170.