How to use this template: Complete each section after you've built and tested the corresponding part of your system. Do not write placeholder text — if a section isn't done yet, leave it blank and come back. Every section below is required for submission. One-liners will not receive full credit.
This system covers student reviews of CS 367 (Computer Systems and Programming) professors at George Mason University. CS 367 is one of the most demanding required courses in the CS curriculum, covering systems programming, memory management, assembly, and shell implementation. Students consistently seek peer advice before choosing a professor, but official GMU channels — course catalogs, department pages, and syllabi — provide no information about teaching style, exam difficulty, project workload, or how much supplemental support a professor provides.
Rate My Professors and Coursicle contain this knowledge, but it is buried across dozens of reviews per professor and is not searchable in natural language. A student asking "which professor gives the most useful feedback on projects?" has no efficient way to find the answer. This RAG system makes that collective student knowledge directly queryable, grounded, and cited.
| # | Source | Type | URL or file path |
|---|---|---|---|
| 1 | Coursicle — Yutao Zhong (CS 367) | Scraped student reviews | https://www.coursicle.com/gmu/?professor=Yutao+Zhong&type=reviews |
| 2 | Rate My Professors — Yutao Zhong | Dated student reviews | https://www.ratemyprofessors.com/professor/1100291 |
| 3 | Coursicle — Kevin Andrea (CS 367) | Scraped student reviews | https://www.coursicle.com/gmu/?professor=Kevin+Andrea&type=reviews |
| 4 | Rate My Professors — Kevin Andrea | Dated student reviews | https://www.ratemyprofessors.com/professor/kevin-andrea |
| 5 | Coursicle — Prof. Mughal (CS 367) | Scraped student reviews | https://www.coursicle.com/gmu/?professor=Mughal&type=reviews |
| 6 | Rate My Professors — Prof. Mughal | Dated student reviews | https://www.ratemyprofessors.com/professor/mughal-gmu |
| 7 | Coursicle — Hamza TA (CS 367) | TA recitation reviews | https://www.coursicle.com/gmu/ |
| 8 | Coursicle — John Otten (CS 262) | Prereq course reviews for context | https://www.coursicle.com/gmu/?professor=John+Otten&type=reviews |
| 9 | Combined CS 367 reviews (all professors) | Cleaned and merged local file | documents/cs367_reviews.txt |
| 10 | RMP dated reviews — CS 367 all professors | Reviews with explicit submission dates | documents/cs367_reviews.txt |
Chunk size: One review per chunk — no fixed character limit. Each student review is treated as one self-contained chunk, split on the --- delimiter that separates reviews in the collected text files.
Overlap: None. Overlap is used to preserve context across arbitrary character cuts. Since we are not cutting arbitrarily — we are splitting at natural review boundaries — overlap adds no value and would only introduce noise by duplicating content across chunks.
Why these choices fit your documents: The source documents are structured collections of short student reviews, each expressing a single student's complete opinion about a professor. Splitting mid-review would destroy semantic coherence — a sentence like "exams were very fair" means nothing without the surrounding context identifying the professor and course. By making each review its own chunk, every embedding captures a focused, attributable opinion. This approach also eliminates the MemoryError caused by the initial sliding-window character chunking approach on the 66,000-character combined file, which produced an infinite loop when the overlap size exceeded the boundary snap distance.
Final chunk count: Approximately 200–250 individual review chunks across all documents (exact count printed at ingestion time).
Sample chunks:
--- Chunk 1 (source: cs367_reviews.txt) ---
CS 367
3y
Sophomore
CS
She gives great lectures, but i do prefer Andreas videos because they seemed
to be aligned with the quizzes more. Shes also very nice if you come to her
office hours or stick around after class. Midterm and final were easy, just do
the practice ones. Start projects early, esp p1 (floats) and p3 (shell; this
one will take VERY LONG, due near finals)
--- Chunk 2 (source: cs367_reviews.txt) ---
Date: Dec 22nd, 2025
Course: CS367
Quality: N/A
Comment: Class with professor zhong was great this semester. Grading criteria
and the class schedule are cleanly laid out at the beginning of the semester.
Attendance is answering questions via PollEV, and TBH it was not so bad at all.
There are lots of resources to complete projects. The tests were great, zhong
provides a very comprehensive study guide.
--- Chunk 3 (source: cs367_reviews.txt) ---
CS 367
3y
Junior
CS
Kevin Andrea is extremely caring and wants you to learn. He will spend hours
making videos so that you can understand the material. After the midterm, he
took 10 mins to ask how we are doing in the class & give any feedback to
improve him. He starts online lectures with so much enthusiasm that makes me
want to participate. The real GOAT @ GMU
--- Chunk 4 (source: cs367_reviews.txt) ---
Date: May 14th, 2025
Course: CS367
Quality: N/A
Comment: Professor Mughal is a great teacher if you participate in the class
and go to office hours. His lectures can be a bit difficult to understand if
you do not keep up, but the important information is usually presented towards
the end half of the class. The projects and exams are mid-medhigh difficulty
so take time to do those. Weekly online quizzes.
--- Chunk 5 (source: cs367_reviews.txt) ---
CS 367
2y
Sophomore
CS
He is waay better than the ratings. This course is really easy exams, cuz he
takes his exams based of andrea's section. His lectures are a little boring,
but that is due to the horrible slides. Projects are mid diff, just start early
and your fine. That was one of the best classes i've ever took, in terms of
how well i did in it. I even UTA it now
Model used: all-MiniLM-L6-v2 via the sentence-transformers library, running entirely locally with no API key or rate limits.
Production tradeoff reflection: all-MiniLM-L6-v2 was chosen for this project because it runs locally, requires no credentials, and is fast enough to embed hundreds of short reviews in seconds. Its 256-token context window is well-matched to individual student reviews, which rarely exceed 150 tokens.
For a production deployment, I would weigh several tradeoffs. First, context length: if the system were extended to include longer documents like course syllabi or housing guides, a model like OpenAI's text-embedding-3-large (8,191 token limit) would prevent truncation. Second, domain-specific accuracy: student reviews use informal language ("goated," "mid diff," "cooked") that general-purpose models were not trained on — a model fine-tuned on social or educational review text might produce better embeddings for this vocabulary. Third, multilingual support: if the system were expanded to serve students who write reviews in other languages, paraphrase-multilingual-MiniLM-L12-v2 would be necessary. Fourth, latency and scale: local models add no network latency but cannot scale horizontally without additional infrastructure; API-hosted models (Cohere Embed, OpenAI) handle scaling automatically but introduce cost and network dependency.
System prompt grounding instruction:
The following system prompt is passed to llama-3.3-70b-versatile via the Groq API on every query:
You are a helpful assistant for GMU students researching CS 367
(Computer Systems and Programming) professors and course experiences.
CRITICAL RULES — follow these exactly:
1. Answer ONLY using information from the provided student review excerpts below.
2. Do NOT use your general training knowledge about professors, courses, or GMU.
3. If the provided excerpts do not contain enough information to answer the
question, respond with exactly: "I don't have enough information in the
provided reviews to answer that question."
4. Always cite which source document(s) your answer draws from.
5. Be specific and quote or paraphrase student language where helpful.
6. Keep answers concise (3–6 sentences) unless the question requires more detail.
The retrieved chunks are formatted as numbered excerpts with their source filename embedded in a header ([Excerpt 1 — Source: cs367_reviews.txt]) before being injected into the user message. This ensures the model sees both the content and the attribution label for every chunk it reasons over.
How source attribution is surfaced in the response: Source filenames are collected from the metadata of every retrieved chunk and appended programmatically to every response by generation.py — the LLM does not control whether attribution appears. The Streamlit interface displays sources in a dedicated "Retrieved from" panel below the answer, and the raw chunks are available in an expandable section showing distance scores.
| # | Question | Expected answer | System response (summarized) | Retrieval quality | Response accuracy |
|---|---|---|---|---|---|
| 1 | What do students say about Professor Zhong's exams? | Exams are fair, similar to practice problems and study guides | Correctly cited multiple reviews saying exams mirror the study guide and practice problems; mentioned PollEV attendance | Relevant | Accurate |
| 2 | Which CS 367 professor is most recommended for students who struggle with the material? | Kevin Andrea, due to his video lectures and supplemental resources | Correctly identified Andrea, cited reviews praising his videos, enthusiasm, and accessibility | Relevant | Accurate |
| 3 | How hard are the projects in CS 367? | Difficult and time-consuming; start early, especially P3 (shell) | Returned correct advice about starting early; cited specific project warnings (P1 floats, P3 shell) | Relevant | Accurate |
| 4 | What is Professor Mughal's teaching style like? | Decent but dry lectures; fair exams; weekly quizzes; project-based grading | Returned partially correct answer — captured "dry lectures" and "weekly quizzes" but missed the curve detail on exams | Partially relevant | Partially accurate |
| 5 | Is CS 367 hard regardless of which professor you take? | Yes — all reviews describe it as inherently difficult | System retrieved reviews that mentioned difficulty but also mixed in professor-praise chunks; answer was hedged rather than direct | Partially relevant | Partially accurate |
Retrieval quality: Relevant / Partially relevant / Off-target Response accuracy: Accurate / Partially accurate / Inaccurate
Question that failed: "Is CS 367 hard regardless of which professor you take?"
What the system returned: The system returned a hedged answer that focused more on how good Kevin Andrea makes the course feel rather than confirming the underlying difficulty is consistent across sections. It pulled several Andrea-praising reviews ("CS367 can be seen as difficult, but with the right professor it is easy") which, while technically relevant, led the model to emphasize the professor's impact rather than the universal difficulty of the course material.
Root cause (tied to a specific pipeline stage): This is a retrieval-stage failure. The query contains the word "hard" and "professor," which semantically overlap heavily with reviews that praise professors for making a hard course manageable. The top-5 retrieved chunks happened to be Andrea reviews that frame difficulty as something the professor mitigates — the model had no chunks from the document that state difficulty is inherent regardless of instructor. The embedding model could not distinguish between "the course is hard" and "the course is hard but Andrea makes it easy" because both contain similar vocabulary.
What you would change to fix it: Increase top_k from 5 to 8–10 for queries that ask comparative or universal questions, so that reviews from multiple professors are more likely to appear in the retrieved set. Alternatively, add a metadata filter that requires retrieved chunks to come from at least two different professor sources when the query does not mention a specific professor name.
One way the spec helped you during implementation: The planning.md requirement to write a chunking strategy before touching any code forced an early decision about document structure. Because I described the documents as "short reviews separated by --- delimiters" in the spec, it was immediately clear when implementing ingestion that character-based chunking was the wrong approach — the spec gave me a reference point to evaluate the generated code against and catch the mismatch before it caused the MemoryError in production.
One way your implementation diverged from the spec, and why: The spec originally called for character-based chunking with a size of 400 characters and 80-character overlap, following the project template's default suggestion. During implementation this caused a MemoryError on the 66,000-character combined review file because the overlap logic created an infinite loop when boundary snapping caused insufficient forward progress. The implementation was changed to review-based chunking (splitting on ---) with no overlap, which is actually a better fit for the document structure — each review is semantically complete on its own. The spec was updated to reflect this change.
Instance 1
-
What I gave the AI: The structure of the
.txtreview files (showing the---delimiter format, VM artifact prefixes, "Reply" lines, and vote count lines), and a request to implementload_documents(),clean_text(), andchunk_text()functions for an ingestion pipeline targeting ChromaDB withsentence-transformers. -
What it produced: A complete
ingestion.pywith regex-based cleaning, a sliding-window character chunker (400 chars, 80 overlap), and ChromaDB upsert logic with source metadata. -
What I changed or overrode: The character-based
chunk_text()function caused aMemoryErroron the full document. I directed Claude to rewrite the chunking strategy to split on---delimiters instead, making each review its own chunk with no overlap. I also removed theCHUNK_SIZEandCHUNK_OVERLAPconfig constants since they were no longer relevant. Instance 2 -
What I gave the AI: The grounding requirement (answers only from retrieved chunks, no training knowledge, source attribution in every response), the Groq model name (
llama-3.3-70b-versatile), and a description of the desired output format (answer + list of source filenames). -
What it produced: A
generation.pywith a system prompt, context formatting function, and Groq API call, plus a Streamlitapp.pywith a text input, answer panel, sources panel, and expandable chunk viewer. -
What I changed or overrode: The initial system prompt used soft language ("try to answer only from the documents"). I directed Claude to replace it with explicit numbered rules including a specific fallback phrase for out-of-scope questions ("I don't have enough information in the provided reviews to answer that question") and a rule prohibiting use of training knowledge. I also added the suggested-question buttons to the Streamlit UI, which were not in the original generated code.