feat: implement agentic DuckDuckGo web-search fallback#516
Open
tejask011 wants to merge 6 commits into
Open
Conversation
Adds fallback logic to the RAG service when local FAISS context fails, injecting duckduckgo_search results as custom context. Updates the React UI to display web source citations with proper external links.
|
@tejask011 is attempting to deploy a commit to the firefistisdead's projects Team on Vercel. A member of the Team first needs to authorize it. |
Contributor
|
Warning Review limit reached
More reviews will be available in 43 minutes and 57 seconds. Learn how PR review limits work. Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file). ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. 📝 WalkthroughWalkthroughAdds a DuckDuckGo web-search fallback used when RAG retrieval is insufficient, updates /ask and /ask/stream to attempt the fallback, enriches citation metadata with ChangesWeb Search Fallback for Insufficient Evidence
sequenceDiagram
participant Client
participant RAGService
participant perform_web_search
participant DuckDuckGo
participant SessionStore
Client->>RAGService: POST /ask (query)
RAGService->>RAGService: run evidence gate on retrieved docs
alt evidence insufficient
RAGService->>perform_web_search: perform_web_search(query)
perform_web_search->>DuckDuckGo: DuckDuckGo search
DuckDuckGo-->>perform_web_search: results (url, title, snippet)
perform_web_search-->>RAGService: Documents with metadata {type, url}
RAGService->>RAGService: replace docs, set intent="overview"
RAGService-->>Client: overview response with web citations
else no web results
RAGService->>SessionStore: append refusal, mark session dirty
RAGService-->>Client: insufficient-context refusal (or streamed SSE)
end
🎯 3 (Moderate) | ⏱️ ~25 minutes
🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
github-actions
Bot
added
backend
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@frontend/src/components/ChatPanel/MessageBubble.jsx`:
- Around line 366-369: The onClick handler in MessageBubble.jsx opens
third-party URLs directly (window.open(source.url, '_blank')) without validating
the URL or preventing tabnabbing; update the handler (the branch using
isWebSource and source.url) to first parse and validate source.url (e.g., try
new URL(source.url) and ensure protocol is http: or https:, or prepend https://
if appropriate), then open it using window.open with noopener/noreferrer (e.g.,
window.open(validUrl, '_blank', 'noopener,noreferrer')), and as a fallback set
the returned window's opener to null (const w = window.open(...); if (w)
w.opener = null) to ensure window.opener is not exposed.
In `@rag-service/main.py`:
- Around line 3479-3484: The code replaces PDF docs with web_docs in the
evidence fallback (see logger.info, perform_web_search, docs, intent) but
doesn't propagate retrieval origin; update the flow to set and carry a
retrieval_origin flag (e.g., retrieval_origin = "web" or "pdf") whenever you
assign docs = web_docs and when original PDF retrieval succeeds, and thread that
flag through the answer-building and prompt-construction functions (including
the streaming prompt path and any functions that generate headers or extractive
intro text) so they switch wording/instructions when any doc.metadata.type ==
"web" (or retrieval_origin == "web"); ensure code paths that synthesize overview
vs PDF-specific language read the flag and produce web-appropriate provenance
statements.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 890edd76-693e-4451-aeac-9824130bba60
📒 Files selected for processing (3)
frontend/src/components/ChatPanel/MessageBubble.jsxrag-service/main.pyrag-service/requirements.txt
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
FireFistisDead
added
level:intermediate
quality:clean
gssoc:approved
and removed
level:intermediate
level:advanced
labels
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| "Evidence gate refused answer session_id=%s intent=%s best_score=%s retrieved_chunks=%s", | ||
| session_id, | ||
| intent, | ||
| best_score, |
| "Stream evidence gate refused session_id=%s intent=%s best_score=%s", | ||
| session_id, | ||
| intent, | ||
| best_score, |
Owner
|
@tejask011 check the tests as they are not passi |
Author
please review it now |
Owner
|
@tejask011 there are merge conflicts |
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
rag-service/main.py (1)
3410-3416:⚠️ Potential issue | 🔴 Critical | ⚡ Quick winRoute handlers are shadowed, so
/askand/summarizenever execute their full logic.Both decorated handlers currently stop after
cleanup_expired_sessions(). The later same-nameddefblocks are not the registered FastAPI endpoints, so these routes returnnullinstead of running retrieval/generation logic.✅ Minimal patch
`@app.post`("/ask") def ask_question(data: Question, _ready: None = Depends(require_models_ready)): cleanup_expired_sessions() - - -def ask_question(data: Question): question = (data.question or "").strip() ...`@app.post`("/summarize") def summarize_pdf(data: SummarizeRequest, _ready: None = Depends(require_models_ready)): cleanup_expired_sessions() -def summarize_pdf(data: SummarizeRequest): session_id = str(data.session_id) ...Also applies to: 4211-4215
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@rag-service/main.py` around lines 3410 - 3416, The FastAPI route handlers are shadowed by duplicate function definitions (e.g., the decorated `@app.post`("/ask") def ask_question(...) currently only calls cleanup_expired_sessions() while a later def ask_question(data: Question): contains the real logic); remove the duplicate/unregistered function and move the full handler logic into the decorated function (keep the dependency require_models_ready in the signature and call cleanup_expired_sessions() before running the question processing), and apply the same fix for the /summarize handler so the registered endpoints execute the retrieval/generation code instead of returning null.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@rag-service/main.py`:
- Around line 3988-3991: The merged list keeps per-session local rank values
which distort cross-session ranking used by diversify_retrieved_documents();
after sorting all_scored_candidates and slicing scored_candidates, recompute
global ranks by iterating scored_candidates in order and updating each
candidate's rank to its new global position (e.g., for i, cand in
enumerate(scored_candidates): set cand.rank = i+1 or rebuild the tuple as
(cand_doc, cand_score, i+1) if candidates are tuples) so
diversify_retrieved_documents() sees correct global ranks when applying
thresholds.
- Around line 3895-3907: The loop over session_id_list (for idx, session_id in
enumerate(session_id_list)) builds all_scored_candidates and
all_indexed_documents but later persistence uses a single session_id, so history
updates and dirty flags are only written for one session; fix by performing
per-session persistence: either move the refusal/success persistence logic into
the loop so each iteration uses the current session_id and secret_list[idx] (or
None) when appending chat history and marking dirty, and compute a
session-scoped cache_key (e.g., include session_id) before using it;
alternatively, keep aggregation but then after the loop iterate over
session_id_list and persist the corresponding subset of results for each
session_id (mapping candidates/documents back to each session), ensuring you
reference session_id_list, secret_list, the for idx, session_id loop,
all_scored_candidates, all_indexed_documents, and cache_key when making the
per-session writes.
---
Outside diff comments:
In `@rag-service/main.py`:
- Around line 3410-3416: The FastAPI route handlers are shadowed by duplicate
function definitions (e.g., the decorated `@app.post`("/ask") def
ask_question(...) currently only calls cleanup_expired_sessions() while a later
def ask_question(data: Question): contains the real logic); remove the
duplicate/unregistered function and move the full handler logic into the
decorated function (keep the dependency require_models_ready in the signature
and call cleanup_expired_sessions() before running the question processing), and
apply the same fix for the /summarize handler so the registered endpoints
execute the retrieval/generation code instead of returning null.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 997f9475-7967-42c1-9fef-16c26836d62d
📒 Files selected for processing (2)
rag-service/main.pyrag-service/requirements.txt
🚧 Files skipped from review as they are similar to previous changes (1)
- rag-service/requirements.txt
Contributor
There was a problem hiding this comment.
Caution
Inline review comments failed to post. This is likely due to GitHub's internal server error or limits when posting large numbers of comments. If you are seeing this consistently it is likely a permissions issue. Please check "Moderation" -> "Code review limits" under your organization settings.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
rag-service/main.py (1)
3410-3416:⚠️ Potential issue | 🔴 Critical | ⚡ Quick winRoute handlers are shadowed, so
/askand/summarizenever execute their full logic.Both decorated handlers currently stop after
cleanup_expired_sessions(). The later same-nameddefblocks are not the registered FastAPI endpoints, so these routes returnnullinstead of running retrieval/generation logic.✅ Minimal patch
`@app.post`("/ask") def ask_question(data: Question, _ready: None = Depends(require_models_ready)): cleanup_expired_sessions() - - -def ask_question(data: Question): question = (data.question or "").strip() ...`@app.post`("/summarize") def summarize_pdf(data: SummarizeRequest, _ready: None = Depends(require_models_ready)): cleanup_expired_sessions() -def summarize_pdf(data: SummarizeRequest): session_id = str(data.session_id) ...Also applies to: 4211-4215
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@rag-service/main.py` around lines 3410 - 3416, The FastAPI route handlers are shadowed by duplicate function definitions (e.g., the decorated `@app.post`("/ask") def ask_question(...) currently only calls cleanup_expired_sessions() while a later def ask_question(data: Question): contains the real logic); remove the duplicate/unregistered function and move the full handler logic into the decorated function (keep the dependency require_models_ready in the signature and call cleanup_expired_sessions() before running the question processing), and apply the same fix for the /summarize handler so the registered endpoints execute the retrieval/generation code instead of returning null.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@rag-service/main.py`:
- Around line 3988-3991: The merged list keeps per-session local rank values
which distort cross-session ranking used by diversify_retrieved_documents();
after sorting all_scored_candidates and slicing scored_candidates, recompute
global ranks by iterating scored_candidates in order and updating each
candidate's rank to its new global position (e.g., for i, cand in
enumerate(scored_candidates): set cand.rank = i+1 or rebuild the tuple as
(cand_doc, cand_score, i+1) if candidates are tuples) so
diversify_retrieved_documents() sees correct global ranks when applying
thresholds.
- Around line 3895-3907: The loop over session_id_list (for idx, session_id in
enumerate(session_id_list)) builds all_scored_candidates and
all_indexed_documents but later persistence uses a single session_id, so history
updates and dirty flags are only written for one session; fix by performing
per-session persistence: either move the refusal/success persistence logic into
the loop so each iteration uses the current session_id and secret_list[idx] (or
None) when appending chat history and marking dirty, and compute a
session-scoped cache_key (e.g., include session_id) before using it;
alternatively, keep aggregation but then after the loop iterate over
session_id_list and persist the corresponding subset of results for each
session_id (mapping candidates/documents back to each session), ensuring you
reference session_id_list, secret_list, the for idx, session_id loop,
all_scored_candidates, all_indexed_documents, and cache_key when making the
per-session writes.
---
Outside diff comments:
In `@rag-service/main.py`:
- Around line 3410-3416: The FastAPI route handlers are shadowed by duplicate
function definitions (e.g., the decorated `@app.post`("/ask") def
ask_question(...) currently only calls cleanup_expired_sessions() while a later
def ask_question(data: Question): contains the real logic); remove the
duplicate/unregistered function and move the full handler logic into the
decorated function (keep the dependency require_models_ready in the signature
and call cleanup_expired_sessions() before running the question processing), and
apply the same fix for the /summarize handler so the registered endpoints
execute the retrieval/generation code instead of returning null.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 997f9475-7967-42c1-9fef-16c26836d62d
📒 Files selected for processing (2)
rag-service/main.pyrag-service/requirements.txt
🚧 Files skipped from review as they are similar to previous changes (1)
- rag-service/requirements.txt
🛑 Comments failed to post (2)
rag-service/main.py (2)
3895-3907:
⚠️ Potential issue | 🟠 Major | 🏗️ Heavy liftMulti-session retrieval writes chat history to only one session.
The loop collects documents from multiple sessions, but downstream refusal/success persistence still appends chat and marks dirty using a single
session_idvalue (the last iterated one). That drops history updates for other queried sessions and creates inconsistent per-session state.💡 Suggested direction
- for idx, session_id in enumerate(session_id_list): + validated_session_ids = [] + for idx, session_id in enumerate(session_id_list): ... _require_session_secret(session, secret) + validated_session_ids.append(session_id) ... - with sessions_lock: - current_session = sessions.get(session_id) - if current_session: - append_chat_exchange(...) - _mark_session_dirty(session_id) + with sessions_lock: + for sid in validated_session_ids: + current_session = sessions.get(sid) + if current_session: + append_chat_exchange(...) + _mark_session_dirty(sid)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@rag-service/main.py` around lines 3895 - 3907, The loop over session_id_list (for idx, session_id in enumerate(session_id_list)) builds all_scored_candidates and all_indexed_documents but later persistence uses a single session_id, so history updates and dirty flags are only written for one session; fix by performing per-session persistence: either move the refusal/success persistence logic into the loop so each iteration uses the current session_id and secret_list[idx] (or None) when appending chat history and marking dirty, and compute a session-scoped cache_key (e.g., include session_id) before using it; alternatively, keep aggregation but then after the loop iterate over session_id_list and persist the corresponding subset of results for each session_id (mapping candidates/documents back to each session), ensuring you reference session_id_list, secret_list, the for idx, session_id loop, all_scored_candidates, all_indexed_documents, and cache_key when making the per-session writes.
3988-3991:
⚠️ Potential issue | 🟠 Major | ⚡ Quick winRecompute global ranks after merging cross-session retrieval results.
After sorting merged candidates by score, the code keeps per-session local
rankvalues.diversify_retrieved_documents()uses rank thresholds, so local ranks from different sessions distort relevance selection.🔧 Minimal fix
all_scored_candidates.sort(key=lambda x: x[1]) - scored_candidates = all_scored_candidates[:ASK_RETRIEVAL_CANDIDATES] + scored_candidates = [ + (doc, score, global_rank) + for global_rank, (doc, score, _local_rank) in enumerate( + all_scored_candidates[:ASK_RETRIEVAL_CANDIDATES] + ) + ] indexed_documents = all_indexed_documents🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@rag-service/main.py` around lines 3988 - 3991, The merged list keeps per-session local rank values which distort cross-session ranking used by diversify_retrieved_documents(); after sorting all_scored_candidates and slicing scored_candidates, recompute global ranks by iterating scored_candidates in order and updating each candidate's rank to its new global position (e.g., for i, cand in enumerate(scored_candidates): set cand.rank = i+1 or rebuild the tuple as (cand_doc, cand_score, i+1) if candidates are tuples) so diversify_retrieved_documents() sees correct global ranks when applying thresholds.
Author
|
check it now see i made changes |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds fallback logic to the RAG service when local FAISS context fails, injecting DuckDuckGo search results as additional context. Updates the React UI to display web source citations with proper external links and visual differentiation.
Summary
This PR addresses the "insufficient context" limitation in the RAG pipeline by introducing an Agentic Web-Search Fallback.
Changes Made
Backend (
rag-service/main.py)Added
duckduckgo-searchas a dependency.Implemented fallback logic that triggers when
passes_evidence_gate()fails.Executes a DuckDuckGo search using the user's query when retrieved document context is insufficient.
Injects top web search results into the LLM prompt as supplemental context.
Opens external URLs in a new browser tab when a web citation is clicked.
Related issue
Closes #515
Testing
Test Scenarios
Checklist:
Notes
Deployment Requirements
The deployment environment must have internet access for DuckDuckGo search requests.
requirements.txthas been updated with:duckduckgo-search>=5.0.0Run:
during deployment to install the new dependency.
Security
labels
GSSOC
Summary by CodeRabbit
New Features
Style