Read document status from a local snapshot (fix is_* N+1) — SDK 1.2.6#411
Merged
Conversation
…calls
Document.status / is_processing / is_ingested / is_failed / error now read the status
already carried on the document (system_metadata) instead of calling get_document_status on
every access, which caused an N+1 when iterating documents. status now also returns `as_of`
and `source` ("local"/"not_loaded"); when status was not fetched (e.g. projected away),
is_* return False and make no network call.
- Add Document.refresh() for an explicit live re-fetch; docstrings point to refresh()/
wait_for_completion() for the current status.
- Add a cheap `status` projection: list_documents(fields=[..., "status"]) reads
system_metadata->>'status' via JSON-path (no document text), so is_* resolve locally.
- Bump SDK to 1.2.6; add SDK + server projection tests incl. a call-counting regression guard.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes an N+1 in the SDK: reading
Document.is_failed/is_processing/is_ingested/status/errormade aget_document_statusAPI call on every access, so building a list of dicts over N documents fired ~3N extra requests — erasing the speed win from field projection.Now these read the status the document already carries (
system_metadata) — zero network calls. Measured against a live server: 0 extra calls for 50 docs (was ~150).Changes
SDK (
models.py)status/is_*/errorread localsystem_metadatainstead of calling the server.statusreturnsas_of(when the snapshot was pulled) andsource("local"/"not_loaded"). If status wasn't fetched (e.g. projected away),is_*returnFalseand make no call.Document.refresh()for an explicit live re-fetch; docstrings point torefresh()/wait_for_completion()for the current status.Server (
postgres_database.py,routes/utils.py)status(anderror/timestamps) is now a cheap projectable field:list_documents(fields=[..., "status"])readssystem_metadata->>'status'via JSON-path (no document text) and returns it in a slimsystem_metadata, sois_*resolve locally with zero extra calls.Release: bump SDK to 1.2.6 (published to PyPI), CHANGELOG updated.
Behavior note
status/is_*now reflect the document as fetched (a snapshot), not a live value. Polling viawhile doc.is_processing:should usedoc.wait_for_completion()(unchanged — it re-fetches) or the newdoc.refresh().Tests
test_document_status.pyincl. a call-counting regression guard assertingis_*make zero client calls (the gap that let this bug through).test_document_projection.py—statusresolves to a JSON-path (not the full blob), slimsystem_metadatareassembly, app-layer projection keeps it.wait_for_completionverified unaffected.🤖 Generated with Claude Code