Entity caching clean/22 benchmarks#1539
Draft
jensneuse wants to merge 23 commits into
Draft
Conversation
Pin github.com/wundergraph/astjson to commit 0d295948 (branch feat/two-pass-parser) in both the v2 and execution modules. This commit provides the StructuralCopy / StructuralCopyWithTransform / Transform / two-pass-parser primitives the entity-caching foundation depends on, and drops the 'changed bool' return from MergeValues / MergeValuesWithPath. Adapt all 9 MergeValues / MergeValuesWithPath call sites to the new (*Value, error) signature. No logic or behavior change. Real packages build (go build ./pkg/...) and vet clean; the pre-existing v2/doc.go package-main quirk is unrelated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Introduce the minimal, declarations-only foundation for entity caching: the router-facing LoaderCache/CacheEntry L2 backend contract, the engine-internal CacheKeyTemplate/CacheKey seam, EntityCacheInvalidationConfig, and the per-fetch FetchCacheConfiguration skeleton (resolve/cache.go). Nothing implements or calls these yet; loader and resolvable are untouched. Also lands the full re-implementation spec package under docs/entity-caching/: the architecture spec and integration seam, the directive inventory, one spec + ADR per directive, the stacked-PR plans for graphql-go-tools and the router, the astjson dependency spec, and the test/benchmark plan. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add the alias-aware metadata the entity cache uses to normalize and denormalize a fetch's field shape: Field.OriginalName (schema name behind an alias), Field.CacheArgs (per-field argument metadata for the cache-key arg hash), Object.HasAliases (fast-path flag), the CacheFieldArg type, and the pure ComputeHasAliases(*Object) tree walk. All new fields are json:",omitempty" and default to zero, so existing JSON snapshots are unchanged; Object.Copy/Field.Copy propagate them. Nothing populates or reads these yet (the planner populates them and the copy helpers consume them in later PRs). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add the four Loader copy helpers that isolate a cached value from the live response tree while renaming the field shape between response aliases and schema names: structuralCopyNormalized / structuralCopyDenormalized (L2, project unlisted fields) and their *Passthrough variants (L1, keep unlisted fields). The buildNormalizeTransform / buildDenormalizeTransform builders turn a ProvidesData *Object into an astjson.Transform, recursing through nested objects and arrays of objects. New file only; nothing calls the helpers yet (the loader wires them in a later PR). Round-trip, projection, passthrough, nested, and array cases are covered with exact full-JSON assertions, race-clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…etadata Add the declarative per-subgraph cache configuration carried in FederationMetaData: EntityCacheConfiguration, RootFieldCacheConfiguration (+EntityKeyMapping/FieldMapping), MutationFieldCacheConfiguration, MutationCacheInvalidationConfiguration, and SubscriptionEntityPopulationConfiguration, each with a typed collection and a lookup method. SubscriptionEntityPopulationConfigurations.FindByTypeAndFieldName matches on BOTH TypeName and FieldName and misses when either is empty, preventing silent no-op subscription cache wiring. All new fields are json:",omitempty" so existing plan/federation snapshots are unchanged. Nothing populates or reads these yet. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add the per-request entity-caching toggles on Context.ExecutionOptions.Caching: EnableL1Cache, EnableL2Cache, EnableCacheAnalytics, an optional L2CacheKeyInterceptor (with L2CacheKeyInterceptorInfo), and GlobalCacheKeyPrefix. All default off/empty, so a request with no caching config behaves identically to today. No code reads these flags yet (the loader wires them in a later PR). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Implement the two CacheKeyTemplate implementations. EntityQueryCacheKeyTemplate renders {"__typename":T,"key":{...}} from @key fields (composite/nested/array, int->string coercion, __typename fallback, omitting any item missing a key field). RootQueryCacheKeyTemplate renders {"__typename":"Query","field":F,"args":{...}} with alphabetically-sorted args (recursively sorted nested objects) and can derive entity-shaped keys via EntityKeyMappings (variable paths, array-index, batch list rendering with BatchIndex, per-mapping missing-skip) so a root query and an _entities fetch share one L2 entry byte-for-byte.
Adds KeyField/ParseKeyFields, RootField/RootFieldArgument, EntityKeyMappingConfig/EntityFieldMappingConfig, and a CacheKey.BatchIndex field. Templates are constructed only by tests for now; the planner attaches them in a later PR. Exhaustive table-driven tests assert exact key bytes for every shape.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…hes) First behavior PR: when EnableL1Cache/EnableL2Cache are set, single (non-batch) entity and root fetches read from and write to the per-request L1 map and the external L2 backend, through a small hook seam. The loader's fetch-tree walker, four-phase machinery, and mergeResult shape are untouched (loader.go diff is +36 additive lines). Caching enters via *Loader hooks in a new loader_cache.go (prepareCacheKeys, tryL1CacheLoad, tryL2CacheLoad, populateL1Cache, updateL2Cache, mergeCacheResult, populateCacheAfterMerge) plus additive fields on result, Loader, ResolverOptions, and a Cache *FetchCacheConfiguration on SingleFetch/EntityFetch. Reads StructuralCopy out before merge; writes StructuralCopy in; L1 merge-into-existing uses working-copy-and-swap; L1 reads enforce field-widening. Both flags default off, so existing behavior is byte-identical (full resolve suite passes unchanged). Adversarial copy-invariant tests prove the cache survives post-merge mutation of the response tree. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extend entity caching to BatchEntityFetch. With caching on, batched entities are looked up per-item in L1/L2: all-hit skips the source load and splices cached values in; partial-hit (when EnablePartialCacheLoad) refetches only the missed entities by re-rendering a reduced representation list, then splices cached hits with fresh results; all-miss loads normally and populates the cache. Multi-candidate batch items merge correctly. Reuses the PR #7 hook seam; loader.go gains only an additive batch-path branch and a reduced-input renderer. Cached values are StructuralCopy'd out before splicing (the two remaining budgeted copy sites). Off unless flags + a batch entity fetch. Tests cover all-miss->all-hit, partial (exact refetch counts), multi-candidate merge, L2-disabled, and two batch copy-invariant guards; full suite passes unchanged, race-clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When NegativeCacheTTL > 0, an entity that resolves to not-found (its value is JSON null / TypeNull) is stored as a null sentinel and served from cache on subsequent lookups without a subgraph call, expiring per NegativeCacheTTL and overwritten by a real value once the entity exists. A present entity with a nullable field set to null (TypeObject) is not treated as absent and follows the normal positive-cache path. Adds NegativeCacheTTL to FetchCacheConfiguration and NegativeCacheHit to CacheKey; loader changes are additive and gated on negativeCacheEnabled (default off, so existing behavior is byte-identical). Tests cover store/serve, TTL expiry, overwrite-after-expiry, and the nullable-field guard. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a per-request analytics collector recording cache events (L1/L2 reads and writes, negative hits, fetch timings, header impact, cache-op errors), read once via Context.GetCacheStats() which snapshots and releases the pooled collector. The collector is allocated lazily only when EnableCacheAnalytics is set; with it off, GetCacheStats returns an empty snapshot and no collector is allocated — zero overhead, zero behavior change. CacheAnalyticsSnapshot exposes event slices plus derived metrics (L1HitRate, L2HitRate, CachedBytesServed, EventsByEntityType) and dedups one event per (CacheKey, Kind). Record calls are wired into the PR #7-9 loader paths, each a no-op when analytics is off. Shadow/mutation/subscription events are added by their own later PRs. Full resolve suite passes unchanged and race-clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wire entity caching into the planner. When a datasource has entity- or root-field cache config (and DisableEntityCaching is unset), the visitor builds a cachingPlannerState that tracks the per-fetch field shape and, at fetch finalization, attaches a FetchCacheConfiguration (with the PR #3 cache-key template) and ProvidesData. configureFetchCaching decides L2 on/off: entity fetches enable L2 only when the type has config (else key template kept, L2 off); root fetches enable L2 only when all root fields share an identical config. Postprocess preserves Cache when converting SingleFetch to EntityFetch/BatchEntityFetch. Strictly opt-in: v.caching is nil unless caching is configured, so with no config (or DisableEntityCaching) the planner output is byte-identical — every existing plan/datasource/postprocess/resolve test passes unchanged, no snapshot edits. UseL1Cache stays false (PR #12 enables it). Adds DisableEntityCaching and DisableFetchProvidesData config flags. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a FetchTreeProcessor that turns UseL1Cache on for an entity fetch only when a provider/consumer relationship exists for the same entity type: canRead (a prior fetch, or the union of prior providers, supplies all fields the fetch needs) or canWrite (a later fetch could read what it produces). It walks the fetch tree in dependency stages (parallel siblings never provide for each other), dispatches on the concrete EntityFetch/BatchEntityFetch types, and compares normalized cache field names via objectProvidesAllFields containment. The pass operates only on resolve.FetchTreeNode + public fetch types (no plan import), runs last in processFetchTree after concrete-fetch conversion, and is gated by DisableOptimizeL1Cache. It only touches fetches that carry cache config, so with no caching it is a no-op — existing postprocess/plan/resolve tests pass unchanged, race-clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…invalidate Add mutation semantics for entity caching, reached only for mutation operations with config present. Mutations always skip L2 reads (fetch fresh); skip L2 writes unless EnableMutationL2CachePopulation (with an optional TTL override); and after a successful mutation delete the impacted entity keys from L2 (same key transform pipeline), skipping the delete for any key written in the same fetch. Emits MutationEvent analytics. Planner-side configureMutationEntityImpact reads MutationFieldCacheConfig / MutationCacheInvalidationConfig and attaches a MutationEntityImpactConfig to the fetch; resolve-side logic lives in a new mutation_cache.go. Non-mutation paths are unchanged — full resolve/plan suites pass, race-clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Delete L2 entries when a subgraph response carries extensions.cacheInvalidation.keys (a list of {typename, key} entity descriptors). For each, the loader builds the canonical entity cache key (same shape + number coercion as the key template), applies the full L2 key transform pipeline (GlobalCacheKeyPrefix, header hash, interceptor), groups deletes per cache backend, dedups against keys written in the same fetch, and calls Delete. Records a CacheInvalidationEvent in analytics.
populateCacheAfterMerge now returns the written L2 keys so the invalidation pass can skip same-fetch writes. Only fires when the extension is present, L2 is enabled, and configs exist — responses without the extension are byte-identical. Full resolve suite passes, race-clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add shadow mode for staleness measurement. When a fetch's config sets ShadowMode, the loader still performs the L2 read (to obtain the cached value) and the L2 write (to keep the entry warm), but never serves the cached value — it always loads from source and serves fresh. After merge it compares cached vs fresh per entity (xxhash + byte size) and records a ShadowComparisonEvent; the snapshot exposes ShadowFreshnessRate (matched / total). ShadowMode propagates from EntityCacheConfiguration/RootFieldCacheConfiguration through configureFetchCaching to FetchCacheConfiguration. Off unless configured — non-shadow paths are byte-identical. Full resolve/plan suites pass, race-clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add the plan-time metadata for the @requestScoped directive: RequestScopedField {FieldName, TypeName, L1Key} carried in FederationMetaData.RequestScopedFields, with symmetric lookups (RequestScopedFieldsForType, RequestScopedExportsForField returning the field's own L1 key, RequestScopedRequiredFieldsByKey, RequestScopedFieldsByL1Key) and ValidateRequestScopedFields (errors on a missing key, warns when a key appears on only one field in the subgraph). Additive only; no emitter or runtime reads these yet.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Emit a resolve.RequestScopedField per @requestScoped field during fetch cache configuration: root fetches enumerate their root fields via RequestScopedExportsForField; entity fetches enumerate RequestScopedFieldsForType plus interface types reachable through InterfaceObjects (concrete->interface). Emission is symmetric (every co-keyed field is both reader and writer), deduped by FieldName\x00L1Key, with schema names mapped to response keys (aliases).
Adds resolve.RequestScopedField{FieldName, FieldPath, L1Key, ProvidesData *Object} and RequestScopedFields on FetchCacheConfiguration; ProvidesData stays nil (the visitor fills it next). Data only — the runtime ignores RequestScopedFields until the coordinate-L1 runtime PR. Subgraphs without the directive emit nothing, so behavior is unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…lation
Widen co-keyed @requestScoped selection sets so participants share an identical L1 shape, and populate ProvidesData on each emitted RequestScopedField. propagateRequestScopedWidening (run in LeaveDocument) groups participants by {L1Key, dsHash}, builds the union of datasource-owned selections (incl. hidden @requires deps) for groups of >=2 same-return-type fields, mints synthetic aliases on response-key collision, threads fetch-only aliases through the datasource planner, and injects the missing fields/fragments into the AST while keeping injected fields out of the user response shape; interface-object fallback is supported. populateRequestScopedFieldsProvidesData matches each emitted field to its object-valued planner sub-Object by response key (dropping scalar/no-match hints) and runs ComputeHasAliases.
Conservative: rewrites only when >=2 co-keyed same-type participants exist; absent the directive it is a no-op. Gated on v.caching — existing plan/datasource snapshot tests pass unchanged (byte-identical default). The runtime still ignores this data until the coordinate-L1 runtime PR.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make co-keyed @requestScoped fields share a fetch via a per-request coordinate L1 (requestScopedL1 on the Loader, main-thread only). tryRequestScopedInjection (parallel Phase 1.5, Phase 3.5 retry, resolveSingle) injects a stored value when it contains all fields in the hint's ProvidesData (field-widening via validateItemHasRequiredData, collect-then-inject all-or-nothing), copying out via structuralCopyDenormalized, and skips the fetch on a full hit. exportRequestScopedFields stores each participating field's value (copy-in via structuralCopyNormalized; working-copy-and-swap into an existing entry) so the first-resolved field populates and later co-keyed fields skip. Symmetric model, gated on EnableL1Cache. Sets LoadSkipped + a cacheTraceRequestScopedHits counter (the trace fold is the trace PR). New logic lives in loader_request_scoped.go; loader.go gains only gated hook calls. Default behavior byte-identical (nothing emitted unless the directive is composed). requestScoped tests pass race-clean; the federation E2E test lands with the execution config factory. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…te/invalidate) On each subscription event, populate or invalidate L2 based on the event's entity content: fields beyond @key -> Set the full entity (with __typename) under the rendered entity key (GlobalCacheKeyPrefix + header prefix + interceptor); only @key fields + EnableInvalidationOnKeyOnly -> Delete the entry. Fires ResolverOptions.OnSubscriptionCacheWrite / OnSubscriptionCacheInvalidate callbacks. The per-event hook (processSubscriptionEntityCache) early-returns unless L2 is enabled and population config is present, so non-caching subscriptions are byte-identical. Planner-side configureSubscriptionEntityCachePopulation attaches the config via FindByTypeAndFieldName (BOTH TypeName and FieldName mandatory — empty FieldName is a silent no-op, regression-guarded). Tests are deterministic (channel-backed Flush, no sleeps); subscription cache tests pass race-clean, plan suite unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…pstone) Part A (v2 trace): add CacheTrace/CacheTraceEntity emitted under extensions.trace.fetches[].fetch.trace.cache_trace, gated on TracingOptions.Enable && !ExcludeCacheStats AND the fetch actually caching (nil otherwise — ART output byte-identical when caching is off). Keys only when !ExcludeRawInputData. Folds the requestScoped hit counter into L1 totals at trace-build time. Part B (execution/engine): add SubgraphCachingConfig + SubgraphCachingConfigs.FindBySubgraphName + the WithSubgraphEntityCachingConfigs option; dataSourceMetaData copies the per-subgraph caching slices onto the matching datasource's FederationMetaData via the subgraph-name fallback, only when configs are provided (byte-identical otherwise). First full E2E federation caching tests prove caching reduces real subgraph calls (exact call-count assertions): a root-field L2 hit serves the second request with zero subgraph calls; an entity L2 hit avoids the product fetch (reviews still resolves, asserted honestly). Both modules build; the full execution engine suite and the v2 caching/trace suites pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…r, analytics) Add Go benchmarks (bench_test files only, no production changes) covering the caching engine: the StructuralCopy helper primitives (L1/L2 read/write, with vs without transform), the Copy-Budget pair (caching vs non-caching merge paths so per-request copy overhead is measurable 1:1), the overhead ladder (Disabled / ConfiguredButDisabled / L1Only / L1L2_Miss / L1L2_Hit), and the analytics micro-benches (disabled vs enabled, field hashing). Validates the design: ConfiguredButDisabled has identical allocs to Disabled (no guard leak), L1L2_Hit is cheaper than L1L2_Miss, and analytics adds zero allocations when off. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
@coderabbitai summary
Checklist
Open Source AI Manifesto
This project follows the principles of the Open Source AI Manifesto. Please ensure your contribution aligns with its principles.