Skip to content

docs(webhooks): clarify rate limiting does not interrupt in-flight enrichments#1499

Open
george-dilthey wants to merge 1 commit into
mainfrom
docsbot/doc-1081-intercom-tag-0710-46ee9689
Open

docs(webhooks): clarify rate limiting does not interrupt in-flight enrichments#1499
george-dilthey wants to merge 1 commit into
mainfrom
docsbot/doc-1081-intercom-tag-0710-46ee9689

Conversation

@george-dilthey

Copy link
Copy Markdown
Contributor

What changed

Added one sentence to the Throughput subsection of the Limits section in docs/webhook-integration-guide.md, explicitly stating that the webhook rate limiter operates only at the point of entry — rows already accepted into the table and running through enrichment columns are not affected by throttling on new incoming requests.

Why

Tanner Phillips (Angi, Enterprise) asked via the shared Slack channel: "if we sent MORE than 50k rows in the same batch, say in the same 30 second period — Would this crash? Bump a row before its completed? Actually work but be highly discouraged?"

The existing doc already covered: 10 RPS limit, burst of 20, 100KB payload max, 50k submission cap, auto-delete bypassing the cap, 429 errors returned when throttled. What was missing was an explicit answer to "bump a row before its completed?" — whether exceeding the inbound rate limit could interrupt enrichments already running on rows already in the table.

Human support agent Bo Morin answered: "It won't crash or bump in-flight rows… Clay will rate-limit the excess and return rate-limit errors instead of crashing or interrupting rows that are already enriching." This specific guarantee — that the rate limiter only affects ingestion and not the enrichment pipeline — was not documented.

The verifier sub-agent confirmed (PASS) via clay-base/apps/api/v3/recordFlow/services/webhook.service.ts:436-499 that the rate limit check is at ingestion time and the enrichment queue is a separate pipeline entirely unaffected by webhook throttling. All other limits cited (10 RPS, burst 20, 100KB, 50k cap, passthrough bypass) were also confirmed PASS against the same file.

Source link(s)

Uncertainties

None. All claims verified PASS by the verifier sub-agent against production code.

Related observations

  • PR docs(webhooks): correct submission limit deletion behavior + add FAQ #1492 (docsbot/doc-1073-intercom-tag-0710-a1892d0d) also edits this file but in a completely different section — it changes the Submission limit paragraph (counter-decrement on row deletion) and adds a deletion FAQ. My change is in the Throughput section addressing an entirely different concern (in-flight enrichment safety). The two changes do not conflict. A reviewer merging both should verify line ordering remains coherent.
  • The passthrough/auto-delete feature has a UI gate (useIsPassThroughTablesEnabled) but the backend enforcement of the cap bypass is unconditional — noted by the verifier but not a documentation concern since the feature is already available to Enterprise customers.
  • The verifier noted that numSourceRecords can decrement on row deletion in theory for non-passthrough tables (relevant to PR docs(webhooks): correct submission limit deletion behavior + add FAQ #1492's topic), but this doesn't affect the Throughput section I edited.

Closes DOC-1081

@linear-code

linear-code Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

DOC-1081

@george-dilthey george-dilthey marked this pull request as ready for review July 10, 2026 18:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant