Skip to content

chore(api): OpenAPI audit - document every endpoint, delete 27 dead ones, guard drift#984

Open
edwh wants to merge 3 commits into
masterfrom
chore/api-surface-audit
Open

chore(api): OpenAPI audit - document every endpoint, delete 27 dead ones, guard drift#984
edwh wants to merge 3 commits into
masterfrom
chore/api-surface-audit

Conversation

@edwh

@edwh edwh commented Jul 6, 2026

Copy link
Copy Markdown
Member

Summary

Full audit of the 272 routes across apiv2, the routing server and the spatial KNN server: every endpoint's callers swept across the monorepo, every zero-caller candidate adversarially re-verified (including out-of-repo callers: TrashNothing, IPN webhooks, the housekeeper extension), and the OpenAPI definitions made to match what handlers actually do - with drift guards so they stay that way.

Deleted - 27 verified-dead endpoints

  • apiv2 (17): the four /isochrone CRUD routes (dead since the feat(browse): flip "Nearby" from per-user isochrones to the rippling reach model #921 reach flip - /isochrone/message + /message/count live on), plain GET /messages (+18 tests of it; ModTools uses /modtools/messages), GET /activity, DELETE /message/{id}, GET /modtools/admin/{id}, GET /modtools/alert/{id}, PATCH /config/admin, DELETE /noticeboard/{id}, action-based POST /story + DELETE /story/{id} (live /story/like+/story/unlike kept, their tests rewritten against the live routes - previously untested), POST/DELETE /team, PATCH /microvolunteering, GET /simulation
  • routing (1): /v1/posts-for-member
  • spatial (5): /v1/datasets, per-dataset status + GET within + GET-form within_coords (POST form the routing server uses stays), global /v1/rebuild

Kept despite zero in-repo callers (external users verified)

/changes (TN polls it), TN message endpoints, Stripe + PayPal IPNs, housekeeper-extension endpoints incl. /donations/bulk, /userdump (designed API-key third-party access), spammers export, LoveJunk webhook.

Live bug fixed

ModTools alert click-tracking posted to /alert - never a registered route (handler is at /modtools/alert) - silently 404ing in production. Client path corrected; endpoint documented.

Docs

~40 previously undocumented endpoints gain swagger blocks derived from their handlers; param docs fixed wherever they lied (headline: /location/typeahead documented term, reads q); stale entries removed (spam_keywords, worry_words, /demo ghost, /systemlogs prefix). apiv2's swagger.json is regenerated by the repo generator, not hand-edited - which surfaced that gofmt mangles go-swagger annotations (param bullets + the SecurityDefinitions meta block); reverted and worth remembering: don't gofmt swagger.go.

Drift guards

Each server gains a bidirectional registered-routes-vs-swagger test (spatial's route setup refactored into buildPublicApp/buildAdminApp for introspection), so definitions can't silently rot again.

Also

3 dead client wrappers removed from iznik-nuxt3 (zero callers each).

Code Quality Review

  • Every deletion was double-verified: workflow verdict + independent re-check; two candidates were reprieved on evidence (RecordAlert - live callers via the path bug; spatial upsert - batch integration callers).
  • Response models preserved where they document live shapes (ListMessagesResponse now correctly referenced by /modtools/messages).
  • Pre-existing dangling housekeeperTasksResponse annotation attached to a real type.

Test Plan

  • Go apiv2 via status API: 3,313 pass, including the new drift guard and the swagger regeneration test.
  • routing + spatial module suites green incl. new drift guards.
  • Unit suite for the frontend: 14,134 pass (wrapper removals + alert path fix).
  • Laravel untouched.

Second commit: 15 drift-bug fixes from the overlap review

A follow-up workflow reviewed all 274 routes for overlapping/duplicate function (35 confirmed overlaps, adversarially verified). This PR now also fixes every overlap with live consequences:

  • GiftAid notifications: one canonical helper replaces two divergent copies (soft-deleted declarations were suppressing IPN-path notifications); bulk donation upload now uses the same canonical donor matching as the IPNs (gmail variants + prior-donation fallback), unblocking gift-aid linkage for recurring donors.
  • Chat review path gains the missing held-reply exclusion; AMP chat replies now enter the processing pipeline (push notifications were silently dropped) and reopen closed rosters.
  • Volunteering V2 regains V1's per-group setting gate (live regression) and V1-parity fields; community events: creators see their own pending events on group pages.
  • Newsfeed badge vs feed: Count() no longer applies a hard-coded 1-mile default the feed doesn't use (plus an inverted error check enabling filtering on garbage input).
  • Typeahead: duplicated SQL unified; whitespace normalisation was a literal-backslash-s no-op, now real.
  • Tryst calendar links restored to the V1 internal format (names, 15 min, Europe/London); self-service account deletion now tears down sessions like Forget.
  • Email stats: shared rate/TN helpers; FORCE INDEX dropped when unbounded. Compact email tracking now records scroll depth like the long form (Go + batch).
  • Status runner: the Go suite runner gets an explicit 20-minute limit (the suite legitimately outgrew the default 10 minutes).

Suites on the final tree: Go 3,418 pass, Laravel 4,733 pass, frontend units 14,134 pass.

@netlify

netlify Bot commented Jul 6, 2026

Copy link
Copy Markdown

Deploy Preview for golden-caramel-d2c3a7 canceled.

Name Link
🔨 Latest commit c62a69f
🔍 Latest deploy log https://app.netlify.com/projects/golden-caramel-d2c3a7/deploys/6a5553985a25c600080f445a

@FreegleGeeks FreegleGeeks added CI: Passed CI build passed CI: Running CI build is currently running and removed CI: Passed CI build passed CI: Running CI build is currently running labels Jul 6, 2026
edwh added a commit that referenced this pull request Jul 9, 2026
…dpoints

feat: deprecate-and-observe PR #984's audited apiv2 endpoints (registry)
edwh and others added 2 commits July 12, 2026 18:55
… ones, guard drift

Audit of all 272 routes across apiv2 (iznik-server-go), the routing server
(iznik-routing-go) and the spatial KNN server (iznik-spatial-go): every caller
in the monorepo was swept per endpoint, every zero-caller candidate adversarially
re-verified including out-of-repo callers (TrashNothing, IPN webhooks, the
housekeeper extension, app shell), and the OpenAPI definitions synced to what
the handlers actually do.

DELETED (verified unused, high confidence - registrations, handlers, docs, tests):
- apiv2: GET/PUT/PATCH/DELETE /isochrone (per-user isochrone CRUD, dead since the
  PR #921 reach flip; /isochrone/message and /message/count live on), plain
  GET /messages (ModTools uses /modtools/messages; ~18 dead-endpoint tests went
  with it), GET /activity, DELETE /message/{id}, GET /modtools/admin/{id},
  GET /modtools/alert/{id}, PATCH /config/admin, DELETE /noticeboard/{id},
  POST /story + DELETE /story/{id} (action-based; live /story/like + /story/unlike
  kept - their tests rewritten against the live routes, which previously had none),
  POST /team, DELETE /team, PATCH /microvolunteering, GET /simulation
- routing: GET /v1/posts-for-member
- spatial: GET /v1/datasets, GET /v1/{dataset}/status, GET /v1/{dataset}/within,
  GET form of within_coords (the POST form the routing server uses stays),
  global POST /v1/rebuild (per-dataset rebuild stays)

KEPT despite zero in-repo callers (external users verified): /changes (TN polls),
TN message endpoints, Stripe/PayPal IPNs, housekeeper extension endpoints incl.
/donations/bulk, /userdump (API-key third-party access), spammers export,
LoveJunk webhook.

LIVE BUG FIXED: ModTools alert click-tracking posted to /alert, a path that was
never registered (handler lives at /modtools/alert) - silently 404ing in
production. Client path corrected; endpoint kept and documented.

DOCS: ~40 previously undocumented endpoints now have swagger:route blocks derived
from their handlers (all /helper/*, /bulkoffer/update/{token}, /rippling/metrics,
/message/{id}/reach, /messages/markseen, concern_keywords, six /user/{id}/*
routes, /amp/*, /avatar/{name}, /discourse_sso, /version, email-tracking /e/d/*,
routing's catchment/group-proximity/group-extent/group-actives/groups list/
reachable-groups, spatial's admin-port upsert/rebuild/remove with request bodies).
Param mismatches fixed everywhere docs lied about handlers - most notably
/location/typeahead documented 'term' but reads 'q'. Stale swagger entries
removed (spam_keywords, worry_words, /demo ghost, /systemlogs prefix).

apiv2's swagger.json is REGENERATED from swagger.go with the repo generator (not
hand-edited); fixing that surfaced and reverted gofmt mangling of go-swagger
annotations (param bullets and the SecurityDefinitions meta block - do not gofmt
swagger.go), and attached the dangling housekeeperTasksResponse definition.

DRIFT GUARDS: each server gains a test comparing registered routes against its
swagger paths bidirectionally (apiv2 test/swagger_sync_test.go, routing
swagger_sync_test.go, spatial route_drift_test.go - the latter with route setup
refactored into buildPublicApp/buildAdminApp for introspection), so docs cannot
silently rot again.

Also: 3 dead client wrapper methods removed from iznik-nuxt3 (MessageAPI.del,
ConfigAPI.patchAdminv2, NoticeboardAPI.del - zero callers each).

Suites green via status API on exactly this tree: Go 3,313; vitest 14,134;
routing + spatial module tests green incl. new drift guards.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U3Hp9AG5RzAxVQVzXpkwgn
…view

Follow-up to the endpoint audit: a workflow review of all 274 routes for
overlapping/duplicate function confirmed 35 overlaps; these are the fixes for
every one with live consequences.

- donations: one canonical triggerGiftAidNotification (deleted IS NULL filter +
  ORDER BY id DESC + INSERT IGNORE, uniform columns) replacing two divergent
  copies; BulkUploadDonations now matches donors via
  MatchUserByEmailOrPriorDonation (gmail dot/plus variants + prior-donation
  fallback) instead of exact email, unblocking gift-aid linkage for recurring
  PayPal Giving Fund donors.
- chat: participant fetch for the review path gains the missing NOT EXISTS
  rippling_held_replies filter (held replies no longer leak).
- amp: PostChatReply now sets processingrequired=1/processingsuccessful=0
  (push notifications were silently dropped) and reopens CLOSED rosters,
  matching the normal chat path.
- location: typeahead SQL deduplicated into one shared query used by both
  endpoints; fixed the whitespace-normalisation no-op (ReplaceAll of the
  literal two-character sequence backslash-s).
- newsfeed: Count() no longer hard-codes a 1609m default the Feed doesn't use,
  and its inverted error check no longer enables filtering on a garbage
  distance param - badge and feed now agree.
- volunteering: V2 group listing regains V1's per-group volunteering-setting
  gate (the live regression); GET /volunteering/{id} gains renewed/url/online
  and https:// contacturl normalisation for V1 parity.
- communityevent: creators now see their own pending events on group pages,
  matching the personal feed.
- tryst: calendarLink rewritten to V1 form (/calendar?data= with participant
  names, 15-minute duration, Europe/London; embedded tzdata); parse fallbacks
  extracted to parseArrangedFor and covered.
- user: self-service account deletion now tears down sessions and skips the
  post-auth check, matching Forget.
- email stats: shared rate-formula + TN-exclusion helpers across the three
  stats endpoints; StatsByType drops FORCE INDEX when unbounded.
- email tracking: compact-format images accept ?s= and apply the same
  max-update scroll-depth logic as the long form; the batch digest now encodes
  the scroll estimate on compact URLs.
- status runner: go test gets -timeout 20m (the suite passed 9 minutes of wall
  clock and Go's default 10m limit started killing healthy runs).

The group lat/lng-vs-polygon divergence (apiv2 vs routing group lists) is
documented in the review as keep-both; no code change.

Suites on exactly this tree via status API: Go 3,418; Laravel 4,733;
vitest 14,134 (unchanged since the audit commit - no JS touched).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MAMpjuyJgnUSEAsdUcpJr6
@edwh edwh force-pushed the chore/api-surface-audit branch from 05edd49 to 5e7733c Compare July 12, 2026 17:57
@FreegleGeeks FreegleGeeks added CI: Running CI build is currently running CI: Failed CI build failed and removed CI: Passed CI build passed CI: Running CI build is currently running labels Jul 12, 2026
…ggerSync

The new drift-guard test (TestSwaggerSync) failed CI because /town/near,
/rippling/analytics, /voicepost/chunk and /voicepost/finish are registered
in routes.go but had no swagger:route entry in swagger/swagger.go - the
actual source the go-swagger generator reads. The @Router-style comments
sprinkled near these handlers use swaggo syntax, which go-swagger's
`generate spec` does not parse, so they silently produced nothing.

Added proper swagger:route blocks for all four endpoints and regenerated
swagger.json. Verified zero drift by replicating TestSwaggerSync's exact
route-extraction/comparison logic against the current routes.go and
swagger.json (registered/documented sets now match, modulo the existing
allowlist).
@FreegleGeeks FreegleGeeks added CI: Running CI build is currently running CI: Passed CI build passed and removed CI: Failed CI build failed CI: Running CI build is currently running labels Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI: Passed CI build passed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants