[MET-569] Add CoinMarketCap DEX adapter (/cmc summary, ticker, assets)#17
[MET-569] Add CoinMarketCap DEX adapter (/cmc summary, ticker, assets)#17metajinglun wants to merge 10 commits into
Conversation
Adds /cmc/summary, /cmc/ticker, and /cmc/assets under a new src/routes/coinmarketcap.ts, mirroring the CoinGecko adapter: pairs are built from on-chain DAO discovery (getAllDaos) plus rolling-24h spot metrics from the served user_pool ETL. summary/ticker require the served DB and return 503 (never zero volume) when it is unavailable; assets is pure on-chain metadata and does not. Token→CMC mapping is controlled by an optional CMC_ALLOWED_MINTS env allowlist (empty = serve all, same default as the other adapters). No dedicated auth — CMC reuses the shared trusted-API-key rate-limit tiers. Includes response types, route tests, README docs, and example.env. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- config: validate CMC_ALLOWED_MINTS entries as Solana pubkeys at startup (fail fast on a typo instead of silently serving an empty /cmc feed), matching the EXCLUDED_DAOS pattern. - summary: add the `type: "spot"` market discriminator — every pair we surface is a spot market (conditional pools are never selected). - document that the per-pair catch only skips uncomputable pairs; infra/DB outages still surface as 5xx before the loop (getAllDaos / getSpotRolling24hMetrics both throw). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- allowlist: when CMC_ALLOWED_MINTS is set but matches zero discovered DAOs, fail closed (503 + alert) instead of serving an empty feed a poller would read as "delisted". - volume: malformed 24h volume from the served ETL for an INCLUDED pair now surfaces as 500 (contract-drift) rather than silently dropping the pair; the per-pair catch rethrows AppError so integrity failures propagate while genuine per-pair calc skips are still tolerated. - tests: cover metrics-query failure → 5xx, malformed volume → 5xx, and allowlist filtering + fail-closed behavior. - docs: note /cmc DB requirement and CMC_ALLOWED_MINTS in the env table. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Iteration-3 review follow-ups: - allowlist: fail closed if ANY configured mint is absent from discovery (not only when zero match) — a partial feed can look like a delisting. Alert names the missing mints. - metrics: extend the fail-closed rule to a corrupt non-zero 24h high/low (shared parseFinite helper, code CMC_MALFORMED_METRIC) so no corrupt field slips through as a silently-omitted extreme. - document deliberate spec decisions in code + README: price_change_percent_24h is omitted (no reliable 24h-open; never fabricate — CoinGecko parity), and base_id/quote_id carry the mint/contract address (the same id /cmc/assets is keyed by) because our tokens have no CMC unified id yet. - tests: malformed-high → 5xx and partial-allowlist → 503 regressions. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Package Protection
Repository GuardExact dependency versions
Package minimum age
Sensitive wallet / program changes
Overall status: pass Transitive supply-chain threats are covered by the Socket scanner (via |
|
Can we add versioning into this? As that was requested from CMC, so we should support default endpoints as is already defined and out there, but also now have versioning in the api url? |
Add API versioning to the CMC DEX adapter, as requested by CMC. Every endpoint is now also served under an explicit /cmc/v1 prefix (/cmc/v1/summary, /cmc/v1/ticker, /cmc/v1/assets) while the original unversioned paths remain published as-is. Both prefixes map to the same handler (URL aliases), so the current contract is treated as v1 and a future breaking change can land under /cmc/v2. - Register each handler on both prefixes via a cmcPaths() helper - Document the versioning in the README and root endpoint listing - Add tests asserting /cmc/v1/* is byte-identical to /cmc/* and preserves fail-closed (503) semantics Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Added API versioning as requested by CMC (f15f26f). Every CMC endpoint is now also served under an explicit
Both prefixes map to the same handler (they're URL aliases, not a behavioural fork), so the two can never drift. The unversioned path is treated as the current (v1) contract; a future breaking change would land under |
Reject partially-parsed ETL metrics in the CMC feed: parseFinite now uses
a full-string Number() parse instead of parseFloat, so a corrupt value with
a numeric prefix ("12abc") fails closed with CMC_MALFORMED_METRIC instead of
being silently truncated to 12 and served as a valid-looking 200. Blank
strings are rejected too (would otherwise read as a genuine 0). Adds a
regression test for the numeric-prefix case.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…_24h Close the two gaps found checking the /cmc adapter against CoinMarketCap's Section C DEX spec (the C1 Uniswap sample), not just CoinGecko parity. 1. Inline token identity on /cmc/ticker — restore base_name/base_symbol/ quote_name/quote_symbol, which C1 requires in each pair object and which the sibling CoinGecko /api/tickers already emits. A shared identityLabel() helper (mint-prefix fallback) keeps /cmc/ticker and /cmc/assets always consistent and never emits an empty string. 2. price_change_percent_24h on /cmc/summary — computed from the AMM's EXACT price 24h ago: FutarchyAMM price is a pure function of pool reserves and reserves only move on a swap, so the reserves of the last swap >=24h ago (user_pool_swaps) are the pool's exact state 24h ago. Priced through the same calculatePrice as last_price (true mid-vs-mid). Omitted, never fabricated, for markets younger than 24h. Summary-only (CMC's ticker/A2 has no such field). The swaps source is non-essential: a query failure degrades to field-absent (200), not 503 — an absent change is "unknown", not a false financial claim. New: externalDatabaseService.getSpotReserves24hAgo(). Tests cover inline identity + fallback, computed %, young-market omit, source-failure graceful degradation, and ticker exclusion. tsc/knip/repo-guard clean; full suite 137 pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| `SELECT DISTINCT ON (base_mint) | ||
| base_mint AS token, | ||
| amm_base_reserves::text AS base_reserves, | ||
| amm_quote_reserves::text AS quote_reserves | ||
| FROM futarchy.user_pool_swaps | ||
| WHERE source = 'futarchy_amm' | ||
| AND market_kind = 'spot' | ||
| AND block_time <= $1 | ||
| AND base_mint = ANY($2::text[]) | ||
| AND amm_base_reserves IS NOT NULL | ||
| AND amm_quote_reserves IS NOT NULL | ||
| AND amm_base_reserves > 0 | ||
| ORDER BY base_mint, block_time DESC`, | ||
| [cutoff, tokens] | ||
| ); | ||
|
|
||
| const reservesMap = new Map<string, { baseReserves: string; quoteReserves: string }>(); |
There was a problem hiding this comment.
When the same base token has swaps in more than one spot market, this query picks the latest row for the base mint regardless of quote mint or DAO. /cmc/summary then prices those reserves with the current pair's decimals, so price_change_percent_24h can be computed from a different market and returned as valid financial data.
There was a problem hiding this comment.
Fixed in 75cb690. Root cause is shared: the served ETL keys reserves/volume/identity by base mint with no per-pool dimension, so a base-mint collision can't be attributed to the right market. Rather than price one market's 24h-ago reserves with another pair's decimals, assertUniqueBaseMints() now fails the whole feed closed (503 CMC_DUPLICATE_BASE_MINT + alert) when two served DAOs share a base mint. Regression test added.
| for (const dao of allDaos) { | ||
| tokenToDaoMap.set(dao.baseMint.toString(), dao.daoAddress.toString()); | ||
| } | ||
|
|
||
| const baseMints = allDaos.map(dao => dao.baseMint.toString()); | ||
| const spotMetrics = await externalDatabaseService.getSpotRolling24hMetrics(baseMints); | ||
|
|
||
| const volumeByDao = new Map<string, { base_volume_24h: string; target_volume_24h: string; high_24h: string; low_24h: string }>(); | ||
| for (const [token, metrics] of spotMetrics.entries()) { | ||
| const daoAddress = tokenToDaoMap.get(token); | ||
| if (!daoAddress) continue; | ||
| volumeByDao.set(daoAddress, { | ||
| base_volume_24h: metrics.base_volume_24h, | ||
| target_volume_24h: metrics.target_volume_24h, | ||
| high_24h: metrics.high_24h, | ||
| low_24h: metrics.low_24h, | ||
| }); | ||
| } |
There was a problem hiding this comment.
This map keeps only one DAO address per base mint, but the later loop still emits every discovered DAO. If two DAOs share a base mint, only the last one receives the rolling metrics and the other falls through to zero volume/high/low, so CMC can receive a live pair with a valid price and false zero trading activity.
There was a problem hiding this comment.
Fixed in 75cb690. Same root cause — tokenToDaoMap collapsing to one DAO per base mint can no longer silently emit a false zero for the losing pair, because servedDaos() now fails the feed closed (503 CMC_DUPLICATE_BASE_MINT) before the emit loop when two DAOs share a base mint. Regression test asserts /cmc/summary and /cmc/ticker both 503.
|
|
||
| const addAsset = (mint: string, symbol: string | undefined, name: string | undefined): void => { | ||
| // First writer wins: the base token's own metadata is authoritative, and a | ||
| // shared quote (USDC) is identical across pairs, so skipping re-adds is safe. |
There was a problem hiding this comment.
/cmc/assets keeps the first metadata seen for a mint, while /cmc/ticker emits each DAO's inline name and symbol. If two discovered DAOs share a base mint but resolve different metadata, CMC can map ticker.base_id to an asset entry with a different symbol/name than the ticker row.
There was a problem hiding this comment.
Fixed in 75cb690. /cmc/assets now routes through the same servedDaos() guard, so a base-mint collision (which is the only way ticker inline metadata could diverge from the first-writer-wins asset entry) fails closed with 503 rather than mapping ticker.base_id to a divergent asset. A shared quote (USDC) across distinct base mints is unaffected and still serves. Regression test added.
Greptile flagged three defects that all share one root cause: the CMC adapter keys volume, 24h-ago reserves, and asset identity by base mint, but the served-ETL tables carry no per-pool dimension. If two discovered markets ever share a base mint: - getSpotReserves24hAgo (DISTINCT ON base_mint) picks one market's reserves and prices them with the other pair's decimals (P1). - tokenToDaoMap keeps one dao per base mint, so the other pair emits a false zero volume (P1). - /cmc/assets first-writer-wins can map a ticker.base_id to a different symbol/name than the ticker row (P2). Fix them at the common root: assertUniqueBaseMints() fails the whole feed closed (503 CMC_DUPLICATE_BASE_MINT + alert) when two served DAOs share a base mint, since per-market attribution is impossible. Runs after the allowlist, so narrowing to a single side of a collision serves normally. Both feeds route through a shared servedDaos() so they can't diverge. Adds regression tests (summary/ticker/assets fail closed; allowlist narrows a collision to a served single pair) and documents the guard in the README. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Make the 24h-ago reserve snapshot query deterministic. DISTINCT ON (base_mint) with only `ORDER BY base_mint, block_time DESC` could pick any swap row when several share the same block_time, so price_change_percent_24h could be computed from a non-final reserve state (and vary with query plans). Extend the ORDER BY with the same event-order columns the DexScreener /events route uses (slot, signature, inner_group, inner_ix) so the selected row is truly the last swap at or before the cutoff. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…e change getSpotReserves24hAgo used a single DISTINCT ON over user_pool_swaps filtered by `block_time <= now-24h`. That range spans nearly all history, so the planner seq-scanned + sorted the whole table on every /cmc/summary request — measured ~7.9s over 1.4M rows on the served DB (EXPLAIN: Parallel Seq Scan), growing unbounded as swaps accumulate. The existing idx_user_pool_swaps_base_mint (base_mint, block_time) index was not used. Rewrite as a per-token LATERAL LIMIT 1 driven off the input mint array: each token does a backward Index Scan on that index and stops at the first matching row. Measured ~30ms (~260x faster), reads ~0.3% of the buffers, and stays flat as history grows. Identical result set and the same deterministic tie-break ordering (block_time, slot, signature, inner_group, inner_ix). Verified with EXPLAIN ANALYZE against the live served DB. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2 participants
[MET-569] Extend External API for CoinMarketCap
Adds a CoinMarketCap DEX adapter ([Section C] of CMC's integration requirements) for the FutarchyAMM and our tokens. Modeled on the existing CoinGecko adapter — CMC's DEX spec is field-for-field close — so both feeds are built from the same on-chain DAO discovery + rolling-24h ETL metrics.
Endpoints (all under
/cmc/)GET /cmc/summary— 24h overview array of every tradeable pair (price, spread, volume, high/low,type: "spot").GET /cmc/ticker— object keyed byBASE_QUOTEpair withbase_id/quote_id/price/volume/isFrozen.GET /cmc/assets— token identity keyed by mint (name, symbol, contractAddress, maker/taker fee).summary/tickercarry 24h volume → require the served DB and return503(never zero volume) when it's down.assetsis pure on-chain metadata and does not.Mapping & auth (per triage answers)
CMC_ALLOWED_MINTSallowlist (validated as pubkeys at startup; empty = serve all, matching the other adapters).X-API-Key) — nothing new needed.AppError/asyncHandler(503/500 withcode), same as CoinGecko/DexScreener.Financial-integrity behavior
CMC_MALFORMED_METRIC) — contract drift never slips through as a valid-looking partial.CMC_ALLOWED_MINTSfails closed (503,CMC_ALLOWLIST_NO_MATCH) if any configured mint is absent from discovery — a partial feed could read as a delisting.Deliberate spec decisions (flagged for human review)
price_change_percent_24homitted: the served ETL exposes no reliable 24h-ago open, and this repo never fabricates a financial value (a fake0%is worse than omitting). CoinGecko/api/tickersomits it for the same reason. Documented in code + README.base_id/quote_id= Solana mint (contract address): our tokens have no CMC unified cryptoasset id yet; the mint is the canonical DEX identifier and is the same id/cmc/assetsis keyed by, so CMC maps ticker → asset consistently. Emitting the0"unknown" sentinel would make every pair unmappable.Tests & validation
tests/routes/coinmarketcap.test.ts(happy paths, 503 DB-down, metrics-query failure → 5xx, malformed volume/high → 5xx, allowlist filter + partial fail-closed).tsc --noEmitclean;knip(phantom deps) clean; repo-guard pass.Review notes
Went through 3 Codex pre-review iterations. All actionable findings were fixed. Two remaining findings are the documented spec tradeoffs above (resolved on principle). One finding — "new tests fail under Bun" (
app.address().port null) — is a Codex-sandbox Supertest harness issue, not reproducible here:bun testpasses 127/127 using the identicalcreateTestApp+supertest pattern as every existing route test, and it's what CI runs.Opened as draft for human sign-off on the two spec tradeoffs before marking ready.
🤖 Generated with Claude Code
Greptile Summary
This PR adds CoinMarketCap DEX feeds for the FutarchyAMM. The main changes are:
/cmc/summary,/cmc/ticker, and/cmc/assetsroutes./cmc/v1/*aliases for the same handlers.CMC_ALLOWED_MINTS.Confidence Score: 4/5
The new CMC feed needs fixes for markets that share a base mint before merging.
src/services/externalDatabaseService.ts and src/routes/coinmarketcap.ts
Important Files Changed
CMC_ALLOWED_MINTSparsing and startup PublicKey validation.Reviews (1): Last reviewed commit: "feat(external-api): CMC inline ticker id..." | Re-trigger Greptile
Context used: