Add optional Docker dev environment + dummy data seed scripts

README.md

@@ -54,6 +54,25 @@ diesel --database-url "${DATABASE_URL}" migration run
 
 `cargo run pull` will run the timed jobs continuously: grab replay, update ratings, update ranking, update redis, etc.
 
+#### Development database (Docker, optional)
+
+Requires [Docker](https://www.docker.com/) with Compose. If you don't want to
+install Postgres/Redis locally, `docker/docker-compose.dev.yml` provides both,
+plus a disposable `diesel_cli` container for running migrations (useful on
+Windows, where linking `diesel_cli` against `libpq` can be a hassle):
+
+```
+docker compose -p puddle-farm-dev -f docker/docker-compose.dev.yml up -d postgres redis
+docker compose -p puddle-farm-dev -f docker/docker-compose.dev.yml run --rm diesel migration run
+```
+
+Or use `scripts/seed/reset.sh` to do both from scratch (drops any existing
+data/volumes first).
+
+To populate the database with dummy data (players, ratings, matches, rankings)
+so the frontend has something to render without running `cargo run pull`, see
+`scripts/seed/README.md`.
+
 `cargo run hourly` runs the hourly jobs once, then exits.
 
 To generate a new model.rs:

docker/diesel/Dockerfile

@@ -0,0 +1,9 @@
+FROM rust:slim
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    libpq-dev pkg-config \
+    && rm -rf /var/lib/apt/lists/*
+
+RUN cargo install diesel_cli --no-default-features --features postgres
+
+ENTRYPOINT ["diesel"]

docker/docker-compose.dev.yml

@@ -0,0 +1,34 @@
+services:
+  postgres:
+    image: postgres:18
+    container_name: pf-postgres
+    environment:
+      POSTGRES_USER: user
+      POSTGRES_PASSWORD: password
+      POSTGRES_DB: puddle_farm
+    ports:
+      - "5432:5432"
+    volumes:
+      - pgdata:/var/lib/postgresql
+
+  redis:
+    image: redis:latest
+    container_name: pf-redis
+    ports:
+      - "6379:6379"
+
+  # docker compose -f docker/docker-compose.dev.yml run --rm diesel migration run
+  diesel:
+    build: ./diesel
+    container_name: pf-diesel
+    profiles: ["tools"]
+    environment:
+      DATABASE_URL: postgresql://user:password@postgres/puddle_farm
+    volumes:
+      - ../migrations:/app/migrations
+    working_dir: /app
+    depends_on:
+      - postgres
+
+volumes:
+  pgdata:

scripts/seed/README.md

@@ -0,0 +1,55 @@
+# Dev database seed scripts
+
+Generate dummy data for a local backend so every page (Top, Legend, per-character rankings, search, Popularity, Matchup Tables, Rank Distribution) has something to show without running the real `cargo run pull` job (which requires a logged-in Steam client).
+
+## Prerequisites
+
+Requires Docker with Compose, and `bash`/`docker`/`docker compose` on `PATH`.
+
+Start the dev database first (see `docker/docker-compose.dev.yml`):
+
+```bash
+docker compose -p puddle-farm-dev -f docker/docker-compose.dev.yml up -d postgres redis
+docker compose -p puddle-farm-dev -f docker/docker-compose.dev.yml run --rm diesel migration run
+```
+
+Or just run `scripts/seed/reset.sh`, which does both from scratch (and drops any existing data/volumes).
+
+## What gets created
+
+With the defaults (`seed-all.sh` with no args):
+
+- **200 players** (`TestPlayer1`..`TestPlayer200`), one character rating each, spread across all 19 rank tiers (Placement..Diamond 3, ~10+ players per tier).
+- **Top 20** by rating are also written to `leaderboard_legend` (Legend page).
+- **`TestPlayer1`..`TestPlayer8`** additionally get a second character rating, for testing the character switcher on player pages.
+- **2000 regular ranked games** + **200 Vanquisher-tier games** (`value_a`/`value_b >= 10001600`), randomly paired between seeded players. Each match's rating value is offset from the player's base rating (winner up, loser down) so the frontend's per-match rating-change display isn't always 0.
+- Matchup Tables, Popularity, and Rank Distribution pages are all populated by aggregating the above (mirrors the real aggregation SQL in `src/pull.rs`).
+
+## Usage
+
+Run everything in the correct order:
+
+```bash
+bash scripts/seed/seed-all.sh [player_count]   # default 200
+```
+
+Or run scripts individually - **order matters**, each one depends on data from the previous step:
+
+1. `seed-players.sh [count]` - creates `players` / `player_ratings` spread across all 19 rank tiers (Placement through Diamond 3, at least ~10 players per tier), plus the `leaderboard_*` Redis keys (Top / per-character / Legend pages).
+2. `seed-games.sh [game_count] [vanq_game_count]` - creates dummy match history in `games`, required by the matchup/popularity scripts below.
+3. `seed-matchups.sh` - aggregates `games` into `matchup_*` / `matchup_vanq_*` Redis keys, mirroring the SQL in `src/pull.rs::update_matchups()`.
+4. `seed-distribution.sh` - aggregates `player_ratings` into the `distribution_rating` Redis key, mirroring `src/pull.rs::update_distribution()`.
+5. `seed-popularity.sh` - aggregates `games` into `popularity_per_player_*` / `popularity_per_character_*` Redis keys, mirroring `src/pull.rs::update_popularity()`.
+
+All scripts are re-runnable: seeded rows/games are deleted and re-inserted each time, so you can just re-run `seed-all.sh` to reshuffle the data.
+
+Re-running a single script in isolation is only safe for iterating on that script itself. `seed-players.sh` deletes and recreates `games` as part of resetting `player_ratings`, so running it alone leaves `games` empty while the `matchup_*`/`popularity_*` Redis keys still hold stale data from the previous `games` - re-run `seed-all.sh` afterward to restore consistency.
+
+## Reset
+
+`reset.sh` recreates the postgres/redis containers (and volumes) from scratch and re-runs migrations. The backend process holds stale DB/Redis connections after this - restart it afterward, then re-run `seed-all.sh` if you want data again.
+
+## Notes
+
+- Seeded player IDs start at `900000000000001`, well outside the real Steam64 ID range, so seeded data never collides with real players.
+- `seed-games.sh` also inserts a smaller batch of games with `value_a`/`value_b >= 10001600` (the Vanquisher floor) so `matchup_vanq_*` has data too, not just the regular `matchup_*` keys.

scripts/seed/reset.sh

@@ -0,0 +1,25 @@
+#!/usr/bin/env bash
+# Full reset: recreate postgres/redis containers (and volumes) from scratch, re-run migrations.
+# `down -v` is destructive - it drops the pgdata volume unconditionally, no confirmation.
+# The backend process holds stale DB/Redis connections after this runs - restart it afterward.
+set -euo pipefail
+
+REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
+COMPOSE="docker compose -p puddle-farm-dev -f $REPO_ROOT/docker/docker-compose.dev.yml"
+
+$COMPOSE down -v
+$COMPOSE up -d postgres redis
+
+echo "Waiting for Postgres to accept connections..."
+for i in $(seq 1 30); do
+  $COMPOSE exec -T postgres pg_isready -U user > /dev/null 2>&1 && break
+  if [ "$i" -eq 30 ]; then
+    echo "Postgres did not become ready after 30s." >&2
+    exit 1
+  fi
+  sleep 1
+done
+
+$COMPOSE run --rm diesel migration run
+
+echo "Reset complete. Restart the backend process (stale DB/Redis connections)."

scripts/seed/seed-all.sh

@@ -0,0 +1,19 @@
+#!/usr/bin/env bash
+# Runs every seed script in dependency order:
+#   seed-players.sh (players/ratings/leaderboard)
+#     -> seed-distribution.sh (rank distribution, needs only ratings)
+#     -> seed-games.sh (dummy match history)
+#         -> seed-matchups.sh (matchup tables, needs games)
+#         -> seed-popularity.sh (popularity, needs games)
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+N="${1:-200}"
+GAME_COUNT="${2:-2000}"
+VANQ_GAME_COUNT="${3:-200}"
+
+bash "$SCRIPT_DIR/seed-players.sh" "$N"
+bash "$SCRIPT_DIR/seed-distribution.sh"
+bash "$SCRIPT_DIR/seed-games.sh" "$GAME_COUNT" "$VANQ_GAME_COUNT"
+bash "$SCRIPT_DIR/seed-matchups.sh"
+bash "$SCRIPT_DIR/seed-popularity.sh"

scripts/seed/seed-distribution.sh

@@ -0,0 +1,68 @@
+#!/usr/bin/env bash
+# Requires seed-players.sh to have been run first (reads from player_ratings).
+# Bucket/percentage/percentile SQL mirrors src/pull.rs update_distribution().
+set -euo pipefail
+
+REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
+cd "$REPO_ROOT"
+
+PSQL="docker exec -i pf-postgres psql -v ON_ERROR_STOP=1 -U user -d puddle_farm"
+
+ID_BASE=900000000000000
+
+DIST_JSON=$($PSQL -tAc "
+-- Bucket boundaries copied from src/pull.rs update_distribution() - keep both in
+-- sync if rank tiers ever change (also duplicated a third time in seed-players.sh's
+-- ranks CTE, in a different format).
+WITH buckets AS (
+    SELECT
+      unnest(ARRAY[-10000000, 1,    1000, 2000, 3000, 4200, 5400, 6600, 8800,  11000, 13200, 15600, 18000, 20400, 24400, 28400, 32400, 36600, 40800, 10000000, 10001600, 10001700, 10001800]) AS lower_bound,
+      unnest(ARRAY[1, 1000, 2000, 3000, 4200, 5400, 6600, 8800, 11000, 13200, 15600, 18000, 20400, 24400, 28400, 32400, 36600, 40800, 10000000, 10001600, 10001700, 10001800, 200000000]) AS upper_bound
+),
+bucket_counts AS (
+    SELECT
+        b.lower_bound,
+        b.upper_bound,
+        count(t.value) AS bucket_count
+    FROM player_ratings t
+    LEFT JOIN buckets b ON t.value >= b.lower_bound AND t.value < b.upper_bound
+    WHERE t.id >= $ID_BASE  -- deviation from pull.rs: scope to seeded rows only
+    GROUP BY b.lower_bound, b.upper_bound
+),
+percentiles AS (
+    SELECT
+        lower_bound,
+        upper_bound,
+        bucket_count,
+        SUM(bucket_count) OVER (ORDER BY lower_bound) as cumulative_sum,
+        SUM(bucket_count) OVER () as total_count,
+        SUM(CASE WHEN upper_bound != 1 THEN bucket_count ELSE 0 END) OVER (ORDER BY lower_bound) as cumulative_sum_ranked_only,
+        SUM(bucket_count) FILTER (WHERE upper_bound != 1) OVER () as total_count_excluding_placement
+    FROM bucket_counts
+),
+final AS (
+    SELECT
+        p.lower_bound,
+        p.upper_bound,
+        p.bucket_count AS count,
+        CASE
+            WHEN p.upper_bound = 1 THEN CAST(ROUND((p.bucket_count * 100.0 / p.total_count), 2) AS FLOAT)
+            ELSE CAST(ROUND((p.bucket_count * 100.0 / p.total_count_excluding_placement), 2) AS FLOAT)
+        END AS percentage,
+        CASE
+            WHEN p.upper_bound = 1 THEN 0.0
+            ELSE CAST(ROUND((100.0 - ((p.cumulative_sum_ranked_only - p.bucket_count) * 100.0 / p.total_count_excluding_placement)), 2) AS FLOAT)
+        END AS percentile
+    FROM percentiles p
+    ORDER BY p.lower_bound
+)
+SELECT COALESCE(json_agg(row_to_json(final)), '[]') FROM final;
+")
+
+echo "$DIST_JSON" | docker exec -i pf-redis redis-cli -x SET distribution_rating > /dev/null
+
+ONE_MONTH_PLAYERS=$($PSQL -tAc "SELECT count(DISTINCT id) FROM player_ratings WHERE id >= $ID_BASE;" | tr -d '\r')
+docker exec pf-redis redis-cli SET one_month_players "$ONE_MONTH_PLAYERS" > /dev/null
+docker exec pf-redis redis-cli SET last_update_daily "$(date -u +"%Y-%m-%d %H:%M:%S")" > /dev/null
+
+echo "Seeded distribution_rating (one_month_players=$ONE_MONTH_PLAYERS)."

scripts/seed/seed-games.sh

@@ -0,0 +1,89 @@
+#!/usr/bin/env bash
+# Requires seed-players.sh to have been run first (reads from player_ratings).
+set -euo pipefail
+
+REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
+cd "$REPO_ROOT"
+
+PSQL="docker exec -i pf-postgres psql -v ON_ERROR_STOP=1 -U user -d puddle_farm"
+
+ID_BASE=900000000000000
+GAME_COUNT="${1:-2000}"
+VANQ_GAME_COUNT="${2:-200}"
+VANQ_VALUE_FLOOR=10001600
+
+PLAYER_COUNT=$($PSQL -tAc "SELECT count(*) FROM player_ratings WHERE id >= $ID_BASE;" | tr -d '\r')
+if [ "$PLAYER_COUNT" -eq 0 ]; then
+  echo "No seeded player_ratings found (id >= $ID_BASE). Run seed-players.sh first." >&2
+  exit 1
+fi
+
+$PSQL -v id_base="$ID_BASE" -v game_count="$GAME_COUNT" -v vanq_game_count="$VANQ_GAME_COUNT" -v vanq_floor="$VANQ_VALUE_FLOOR" <<'SQL'
+BEGIN;
+
+DELETE FROM games WHERE id_a >= :id_base OR id_b >= :id_base;
+
+-- Regular ranked matches. value_a/value_b get a small per-match offset around
+-- the player's base rating (winner +, loser -) instead of the raw base value,
+-- otherwise every match for a player has the same own_rating_value and the
+-- frontend's per-match rating-change display (own_rating_value delta between
+-- consecutive matches, see frontend/src/utils/Player.tsx groupMatches) is
+-- always 0.
+WITH pool AS (
+  SELECT pr.id, pr.char_id, pr.value, p.name
+  FROM player_ratings pr JOIN players p ON p.id = pr.id
+  WHERE pr.id >= :id_base
+),
+pairs AS (
+  SELECT
+    a.id AS id_a, a.name AS name_a, a.char_id AS char_a, a.value AS value_a,
+    b.id AS id_b, b.name AS name_b, b.char_id AS char_b, b.value AS value_b,
+    (CASE WHEN random() < 0.5 THEN 1 ELSE 2 END) AS winner
+  FROM pool a JOIN pool b ON a.id <> b.id
+  ORDER BY random()
+  LIMIT :game_count
+)
+INSERT INTO games (timestamp, id_a, name_a, char_a, platform_a, id_b, name_b, char_b, platform_b, winner, game_floor, value_a, value_b)
+SELECT
+  now() - (random() * interval '30 days'),
+  id_a, name_a, char_a, 1,
+  id_b, name_b, char_b, 1,
+  winner,
+  0,
+  GREATEST(0, value_a + (CASE WHEN winner = 1 THEN (random() * 20)::int ELSE -(random() * 20)::int END)),
+  GREATEST(0, value_b + (CASE WHEN winner = 2 THEN (random() * 20)::int ELSE -(random() * 20)::int END))
+FROM pairs;
+
+-- Vanquisher-tier matches (value >= threshold), so matchup_vanq_* has data too.
+-- Reuses the same players/chars but overrides value_a/value_b to clear the
+-- Vanquisher floor. Winner gets a higher offset band than the loser, same
+-- reasoning as the regular matches above (non-zero per-match rating change).
+WITH pool AS (
+  SELECT pr.id, pr.char_id, p.name
+  FROM player_ratings pr JOIN players p ON p.id = pr.id
+  WHERE pr.id >= :id_base
+),
+pairs AS (
+  SELECT
+    a.id AS id_a, a.name AS name_a, a.char_id AS char_a,
+    b.id AS id_b, b.name AS name_b, b.char_id AS char_b,
+    (CASE WHEN random() < 0.5 THEN 1 ELSE 2 END) AS winner
+  FROM pool a JOIN pool b ON a.id <> b.id
+  ORDER BY random()
+  LIMIT :vanq_game_count
+)
+INSERT INTO games (timestamp, id_a, name_a, char_a, platform_a, id_b, name_b, char_b, platform_b, winner, game_floor, value_a, value_b)
+SELECT
+  now() - (random() * interval '30 days'),
+  id_a, name_a, char_a, 1,
+  id_b, name_b, char_b, 1,
+  winner,
+  0,
+  :vanq_floor + (CASE WHEN winner = 1 THEN (random() * 100 + 50)::int ELSE (random() * 100)::int END),
+  :vanq_floor + (CASE WHEN winner = 2 THEN (random() * 100 + 50)::int ELSE (random() * 100)::int END)
+FROM pairs;
+
+COMMIT;
+SQL
+
+echo "Seeded $GAME_COUNT ranked games + $VANQ_GAME_COUNT Vanquisher-tier games."

scripts/seed/seed-matchups.sh

@@ -0,0 +1,61 @@
+#!/usr/bin/env bash
+# Requires seed-games.sh to have been run first. Mirrors the aggregation SQL
+# in src/pull.rs update_matchups() so the Redis payload shape matches production.
+set -euo pipefail
+
+REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
+cd "$REPO_ROOT"
+
+PSQL="docker exec -i pf-postgres psql -v ON_ERROR_STOP=1 -U user -d puddle_farm"
+
+ID_BASE=900000000000000
+NUM_CHARS=34
+
+matchup_json() {
+  local char_id="$1"
+  local extra_filter="$2"
+  $PSQL -tAc "
+    SELECT COALESCE(json_agg(json_build_object(
+      'opponent_char', opponent_char, 'wins', wins, 'total_games', total_games
+    )), '[]')
+    FROM (
+      SELECT
+        opponent_char,
+        SUM(CASE WHEN (position = 'a' AND winner = 1) OR (position = 'b' AND winner = 2)
+            THEN 1 ELSE 0 END) AS wins,
+        COUNT(*) AS total_games
+      FROM (
+        SELECT char_b AS opponent_char, winner, 'a' AS position
+        FROM games
+        WHERE char_a = $char_id AND id_a >= $ID_BASE  -- deviation from pull.rs: scope to seeded rows only
+          AND timestamp > now() - interval '1 month'
+          AND game_floor = 0 $extra_filter
+        UNION ALL
+        SELECT char_a AS opponent_char, winner, 'b' AS position
+        FROM games
+        WHERE char_b = $char_id AND id_b >= $ID_BASE  -- deviation from pull.rs: scope to seeded rows only
+          AND timestamp > now() - interval '1 month'
+          AND game_floor = 0 $extra_filter
+      ) combined
+      GROUP BY opponent_char
+      ORDER BY opponent_char
+    ) e;
+  "
+}
+
+GAME_COUNT=$($PSQL -tAc "SELECT count(*) FROM games WHERE id_a >= $ID_BASE;" | tr -d '\r')
+if [ "$GAME_COUNT" -eq 0 ]; then
+  echo "No seeded games found (id_a >= $ID_BASE). Run seed-games.sh first." >&2
+  exit 1
+fi
+
+for c in $(seq 0 $((NUM_CHARS - 1))); do
+  matchup_json "$c" "AND value_a > 0 AND value_b > 0" \
+    | docker exec -i pf-redis redis-cli -x SET "matchup_$c" > /dev/null
+  matchup_json "$c" "AND value_a >= 10001600 AND value_b >= 10001600" \
+    | docker exec -i pf-redis redis-cli -x SET "matchup_vanq_$c" > /dev/null
+done
+
+docker exec pf-redis redis-cli SET last_update_daily "$(date -u +"%Y-%m-%d %H:%M:%S")" > /dev/null
+
+echo "Seeded matchup_0..$((NUM_CHARS - 1)) and matchup_vanq_0..$((NUM_CHARS - 1))."