cardstack · habdelra · May 12, 2026 · May 12, 2026 · May 12, 2026 · May 12, 2026
diff --git a/.claude/skills/indexing-diagnostics/SKILL.md b/.claude/skills/indexing-diagnostics/SKILL.md
diff --git a/QUICKSTART.md b/QUICKSTART.md
@@ -3,7 +3,6 @@
 To build the entire repository and run the application, follow these steps:
 
 1. The 2 main system dependencies to install are:
-
    - [mise](https://mise.jdx.dev/getting-started.html)
    - [docker](https://docs.docker.com/get-docker/)
 
@@ -76,19 +75,16 @@ To build the entire repository and run the application, follow these steps:
    Visit http://localhost:8080. Type in Username = "admin", Password: "password" Homeserver URL: http://localhost:8008
 
 10. Host App
-
-    - Visit http://localhost:4201/
+    - Visit https://localhost:4201/
     - Enter the registration flow and create a Boxel Account
     - When prompted for an authentication token, type in "dev-token"
 
 11. Validate email for login
-
     - Visit SMTP UI at http://localhost:5001/
     - Validate email
-    - Go back to Host http://localhost:4201/ and login
+    - Go back to Host https://localhost:4201/ and login
 
 12. Perform "Setup up Secure Payment Method" flow
-
     - More detailed steps can be found in our [README](README.md) Payment Setup section
 
 13. Run ai bot (Optional):

diff --git a/README.md b/README.md
@@ -34,10 +34,10 @@ For a quickstart, see [here](./QUICKSTART.md)
 
 Two catalog realms run side by side. The **Cardstack Catalog** (served from [cardstack/boxel-catalog](https://github.com/cardstack/boxel-catalog)) is the source of truth for new development and the destination for community submissions. The **Legacy Catalog** (shipped from this monorepo) remains available during the deprecation window for content that hasn't been migrated upstream.
 
-| Realm                  | Source                                                                                  | URL path           | Purpose                                                                                                                          |
-| ---------------------- | --------------------------------------------------------------------------------------- | ------------------ | -------------------------------------------------------------------------------------------------------------------------------- |
-| **Cardstack Catalog**  | `packages/catalog` (clones [boxel-catalog](https://github.com/cardstack/boxel-catalog)) | `/catalog/`        | Official catalog. New listings and community submissions land here via `pr-listing-create` PRs to `cardstack/boxel-catalog`.     |
-| **Legacy Catalog**     | `packages/catalog-realm`                                                                | `/legacy-catalog/` | Historical catalog shipped from this repo. Kept visible in the workspace chooser while existing content migrates upstream.       |
+| Realm                 | Source                                                                                  | URL path           | Purpose                                                                                                                      |
+| --------------------- | --------------------------------------------------------------------------------------- | ------------------ | ---------------------------------------------------------------------------------------------------------------------------- |
+| **Cardstack Catalog** | `packages/catalog` (clones [boxel-catalog](https://github.com/cardstack/boxel-catalog)) | `/catalog/`        | Official catalog. New listings and community submissions land here via `pr-listing-create` PRs to `cardstack/boxel-catalog`. |
+| **Legacy Catalog**    | `packages/catalog-realm`                                                                | `/legacy-catalog/` | Historical catalog shipped from this repo. Kept visible in the workspace chooser while existing content migrates upstream.   |
 
 Both catalogs appear in the workspace chooser by default; the Cardstack Catalog is sorted first, Legacy Catalog last. Both are controlled by the same `SKIP_CATALOG` flag — setting `SKIP_CATALOG=true` skips setup and startup for both.
 
@@ -137,7 +137,7 @@ In order to run the realm server hosted app:
 1. `mise run services:host-build` to re-build the host app (this step can be omitted if you do not want host app re-builds)
 2. `mise run dev` to serve the base and experiments realms
 
-You can visit the URL of each realm server to view that realm's app. So for instance, the base realm's app is available at `http://localhost:4201/base` and the experiments realm's app is at `http://localhost:4201/experiments`.
+You can visit the URL of each realm server to view that realm's app. So for instance, the base realm's app is available at `https://localhost:4201/base` and the experiments realm's app is at `https://localhost:4201/experiments`. (Local dev now serves HTTPS+HTTP/2 — see "Local HTTPS dev access" below for the one-time cert setup.)
 
 Live reloads are not available in this mode, however, if you use start the server with the environment variable `DISABLE_MODULE_CACHING=true` you can just refresh the page to grab the latest code changes if you are running rebuilds (step #1 and #2 above).
 
@@ -191,10 +191,10 @@ Here's what is spun up with `mise run dev`:
 
 | Port  | Description                                                                                   | `mise run dev` | `mise run services:realm-server-base` |
 | ----- | --------------------------------------------------------------------------------------------- | -------------- | ------------------------------------- |
-| :4201 | `/base` base realm                                                                            | ✅             | ✅                                    |
+| :4201 | `/base` base realm (HTTPS+HTTP/2; see "Local HTTPS dev access" below)                         | ✅             | ✅                                    |
 | :4201 | `/skills` skills realm                                                                        | ✅             | 🚫                                    |
 | :4201 | `/experiments` experiments realm                                                              | ✅             | 🚫                                    |
-| :4202 | `/test` host test realm, `/node-test` node test realm                                         | ✅             | 🚫                                    |
+| :4202 | `/test` host test realm, `/node-test` node test realm (HTTPS+HTTP/2)                          | ✅             | 🚫                                    |
 | :4205 | `/test` realm for matrix client tests (playwright controlled)                                 | 🚫             | 🚫                                    |
 | :4206 | Boxel icons server                                                                            | ✅             | 🚫                                    |
 | :4210 | Development Worker Manager (spins up 1 worker by default)                                     | ✅             | 🚫                                    |
@@ -207,9 +207,102 @@ Here's what is spun up with `mise run dev`:
 | :5435 | Postgres DB                                                                                   | ✅             | 🚫                                    |
 | :8008 | Matrix synapse server                                                                         | ✅             | 🚫                                    |
 
+#### Local HTTPS dev access
+
+##### Why
+
+Heavy aggregator cards (Cohort, dashboards) fan out 80+ federated-search
+requests per render. Chrome — including the headless Chromium that
+drives the prerender — caps any single origin at 6 concurrent HTTP/1.1
+connections, so the 80+ requests serialize. A single cohort render
+takes ~4–5 minutes under that ceiling. HTTP/2 multiplexes them over one
+connection and the same render finishes in seconds. Browsers only do
+HTTP/2 over TLS, so the local realm-server terminates a leaf cert.
+
+##### Setup
+
+```
+mise run infra:ensure-dev-cert
+```
+
+That task:
+
+1. Requires [`mkcert`](https://github.com/FiloSottile/mkcert)
+   (`sudo apt install -y mkcert libnss3-tools` on Debian/Ubuntu;
+   `brew install mkcert nss` on macOS). If it's missing, the task
+   prints these instructions and exits non-zero — local dev now
+   speaks HTTPS only and has no HTTP fallback.
+2. Attempts `mkcert -install` once (one-time sudo prompt) so mkcert's
+   root CA lands in your system trust store. Your normal browser then
+   silently accepts the cert. If you decline the sudo prompt the cert
+   is still generated, indexing keeps working, and you can run
+   `mkcert -install` later when convenient.
+3. Generates `~/.local/share/boxel/dev-certs/{localhost.pem,
+localhost-key.pem}`. Idempotent — re-runs are a no-op until the
+   cert is within 7 days of expiry.
+
+After provisioning, `mise run dev` (and `mise run dev-all`) automatically
+brings the realm-server up on `https://localhost:4201` (and
+`https://localhost:4202` for the test-realms server). Node clients
+(worker, scripts, prerender Node-side) trust the cert via
+`NODE_EXTRA_CA_CERTS` (pointed at mkcert's root by `env-vars.sh`); the
+prerender Chromium uses `--ignore-certificate-errors` for belt-and-
+suspenders coverage.
+
+##### Migration after pulling this change
+
+The canonical realm URLs are now `https://localhost:4201/…` and
+`https://localhost:4202/…`, so every row that was written under the old
+`http://localhost:42XX/…` canonical needs its URLs rewritten in place
+— PK columns, FK columns, JSONB documents (`pristine_doc`, `search_doc`,
+`error_doc`, `deps`, `value`, `headers`, etc.), and rendered HTML /
+markdown payloads. The repo ships an auto-run migration that handles
+all of it:
+
+- `pnpm migrate` (which `mise run dev` runs via `--migrateDB`) picks up
+  `packages/postgres/migrations/1779200000000_canonical-url-http-to-https.js`
+  on the next boot.
+- That migration walks `information_schema.columns`, finds every
+  text/varchar/jsonb column on every public table (skipping `modules`,
+  which the realm-server truncates on startup anyway), and runs an
+  in-place `REPLACE` for the two canonicals. WHERE-filtered so it only
+  touches rows that still contain the old URL — idempotent, and a
+  no-op in production (where the canonical URL is never `localhost`).
+
+After the migration runs, the realm-server boots normally on
+`https://localhost:4201/`. The same-port HTTP→HTTPS dispatcher catches
+any lingering `http://localhost:4201/…` requests (e.g. you typed it
+into a browser, or a card still has a stale URL in a rendered HTML
+attribute) and 301-redirects to the canonical https origin.
+
+Personal realm files under `realms/localhost_4201/**/*.json` may still
+have `id`/`relationships` URLs spelled `http://localhost:4201/…`. The
+indexer re-derives canonical URLs from the realm mount root, so those
+files index cleanly under the new canonical and the redirect handles
+runtime fetches; cleaning up the on-disk strings is optional (a
+`sed -i 's|http://localhost:4201|https://localhost:4201|g'` across the
+realm dir does it in one shot).
+
+##### Verify
+
+```
+curl -kI --http2 https://localhost:4201/_alive
+```
+
+Look for `HTTP/2 200`. The `mise run dev` log also confirms with
+`Realm server listening on port 4201 (https/h2)`.
+
+##### CI / hermetic test harness
+
+`env-vars.sh` exports the cert env vars only when the cert files exist
+under `~/.local/share/boxel/dev-certs/`. CI images and the software-
+factory hermetic harness do not provision the cert, so realm-server
+boots HTTP/1.1 on `http://localhost:4201/…` exactly as before. This
+keeps the test harness path unchanged.
+
 #### Using `mise run services:realm-server`
 
-You can also use `mise run services:realm-server` if you want the functionality of `mise run dev`, but without running the test realms. This will enable you to open http://localhost:4201 and allow to select between the cards in the /base and /experiments realm. You must also make sure to run `mise run services:worker` in order to start the workers which are normally started in `mise run dev`.
+You can also use `mise run services:realm-server` if you want the functionality of `mise run dev`, but without running the test realms. This will enable you to open https://localhost:4201 and allow to select between the cards in the /base and /experiments realm. You must also make sure to run `mise run services:worker` in order to start the workers which are normally started in `mise run dev`.
 
 #### Indexing dashboard
 
@@ -610,12 +703,12 @@ BOXEL_ENVIRONMENT=parallel mise run services:ai-bot
 
 In environment mode, services are available at:
 
-| Service       | Hostname                                  |
-| ------------- | ----------------------------------------- |
-| Host app      | `http://host.<slug>.localhost`            |
-| Realm server  | `http://realm-server.<slug>.localhost`    |
-| Matrix        | `http://matrix.<slug>.localhost`          |
-| Icons         | `http://icons.<slug>.localhost`           |
+| Service      | Hostname                               |
+| ------------ | -------------------------------------- |
+| Host app     | `http://host.<slug>.localhost`         |
+| Realm server | `http://realm-server.<slug>.localhost` |
+| Matrix       | `http://matrix.<slug>.localhost`       |
+| Icons        | `http://icons.<slug>.localhost`        |
 
 Where `<slug>` is the lowercased, sanitized form of `BOXEL_ENVIRONMENT` (e.g., `feature/my-branch` becomes `feature-my-branch`).
 

diff --git a/mise-tasks/infra/ensure-dev-cert b/mise-tasks/infra/ensure-dev-cert
@@ -0,0 +1,90 @@
+#!/usr/bin/env bash
+#MISE description="Provision a local dev TLS cert so realm-server can speak HTTP/2"
+#
+# Generates ~/.local/share/boxel/dev-certs/{localhost.pem, localhost-key.pem}
+# using mkcert. Idempotent — re-runs are a no-op if the cert already exists
+# and isn't near expiry.
+#
+# Why this exists
+# ---------------
+# Heavy aggregator-card prerenders (Cohort, dashboards) fan out 80+
+# federated-search requests in a single Chromium tab. Chrome's HTTP/1.1
+# per-origin connection limit (6) serializes them and turns one cohort
+# render into multiple minutes. HTTP/2 multiplexes them over a single
+# TCP connection and the same render finishes in seconds.
+#
+# Browsers only do HTTP/2 over TLS, so the realm-server has to terminate
+# a cert. We use mkcert because:
+#   1. It can install a local CA into the system trust store (one-time,
+#      requires sudo) so the dev's normal browser and command-line tools
+#      silently accept the cert.
+#   2. The leaf cert it generates is short-lived and replaceable; it
+#      lives under the dev's home dir and is never committed.
+#
+# This task tries `mkcert -install` automatically — if it succeeds (or
+# was already done), the dev's whole local toolchain trusts the cert.
+# If the sudo prompt is declined, we still generate the cert so the
+# server can boot; only the manual browser keeps showing warnings until
+# the dev opts in.
+
+set -euo pipefail
+
+CERT_DIR="${BOXEL_DEV_CERT_DIR:-$HOME/.local/share/boxel/dev-certs}"
+CERT_FILE="$CERT_DIR/localhost.pem"
+KEY_FILE="$CERT_DIR/localhost-key.pem"
+
+# Idempotent skip when the cert already exists and isn't within 7 days of
+# expiry. openssl's `-checkend` returns 0 if the cert is valid for at least
+# the given number of seconds.
+if [ -f "$CERT_FILE" ] && [ -f "$KEY_FILE" ]; then
+  if openssl x509 -in "$CERT_FILE" -checkend $((7 * 24 * 60 * 60)) -noout >/dev/null 2>&1; then
+    exit 0
+  fi
+  echo "[ensure-dev-cert] Existing cert at $CERT_FILE is near expiry; regenerating."
+fi
+
+if ! command -v mkcert >/dev/null 2>&1; then
+  cat >&2 <<'EOF'
+[ensure-dev-cert] mkcert not installed — skipping HTTPS cert provisioning.
+The realm-server's `listen()` falls back to plain HTTP when the cert
+env vars are unset, so this is a soft warning (CI and any environment
+without mkcert keep working unchanged). Local dev still gets the
+HTTPS/HTTP-2 throughput win by installing mkcert and re-running:
+
+  Linux (Debian/Ubuntu): sudo apt install -y mkcert libnss3-tools
+  Linux (Fedora/RHEL):   sudo dnf install -y mkcert nss-tools
+  macOS (Homebrew):      brew install mkcert nss
+
+Then: `mise run infra:ensure-dev-cert`
+
+See the repo-root README ("Local HTTPS dev access") for the why.
+EOF
+  exit 0
+fi
+
+mkdir -p "$CERT_DIR"
+
+# Best-effort trust install. On a fresh machine this prompts for sudo
+# once and adds mkcert's root CA to /etc/ssl/certs (Linux) or the system
+# keychain (macOS). On subsequent runs it's a no-op. If the prompt is
+# declined, we still proceed — the cert is generated, indexing works
+# via puppeteer's --ignore-certificate-errors flag and Node clients
+# via NODE_EXTRA_CA_CERTS, and only the manual browser keeps showing
+# warnings until the dev opts in later.
+if ! mkcert -install 2>/dev/null; then
+  cat >&2 <<'EOF'
+[ensure-dev-cert] `mkcert -install` did not complete (sudo declined or
+not available). The cert is still generated below; indexing and Node
+clients pick up the local CA via NODE_EXTRA_CA_CERTS. To silence cert
+warnings in your manual browser later, run:
+
+  mkcert -install      # one-time, requires sudo
+
+EOF
+fi
+
+echo "[ensure-dev-cert] Generating cert at $CERT_FILE"
+mkcert \
+  -cert-file "$CERT_FILE" \
+  -key-file "$KEY_FILE" \
+  localhost 127.0.0.1 ::1
diff --git a/mise-tasks/lib/dev-common.sh b/mise-tasks/lib/dev-common.sh
@@ -243,27 +243,40 @@ sweep_orphaned_services() {
 
 READY_PATH="_readiness-check?acceptHeader=application%2Fvnd.api%2Bjson"
 
+# Pick wait-on's protocol prefix based on the realm-server's scheme. Local
+# dev runs HTTPS+HTTP/2 by default; tests/CI fall back to plain HTTP when
+# `infra:ensure-dev-cert` hasn't run. `${REALM_BASE_URL#*://}` strips
+# whichever scheme is in use to feed wait-on's authority-only form.
+case "$REALM_BASE_URL" in
+  https://*) REALM_READY_SCHEME="https-get" ;;
+  *)         REALM_READY_SCHEME="http-get"  ;;
+esac
+case "$REALM_TEST_URL" in
+  https://*) REALM_TEST_READY_SCHEME="https-get" ;;
+  *)         REALM_TEST_READY_SCHEME="http-get"  ;;
+esac
+
 # Phase 1 readiness URLs
-BASE_REALM_READY="http-get://${REALM_BASE_URL#http://}/base/${READY_PATH}"
-SKILLS_READY="http-get://${REALM_BASE_URL#http://}/skills/${READY_PATH}"
+BASE_REALM_READY="${REALM_READY_SCHEME}://${REALM_BASE_URL#*://}/base/${READY_PATH}"
+SKILLS_READY="${REALM_READY_SCHEME}://${REALM_BASE_URL#*://}/skills/${READY_PATH}"
 PHASE1_URLS="${BASE_REALM_READY}|${SKILLS_READY}"
 
 if [ -z "${SKIP_CATALOG:-}" ]; then
-  PHASE1_URLS="${PHASE1_URLS}|http-get://${REALM_BASE_URL#http://}/catalog/${READY_PATH}"
-  PHASE1_URLS="${PHASE1_URLS}|http-get://${REALM_BASE_URL#http://}/legacy-catalog/${READY_PATH}"
+  PHASE1_URLS="${PHASE1_URLS}|${REALM_READY_SCHEME}://${REALM_BASE_URL#*://}/catalog/${READY_PATH}"
+  PHASE1_URLS="${PHASE1_URLS}|${REALM_READY_SCHEME}://${REALM_BASE_URL#*://}/legacy-catalog/${READY_PATH}"
 fi
 if [ -z "${SKIP_BOXEL_HOMEPAGE:-}" ]; then
-  PHASE1_URLS="${PHASE1_URLS}|http-get://${REALM_BASE_URL#http://}/boxel-homepage/${READY_PATH}"
+  PHASE1_URLS="${PHASE1_URLS}|${REALM_READY_SCHEME}://${REALM_BASE_URL#*://}/boxel-homepage/${READY_PATH}"
 fi
 if [ -z "${SKIP_EXPERIMENTS:-}" ]; then
-  PHASE1_URLS="${PHASE1_URLS}|http-get://${REALM_BASE_URL#http://}/experiments/${READY_PATH}"
+  PHASE1_URLS="${PHASE1_URLS}|${REALM_READY_SCHEME}://${REALM_BASE_URL#*://}/experiments/${READY_PATH}"
 fi
-PHASE1_URLS="${PHASE1_URLS}|http-get://${REALM_BASE_URL#http://}/software-factory/${READY_PATH}"
+PHASE1_URLS="${PHASE1_URLS}|${REALM_READY_SCHEME}://${REALM_BASE_URL#*://}/software-factory/${READY_PATH}"
 
 PHASE1_URLS="${PHASE1_URLS}|${MATRIX_URL_VAL}|http://localhost:5001|${ICONS_URL}"
 
 # Phase 2 readiness URL
-NODE_TEST_REALM_READY="http-get://${REALM_TEST_URL#http://}/node-test/${READY_PATH}"
+NODE_TEST_REALM_READY="${REALM_TEST_READY_SCHEME}://${REALM_TEST_URL#*://}/node-test/${READY_PATH}"
 
 # In environment mode, bootstrap infra before starting services
 if [ -n "$BOXEL_ENVIRONMENT" ]; then