realm-server: HTTPS+HTTP/2 in local dev

.claude/skills/host-test-memory-leak-hunting/SKILL.md

@@ -66,7 +66,7 @@ Snapshots at `t=10` (warm) and `t=50` give a clean delta over 40 tests. Add `t=9
 ### 3. Open a fresh test tab
 
 ```sh
-ENCODED=$(node -e 'console.log(encodeURIComponent("http://localhost:4200/tests/index.html?hidepassed&filter=card-basics"))')
+ENCODED=$(node -e 'console.log(encodeURIComponent("https://localhost:4200/tests/index.html?hidepassed&filter=card-basics"))')
 curl -sX PUT "http://localhost:9333/json/new?${ENCODED}"
 ```
 

.github/actions/init/action.yml

@@ -31,3 +31,20 @@ runs:
       if: ${{ steps.cache.outputs.cache-hit != 'true' }}
       shell: bash
       run: pnpm store prune
+
+    # Provision the mandatory dev cert so the realm-server can speak
+    # HTTPS+HTTP/2 — the only protocol it supports. Local devs get this
+    # via `mise run infra:ensure-dev-cert`; CI runs it explicitly here
+    # so every downstream job starts with the cert in
+    # ~/.local/share/boxel/dev-certs/ and `NODE_EXTRA_CA_CERTS` pointed
+    # at mkcert's root. See packages/realm-server/server.ts and
+    # mise-tasks/infra/ensure-dev-cert.
+    - name: Install mkcert
+      shell: bash
+      run: |
+        sudo apt-get update
+        sudo apt-get install -y mkcert libnss3-tools
+
+    - name: Provision dev TLS cert
+      shell: bash
+      run: mise run infra:ensure-dev-cert

.github/workflows/ci-host.yaml

@@ -108,18 +108,23 @@ jobs:
 
       - name: Disable TCP/UDP network offloading
         run: sudo ethtool -K eth0 tx off rx off
-      - name: Start test services (icons + host dist + realm servers)
-        run: mise run test-services:host | tee -a /tmp/server.log &
-      - name: Create realm users
-        run: pnpm register-realm-users
-        working-directory: packages/matrix
-
+      # Install + restart dbus/upower BEFORE the test services come up.
+      # `sudo service dbus restart` triggers chrome's NetworkChangeNotifier
+      # in any already-running chromium (the prerender's standby probe and
+      # the realm-server's prerender workers), which aborts every in-flight
+      # h2 stream with ERR_NETWORK_CHANGED and leaves wait-for-host-standby
+      # stuck waiting for #standby-ready that never lands.
       - name: Install D-Bus helpers
         run: |
           sudo apt-get update
           sudo apt-get install -y dbus-x11 upower
           sudo service dbus restart
           sudo service upower restart
+      - name: Start test services (icons + host dist + realm servers)
+        run: mise run test-services:host | tee -a /tmp/server.log &
+      - name: Create realm users
+        run: pnpm register-realm-users
+        working-directory: packages/matrix
 
       - name: Live test suite
         run: dbus-run-session -- pnpm test:live
@@ -170,6 +175,18 @@ jobs:
       # https://github.com/actions/runner-images/issues/1187#issuecomment-686735760
       - name: Disable TCP/UDP network offloading
         run: sudo ethtool -K eth0 tx off rx off
+      # Install + restart dbus/upower BEFORE the test services come up.
+      # `sudo service dbus restart` triggers chrome's NetworkChangeNotifier
+      # in any already-running chromium (the prerender's standby probe and
+      # the realm-server's prerender workers), which aborts every in-flight
+      # h2 stream with ERR_NETWORK_CHANGED and leaves wait-for-host-standby
+      # stuck waiting for #standby-ready that never lands.
+      - name: Install D-Bus helpers
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y dbus-x11 upower
+          sudo service dbus restart
+          sudo service upower restart
       - name: Start test services (icons + host dist + realm servers)
         run: mise run test-services:host | tee -a /tmp/server.log &
         env:
@@ -178,13 +195,6 @@ jobs:
         run: pnpm register-realm-users
         working-directory: packages/matrix
 
-      - name: Install D-Bus helpers
-        run: |
-          sudo apt-get update
-          sudo apt-get install -y dbus-x11 upower
-          sudo service dbus restart
-          sudo service upower restart
-
       - name: host test suite (shard ${{ matrix.shardIndex }})
         run: |
           if [ "$PERCY_ENABLED" = "true" ]; then

.github/workflows/ci-software-factory.yaml

@@ -80,10 +80,17 @@ jobs:
         if: ${{ matrix.shard.index == 1 }}
         run: pnpm test:node
         working-directory: packages/software-factory
-      - name: Serve test assets (icons + host dist)
+      - name: Serve boxel-icons
         run: |
-          mise run ci:serve-test-assets &
-          timeout 180 bash -c 'until curl -sf http://localhost:4200 > /dev/null && curl -sf http://localhost:4206 > /dev/null; do sleep 2; done'
+          # SF Playwright tests use the realm-test-harness, which spins up
+          # vite / realm-server on its own dynamic ports — it is hermetic
+          # by design (see packages/software-factory/docs/testing-strategy.md).
+          # The only external service it expects is the icons server on
+          # ICONS_URL (defaults to http://localhost:4206/). Do not start
+          # host-dist on port 4200 here — it collides with the harness
+          # and masks regressions in the harness's host bring-up code.
+          mise run services:icons &
+          timeout 60 bash -c 'until curl -sf http://localhost:4206 > /dev/null; do sleep 1; done'
       - name: Run Playwright tests
         run: pnpm test:playwright:shard ${{ matrix.shard.index }}/${{ matrix.shard.total }}
         working-directory: packages/software-factory

.github/workflows/ci.yaml

@@ -385,7 +385,7 @@ jobs:
       - name: Start test services (icons + host dist + base realm)
         run: |
           MATRIX_REGISTRATION_SHARED_SECRET='xxxx' mise run test-services:matrix | tee -a /tmp/server.log &
-          timeout 300 bash -c 'until curl -sf http://localhost:4200 > /dev/null && curl -sf http://localhost:4206 > /dev/null; do sleep 2; done'
+          timeout 300 bash -c 'until curl -ksf https://localhost:4200 > /dev/null && curl -sf http://localhost:4206 > /dev/null; do sleep 2; done'
       - name: Run Playwright tests
         run: pnpm test:group ${{ matrix.shardIndex }}/${{ matrix.shardTotal }}
         working-directory: packages/matrix
@@ -906,7 +906,7 @@ jobs:
       - name: Serve test assets (icons + host dist)
         run: |
           mise run ci:serve-test-assets &
-          timeout 180 bash -c 'until curl -sf http://localhost:4200 > /dev/null && curl -sf http://localhost:4206 > /dev/null; do sleep 2; done'
+          timeout 180 bash -c 'until curl -ksf https://localhost:4200 > /dev/null && curl -sf http://localhost:4206 > /dev/null; do sleep 2; done'
       - name: Start PostgreSQL for tests
         run: pnpm start:pg | tee -a /tmp/test-services.log &
         working-directory: packages/realm-server
@@ -980,7 +980,13 @@ jobs:
       - name: Start dev stack (icons + host-dist + base realm + prerenderer)
         run: |
           mise run test-services:matrix | tee -a /tmp/server.log &
-          timeout 600 bash -c 'until curl -sf http://localhost:4200 > /dev/null && curl -sf -H "Accept: application/vnd.api+json" http://localhost:4201/base/_readiness-check > /dev/null; do sleep 2; done'
+          # `-k` skips cert verification (the local realm-server speaks
+          # HTTPS with the mkcert leaf cert). `-w '%{http_code}'` + grep
+          # ensures we treat 3xx as still-not-ready instead of letting
+          # `-f` return 0 on the dispatcher's redirect response, which
+          # would race the tests ahead of the base realm finishing its
+          # initial index.
+          timeout 600 bash -c 'until curl -sk -o /dev/null -w "%{http_code}" https://localhost:4200/ | grep -qx 200 && curl -sk -o /dev/null -w "%{http_code}" -H "Accept: application/vnd.api+json" https://localhost:4201/base/_readiness-check | grep -qx 200; do sleep 2; done'
       - name: Run integration tests
         run: pnpm test:integration
         working-directory: packages/boxel-cli

AGENTS.md

@@ -94,10 +94,10 @@
 
 #### Iterating on host tests with the Chrome MCP server
 
-- Start the host app so qunit test runner is available at `http://localhost:4200/tests` (usual `pnpm start` + dependencies).
+- Start the host app so qunit test runner is available at `https://localhost:4200/tests` (usual `pnpm start` + dependencies).
 - Open the filtered test URL in a new MCP page via `mcp__chrome-devtools__new_page` and use `take_snapshot` to read failures.
-- Filtered URL structure: `http://localhost:4200/tests?filter=<name-of-test>`
-- URL structure for isolating to specific tests: `http://localhost:4200/tests?moduleId=<module-id>&testId=<test-id>&testId=...` (visible on the “Rerun” links for failing tests).
+- Filtered URL structure: `https://localhost:4200/tests?filter=<name-of-test>`
+- URL structure for isolating to specific tests: `https://localhost:4200/tests?moduleId=<module-id>&testId=<test-id>&testId=...` (visible on the “Rerun” links for failing tests).
 - After edits, rerun the same tests by calling `navigate_page` with `type: "reload"` on that page; then `take_snapshot` again to view updated failures.
 - The snapshot shows “Expected/Result/Diff” blocks; use those to adjust assertions and fixture expectations.
 - Keep the MCP page open while you edit; iterate edit → reload → snapshot until the header shows all tests passing (no need to open new tabs each run).

QUICKSTART.md

@@ -2,10 +2,17 @@
 
 To build the entire repository and run the application, follow these steps:
 
-1. The 2 main system dependencies to install are:
-
+1. The system dependencies to install are:
    - [mise](https://mise.jdx.dev/getting-started.html)
    - [docker](https://docs.docker.com/get-docker/)
+   - [mkcert](https://github.com/FiloSottile/mkcert) — provisions the
+     local TLS cert the realm-server needs to speak HTTPS+HTTP/2 (local
+     dev has no HTTP fallback). Install with
+     `sudo apt install -y mkcert libnss3-tools` on Debian/Ubuntu or
+     `brew install mkcert nss` on macOS. After install, run
+     `mise run infra:ensure-dev-cert` once before the first
+     `mise run dev` / `pnpm start:all`; subsequent runs are a no-op. See
+     the repo-root [README](README.md#local-https-dev-access) for details.
 
 2. Clone the repo:
 
@@ -52,7 +59,7 @@ To build the entire repository and run the application, follow these steps:
    Note: Ensure that the realm-server is completely started by looking out for tor the test-realm indexing output.
 
    ```zsh
-   Realm http://localhost:4202/test/ has started ({
+   Realm https://localhost:4202/test/ has started ({
    "instancesIndexed": 8,
    "instanceErrors": 0,
    "moduleErrors": 0
@@ -76,19 +83,16 @@ To build the entire repository and run the application, follow these steps:
    Visit http://localhost:8080. Type in Username = "admin", Password: "password" Homeserver URL: http://localhost:8008
 
 10. Host App
-
-    - Visit http://localhost:4201/
+    - Visit https://localhost:4200/
     - Enter the registration flow and create a Boxel Account
     - When prompted for an authentication token, type in "dev-token"
 
 11. Validate email for login
-
     - Visit SMTP UI at http://localhost:5001/
     - Validate email
-    - Go back to Host http://localhost:4201/ and login
+    - Go back to Host https://localhost:4200/ and login
 
 12. Perform "Setup up Secure Payment Method" flow
-
     - More detailed steps can be found in our [README](README.md) Payment Setup section
 
 13. Run ai bot (Optional):