forked from jupyter-book/mystmd
-
Notifications
You must be signed in to change notification settings - Fork 0
chore: add quantecon/ fork maintenance tooling #14
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
3713616
chore: add quantecon/ fork maintenance tooling
mmcky d86e080
chore: disable release and docs workflows on QuantEcon fork
mmcky cd85f46
chore: update sync docs to use git merge (not --ff-only) for fork main
mmcky 7782156
docs: rewrite quantecon/README with clearer workflow explanation
mmcky 8a8af58
chore: revert README banner and workflow changes from PR
mmcky e0f9919
fix: address copilot review comments on build.sh
mmcky 935442f
docs: branch features from upstream/main to keep PR diffs clean
mmcky 0db57e4
Merge branch 'main' into quantecon/fork-maintenance
mmcky File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,175 @@ | ||
| # QuantEcon fork of `mystmd` — maintenance guide | ||
|
|
||
| This fork lets QuantEcon develop and use new `mystmd` features before they have been reviewed and merged by the upstream `jupyter-book/mystmd` team. The goal is to make it as easy as possible to open upstream PRs from feature branches, while also being able to run a combined build that includes all in-progress features. | ||
|
|
||
| ## How it works — the key idea | ||
|
|
||
| ``` | ||
| jupyter-book/mystmd:main ─────┐ | ||
| │ (sync periodically) │ (feature branches start here) | ||
| ▼ │ | ||
| QuantEcon/mystmd:main │ ← mirror + quantecon/ tooling | ||
| │ │ | ||
| │ feature/foo ◄─────┤ ← branched from upstream/main; PR target = upstream main | ||
| │ feature/bar ◄─────┘ ← branched from upstream/main; PR target = upstream main | ||
| ▼ (built by build.sh, which merges feature branches into quantecon) | ||
| QuantEcon/mystmd:quantecon ← combined build: main + all feature branches | ||
| ``` | ||
|
|
||
| **Feature branches serve two purposes simultaneously:** | ||
|
|
||
| 1. They are the source of upstream PRs — targeting `jupyter-book/mystmd:main`, showing only the diff for that one feature. They are branched from `upstream/main` so they don't carry the `quantecon/` folder. | ||
| 2. They are consumed by `build.sh`, which merges them onto the throwaway `quantecon` branch (which inherits the `quantecon/` folder from `main`). | ||
|
|
||
| This means **you never need to choose** between "make it easy to upstream" and "make it available to use now". The feature branch does both. | ||
|
|
||
| ## Branching model | ||
|
|
||
| | Branch | Purpose | | ||
| |---|---| | ||
| | `main` | Mirrors `jupyter-book/mystmd:main` exactly. Synced via the GitHub "Sync fork" button or `git merge upstream/main`. **No direct commits.** | | ||
| | `feature/<name>` | One branch per logical patch. **Branched from `upstream/main`** (not `main`), kept rebased on `upstream/main`. This is the branch you open the upstream PR from. | | ||
| | `quantecon` | Throwaway combined build. Rebuilt by `build.sh` as `main` + all active feature branches. **Never commit here directly — it is always discarded and regenerated.** | | ||
|
|
||
| ## One-time setup | ||
|
|
||
| ```bash | ||
| git remote add upstream https://github.com/jupyter-book/mystmd.git | ||
| ``` | ||
|
|
||
| Verify your remotes look like this: | ||
|
|
||
| ``` | ||
| origin https://github.com/QuantEcon/mystmd.git (fetch/push) | ||
| upstream https://github.com/jupyter-book/mystmd.git (fetch/push) | ||
| ``` | ||
|
|
||
| ## Regular workflow | ||
|
|
||
| ### Sync `main` with upstream | ||
|
|
||
| Either use the **"Sync fork"** button on the GitHub web UI (simplest), or locally: | ||
|
|
||
| ```bash | ||
| git fetch upstream | ||
| git checkout main | ||
| git merge upstream/main | ||
| git push origin main | ||
| ``` | ||
|
|
||
| After syncing, rebuild the `quantecon` branch (see below) so it includes the latest upstream changes. | ||
|
|
||
| ### Develop a new feature | ||
|
|
||
| > **Important:** branch from `upstream/main`, **not** from `main`. This keeps the `quantecon/` folder out of your feature branch, so the upstream PR diff shows only your actual changes. | ||
|
|
||
| ```bash | ||
| git fetch upstream | ||
| git checkout -b feature/<name> upstream/main | ||
| # make your changes and commit them | ||
| git push origin feature/<name> | ||
| ``` | ||
|
|
||
| Then: | ||
| 1. Add the branch name to `quantecon/features.txt` (on `main`, then commit and push) | ||
| 2. Open a PR on GitHub: **base: `jupyter-book/mystmd:main`**, **compare: `QuantEcon/mystmd:feature/<name>`** | ||
| 3. Rebuild the `quantecon` branch so the feature is available immediately | ||
|
|
||
| > **Why this works:** `build.sh` runs from `main` (which has the `quantecon/` folder), checks out the `quantecon` branch, and merges your feature branch into it. Because git merges *changes* (not full trees), the feature branch doesn't need to contain `quantecon/` — the folder comes from `main`, and your feature's changes are layered on top. | ||
|
|
||
| ### Migrating an existing feature branch to use upstream/main as base | ||
|
|
||
| If you already have a feature branch that was created from `main` (and therefore includes the `quantecon/` folder commits), rebase it onto `upstream/main`: | ||
|
|
||
| ```bash | ||
| git fetch upstream | ||
| git checkout feature/<name> | ||
| git rebase --onto upstream/main main | ||
| git push --force-with-lease origin feature/<name> | ||
| ``` | ||
|
|
||
| This replays only your feature's commits on top of `upstream/main`, dropping the `quantecon/` folder commits from the branch's history. The upstream PR diff will then show only your actual changes. | ||
|
|
||
| ### Rebuild the `quantecon` branch | ||
|
|
||
| ```bash | ||
| ./quantecon/build.sh # local only (safe, no push) | ||
| ./quantecon/build.sh --push # build and push to origin/quantecon | ||
| ``` | ||
|
|
||
| The script: | ||
| 1. Syncs local `main` from `origin/main` | ||
| 2. Resets `quantecon` to `main` (discarding any previous build) | ||
| 3. Merges each branch listed in `features.txt` in order (merge commits, not squash) | ||
| 4. Patches `packages/mystmd/package.json` version to append `-qe` (e.g. `1.9.0-qe`). The `copy:version` build step propagates this to `version.ts` at build time, so `myst --version` identifies the QuantEcon build. | ||
| 5. Optionally pushes to `origin/quantecon` | ||
|
|
||
| Run this after every upstream sync, after updating any feature branch, or after adding/removing a feature branch from `features.txt`. | ||
|
|
||
| ### Update a feature branch (e.g. to address upstream reviewer feedback) | ||
|
|
||
| ```bash | ||
| git checkout feature/<name> | ||
| # make changes, amend or add commits | ||
| git push --force-with-lease origin feature/<name> | ||
| # the upstream PR updates automatically | ||
| ./quantecon/build.sh --push # rebuild quantecon with the updated branch | ||
| ``` | ||
|
|
||
| ### Keep a feature branch current with upstream | ||
|
|
||
| If `upstream/main` has moved since you branched: | ||
|
|
||
| ```bash | ||
| git fetch upstream | ||
| git checkout feature/<name> | ||
| git rebase upstream/main | ||
| git push --force-with-lease origin feature/<name> | ||
| ./quantecon/build.sh --push | ||
| ``` | ||
|
|
||
| ## Resolving merge conflicts | ||
|
|
||
| Conflicts are **always fixed on the feature branch**, never on `quantecon` (which is throwaway and rebuilt from scratch each time). | ||
|
|
||
| `build.sh` will abort cleanly and tell you which branch caused the conflict. | ||
|
|
||
| **Feature conflicts with upstream** (upstream changed code the feature touches): | ||
| ```bash | ||
| git fetch upstream | ||
| git checkout feature/<name> | ||
| git rebase upstream/main # resolve conflicts here, then: git add . && git rebase --continue | ||
| git push --force-with-lease origin feature/<name> | ||
| ./quantecon/build.sh --push | ||
| ``` | ||
|
|
||
| **Feature A conflicts with feature B** (two patches touch the same lines): | ||
| ```bash | ||
| # Rebase the later branch on top of the earlier one to establish ordering | ||
| git checkout feature/<name-b> | ||
| git rebase feature/<name-a> | ||
| git push --force-with-lease origin feature/<name-b> | ||
| # Make sure features.txt lists feature/<name-a> before feature/<name-b> | ||
| ./quantecon/build.sh --push | ||
| ``` | ||
|
|
||
| ## When upstream merges a feature | ||
|
|
||
| Once the upstream PR is merged into `jupyter-book/mystmd:main`: | ||
|
|
||
| 1. Sync `main` with upstream — it now contains the feature | ||
| 2. Remove the branch from `features.txt` | ||
| 3. Delete the feature branch | ||
| 4. Rebuild `quantecon` — the feature is now in `main` itself, so nothing is lost | ||
|
|
||
| ```bash | ||
| git fetch upstream | ||
| git checkout main && git merge upstream/main && git push origin main | ||
| # edit features.txt to remove the branch | ||
| git branch -d feature/<name> && git push origin --delete feature/<name> | ||
| ./quantecon/build.sh --push | ||
| ``` | ||
|
|
||
| ## Active patches | ||
|
|
||
| See [features.txt](features.txt) for the current list of carry-patches and links to their upstream PRs. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,199 @@ | ||
| #!/usr/bin/env bash | ||
| # quantecon/build.sh | ||
| # | ||
| # Rebuilds the `quantecon` integration branch by merging all active feature | ||
| # branches (listed in quantecon/features.txt) on top of current `main`. | ||
| # | ||
| # Usage: | ||
| # ./quantecon/build.sh [--push] | ||
| # | ||
| # Options: | ||
| # --push Push the resulting branch to origin after a successful build. | ||
| # | ||
| # Conflict resolution: | ||
| # This script aborts cleanly on any merge conflict. Conflicts must be | ||
| # resolved on the feature branch itself (never on `quantecon`): | ||
| # | ||
| # Feature vs. main: | ||
| # git checkout <branch> && git rebase main | ||
| # # resolve conflicts, git add, git rebase --continue | ||
| # git push --force-with-lease origin <branch> | ||
| # ./quantecon/build.sh [--push] | ||
| # | ||
| # Feature A vs. feature B: | ||
| # git checkout <branch-B> && git rebase <branch-A> | ||
| # # resolve conflicts, update features.txt so branch-A appears before branch-B | ||
| # git push --force-with-lease origin <branch-B> | ||
| # ./quantecon/build.sh [--push] | ||
|
|
||
| set -euo pipefail | ||
|
|
||
| SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" | ||
| FEATURES_FILE="$SCRIPT_DIR/features.txt" | ||
| INTEGRATION_BRANCH="quantecon" | ||
| BASE_BRANCH="main" | ||
| PUSH=false | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # Parse args | ||
| # --------------------------------------------------------------------------- | ||
| for arg in "$@"; do | ||
| case "$arg" in | ||
| --push) PUSH=true ;; | ||
| *) echo "Unknown argument: $arg" >&2; exit 1 ;; | ||
| esac | ||
| done | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # Preflight checks | ||
| # --------------------------------------------------------------------------- | ||
| if ! git rev-parse --git-dir > /dev/null 2>&1; then | ||
| echo "ERROR: not inside a git repository." >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| if [[ -n "$(git status --porcelain)" ]]; then | ||
| echo "ERROR: working tree is dirty. Commit or stash changes before running." >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| # Read feature branches (strip comments and blank lines, extract first token) | ||
| mapfile -t FEATURES < <(awk '/^[[:space:]]*#/{next} /^[[:space:]]*$/{next} {print $1}' "$FEATURES_FILE") | ||
|
|
||
| if [[ ${#FEATURES[@]} -eq 0 ]]; then | ||
| echo "No feature branches listed in $FEATURES_FILE. Nothing to do." | ||
| exit 0 | ||
| fi | ||
|
|
||
| echo "==> Feature branches to merge:" | ||
| for f in "${FEATURES[@]}"; do | ||
| echo " $f" | ||
| done | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # Ensure local main is up to date | ||
| # --------------------------------------------------------------------------- | ||
| echo "" | ||
| echo "==> Fetching origin..." | ||
| git fetch origin | ||
|
|
||
| echo "" | ||
| echo "==> Switching to $BASE_BRANCH and fast-forwarding from origin..." | ||
| git checkout "$BASE_BRANCH" | ||
| # main should be a pure upstream mirror — ff-only guards against accidental commits | ||
| if ! git merge --ff-only origin/"$BASE_BRANCH"; then | ||
| echo "ERROR: could not fast-forward $BASE_BRANCH from origin/$BASE_BRANCH." \ | ||
| "Sync main with upstream first (git merge upstream/main && git push origin main)." >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| BASE_SHA=$(git rev-parse "$BASE_BRANCH") | ||
| echo " Base commit: $BASE_SHA" | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # Verify all feature branches exist (locally or on origin) | ||
| # --------------------------------------------------------------------------- | ||
| echo "" | ||
| echo "==> Verifying feature branches exist..." | ||
| for branch in "${FEATURES[@]}"; do | ||
| if git rev-parse --verify "$branch" > /dev/null 2>&1; then | ||
| echo " $branch (local)" | ||
| elif git rev-parse --verify "origin/$branch" > /dev/null 2>&1; then | ||
| # Create a local tracking branch so 'git merge <branch>' works | ||
| git branch --track "$branch" "origin/$branch" 2>/dev/null || git branch -f "$branch" "origin/$branch" | ||
| echo " $branch (fetched from origin)" | ||
| else | ||
| echo "ERROR: branch '$branch' not found locally or on origin." >&2 | ||
| exit 1 | ||
| fi | ||
| done | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # Rebuild the integration branch | ||
| # --------------------------------------------------------------------------- | ||
| echo "" | ||
| echo "==> Resetting '$INTEGRATION_BRANCH' to '$BASE_BRANCH'..." | ||
| git checkout -B "$INTEGRATION_BRANCH" "$BASE_BRANCH" | ||
|
|
||
| MERGED=() | ||
| for branch in "${FEATURES[@]}"; do | ||
| echo "" | ||
| echo "==> Merging $branch..." | ||
| if git merge --no-ff "$branch" -m "chore: merge $branch into $INTEGRATION_BRANCH"; then | ||
| MERGED+=("$branch") | ||
| else | ||
| echo "" >&2 | ||
| echo "ERROR: merge conflict when merging '$branch'." >&2 | ||
| echo "" >&2 | ||
| echo "Conflicting files:" >&2 | ||
| git diff --name-only --diff-filter=U >&2 | ||
| echo "" >&2 | ||
| echo "Aborting. The '$INTEGRATION_BRANCH' branch has been reset; no changes were pushed." >&2 | ||
| git merge --abort | ||
| git checkout "$BASE_BRANCH" | ||
| git branch -D "$INTEGRATION_BRANCH" | ||
| echo "" >&2 | ||
| echo "How to fix:" >&2 | ||
| if [[ ${#MERGED[@]} -gt 0 ]]; then | ||
| echo " '$branch' conflicts with one of: ${MERGED[*]}" >&2 | ||
| echo " Rebase '$branch' on top of the conflicting branch:" >&2 | ||
| echo " git checkout $branch && git rebase <earlier-branch>" >&2 | ||
| echo " Then update features.txt so the earlier branch appears first." >&2 | ||
| else | ||
| echo " '$branch' conflicts with $BASE_BRANCH." >&2 | ||
| echo " Rebase it: git checkout $branch && git rebase $BASE_BRANCH" >&2 | ||
| fi | ||
| echo " Resolve conflicts, push the feature branch, then re-run this script." >&2 | ||
| exit 1 | ||
| fi | ||
| done | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # Patch version string with -qe suffix | ||
| # --------------------------------------------------------------------------- | ||
| # version.ts is generated/gitignored — patch packages/mystmd/package.json instead. | ||
| # The copy:version build step will propagate the version to version.ts at build time. | ||
| PACKAGE_JSON="packages/mystmd/package.json" | ||
| echo "" | ||
| echo "==> Patching version in $PACKAGE_JSON with '-qe' suffix..." | ||
|
|
||
| CURRENT_VERSION=$(node -p "require('./$PACKAGE_JSON').version") | ||
| QE_VERSION="${CURRENT_VERSION}-qe" | ||
|
|
||
| # Use node to patch the JSON safely (avoids sed quoting issues with JSON) | ||
| node -e " | ||
| const fs = require('fs'); | ||
| const pkg = JSON.parse(fs.readFileSync('$PACKAGE_JSON', 'utf8')); | ||
| pkg.version = '$QE_VERSION'; | ||
| fs.writeFileSync('$PACKAGE_JSON', JSON.stringify(pkg, null, 2) + '\n'); | ||
| " | ||
|
|
||
| git add "$PACKAGE_JSON" | ||
| git commit -m "chore: set version to ${QE_VERSION} for QuantEcon build" | ||
| echo " Version set to ${QE_VERSION}" | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # Summary | ||
| # --------------------------------------------------------------------------- | ||
| echo "" | ||
| echo "==> Build successful." | ||
| echo " Branch '$INTEGRATION_BRANCH' contains $BASE_BRANCH + ${#MERGED[@]} feature branch(es):" | ||
| for f in "${MERGED[@]}"; do | ||
| echo " $f" | ||
| done | ||
| echo " Version: ${QE_VERSION}" | ||
|
|
||
| # --------------------------------------------------------------------------- | ||
| # Optionally push | ||
| # --------------------------------------------------------------------------- | ||
| if $PUSH; then | ||
| echo "" | ||
| echo "==> Pushing '$INTEGRATION_BRANCH' to origin (force-with-lease)..." | ||
| git push --force-with-lease origin "$INTEGRATION_BRANCH" | ||
| echo " Done." | ||
| else | ||
| echo "" | ||
| echo "Tip: run with --push to push to origin/$INTEGRATION_BRANCH." | ||
| fi | ||
|
|
||
| git checkout "$BASE_BRANCH" | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| # Active QuantEcon feature branches | ||
| # One branch name per line. Lines starting with # are ignored. | ||
| # Order matters: branches are merged in sequence. | ||
| # If two branches conflict with each other, rebase the later one on top | ||
| # of the earlier one and update both entries accordingly. | ||
| # | ||
| # Format: | ||
| # <branch-name> [# optional description / upstream PR link] | ||
| # | ||
| # Example: | ||
| # feature/xref-improvements # https://github.com/jupyter-book/mystmd/pull/1 | ||
|
|
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.