Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

Explanation: architecture

This explains why the versioned site is built the way it is — the design reasoning behind the assemble scripts and the build/publish workflow split. For the what (inputs, options, copy-paste snippets), see the reference and tutorial.

The core idea: reconstruct the whole site every deploy

Every deploy rebuilds the complete site tree from authoritative sources and publishes it directly to GitHub Pages via actions/upload-pages-artifact + actions/deploy-pages. There is no gh-pages branch — deploy-pages publishes one artifact as the entire site, which is a whole-site-replace. The four source kinds and their durability are in the reference.

Releases are permanent, so old versions never vanish. PR previews come from CI artifacts and silently drop if the artifact expires and nothing rebuilds — fine for optional preview docs. The default branch is the one fragile required version (artifact-only, yet guarded — assemble hard-fails rather than publish a site missing it), so each deploy keeps a copy of its docs.zip in the Actions cache; a deploy whose fresh artifact has expired restores the branch from there.

The alternative — publishing that zip into the site itself — is permanent, which the cache is not, but it ships a multi-megabyte zip inside every Pages artifact forever, on a site already pressing against a hard 1 GB ceiling (see below). The cache is evicted after 7 days without a read, and by LRU past 10 GB, so a repo that goes quiet for over a week and whose default-branch CI artifact has also expired loses the rung and the deploy hard-fails — loudly, and fixed by one push. That is the accepted trade. (A gh-pages migration keeps its old /main/ content only until the default branch builds docs under the new pipeline — see migrate-from-gh-pages.)

Why the site has a size limit

Reconstructing everything has one cost that grows without bound: upload-pages-artifact tars the whole site into a single artifact, and GitHub Pages rejects that artifact over 1 GB. A long-lived project reaches it. blueapi, at 131 released docs.zips, was already at 452 MB and adding ~5 MB per release — a few years from deploys simply failing, with nothing on the way to say so.

Two things follow.

The engine takes max-releases and max-prs inputs: publish only the N most recent releases (ranked by the tagged commit’s date) and the N most recently built open PRs. Both default to 0 (unlimited), so upgrading never silently deletes versions from an existing site — but the paved path in the tutorial sets them. actions/deploy-pages warns when the uploaded artifact exceeds the limit, and the fix is not destructive: older releases keep their docs.zip assets and come back the moment you raise the cap. See keep-the-site-small.

Ranking is on the release’s created_at, never on parsing the version number: tags are too inconsistent across repos for a parser to be safe, and published_at lies whenever an old release is re-published (blueapi has a 1.3.2-a9 created in 2025 and published in 2026, which under published_at outranks the genuinely newer 1.11.3).

The same pressure is why the default branch’s durable copy lives in the Actions cache rather than in the site, and why the gather caches release assets rather than re-downloading ~450 MB of immutable zips on every event. The caps bound that cache too: its entry is as large as the published release set, and it competes with the repo’s build caches for a 10 GB quota.

Why this replaced the gh-pages + keep_files model

The previous model (mirrored from python-copier-template-example) had three problems for a MyST/book-theme site:

  1. The CI docs.zip artifact is not locally previewable. book-theme emits root-absolute asset URLs (/build/_assets/app.css) regardless of BASE_URL, so opening index.html over file:// resolves assets against the filesystem root → 404 → unstyled, broken. There is no relative-path mode. Local preview means myst start, or serving a BASE_URL-free build over HTTP.

  2. BASE_URL is mandatory and per-version. Each version lives at /<repo>/<version>/ and must be built with BASE_URL=/<repo>/<version>. One build cannot serve two paths.

  3. keep_files: true accumulation drifts. The published site becomes whatever has piled up on gh-pages over time; there is no single source of truth, and the branch history grows without bound.

Reconstructing the live set every deploy and letting deploy-pages replace the whole site makes deletion self-healing: a merged PR or a deleted release simply isn’t gathered next time, so it disappears — no keep_files drift, no branch to prune.

Migrating from gh-pages

Two facts make the cutover safe and fix its ordering:

The docs.zip / version-name contracts

Two contracts (described in the reference) keep build and reconstruction in sync. The design rationale in both is to eliminate sanitisation:

Split build (unprivileged) from publish (privileged)

A pull_request run from a fork gets a read-only GITHUB_TOKEN and no secrets — a deliberate security boundary, so a PR can’t deface the site or exfiltrate secrets. The architecture makes that boundary structural by splitting build from publish:

So a fork’s build can never reach a write token; only trusted code deploys.

Why publishing listens instead of being called

Publishing is not a job in ci.yml. It is a separate publish.yml in the consumer’s repo, triggered by workflow_run when their CI workflow completes, which then calls the publish-gh-pages.yml engine.

It used to be nested, deliberately, so the deploy’s status and URL showed on the PR. Two things made that wrong.

The first is cost. Reconstructing the site is O(the whole site) and completely independent of what changed — the same 24 releases, the same open PRs, the same upload, whichever PR triggered it. Nesting that put a large constant inside every PR’s critical path. The first consumer to adopt this switched PR previews off one day later, because a 650-second deploy had been bolted onto a 40-second docs build.

The second is that the visibility was never the PR author’s to act on. A red check for a wedged Pages origin, on a dependency-bump PR, is noise that trains people to ignore CI.

workflow_run fixes both: the deploy runs afterwards on its own run, and a failure is visible where the people who can fix it are looking.

What listening deleted

The trigger change was mostly subtraction, because three separate pieces of machinery existed only to work around the old shape.

The tag trampoline. A release tag is cut on the merge commit, so it shares the default branch’s just-deployed SHA, and deploy-pages stamps every deployment with pages_build_version = GITHUB_SHA — no input to change it, and the value is server-validated against the OIDC commit claim, so a unique one can’t be forced (it 404s; see actions/deploy-pages#383). Pages silently drops a second deploy of an already-deployed SHA on some events: it reports success and flips the deployment record active, but the origin keeps serving the first artifact. A tag deploy would “succeed” while the site stayed on the pre-tag build.

The old fix was a trampoline: tags re-fired a locally dispatchable shim so the deploy landed as a workflow_dispatch, which does force a re-serve.

workflow_run forces a re-serve too. That was established by experiment rather than inference, because the documented rule does not predict it — on 2026-08-21 four consecutive deploys at the identical build version b24237484c3b445469c2db4ef161410a185fcdbc (a push to main, a tag cut on that same commit, and two pushes to one PR) each updated the live origin. So a tag’s deploy re-serves directly, and the trampoline is gone.

The shim. publish-dispatch.yml existed only because a reusable workflow cannot be workflow_dispatch’d cross-repo, so the trampoline needed a local file to re-fire. No trampoline, no shim. What consumers carry now is one publish.yml that calls the engine.

The in-run artifact injection. A nested publish runs inside the build’s own run, so that run is not yet a completed success and the gather cannot discover it — worse, on a main push the gather would find the previous run and publish a build one commit behind. So the build’s version name was threaded through ci.yml and the shim, and its artifact staged as the highest-priority source. workflow_run fires after the triggering run completes, so the ordinary gather finds it. Nothing to inject, nothing to thread.

The read-only warn job went the same way: a fork’s CI never reaches the engine, because the caller excludes it.

The trap workflow_run brings with it

In a workflow_run run, GITHUB_SHA and github.ref are always the default branch’s HEAD — never the commit that was built. A PR-triggered deploy reports refs/heads/main.

That is mostly harmless (it is why the same-SHA question mattered at all), but it silently breaks anything that asks “was this the default branch?”. Both of the engine’s cache-save steps ask exactly that: gated on github.ref they would fire on every deploy, including one dispatched from another ref, whose entry only that ref could ever read. They test github.event.workflow_run.head_branch instead.

Because that lives in an if: expression rather than a shell script, the gather harness cannot reach it; test/workflow-harness/test_shape.py asserts it structurally, along with the caller’s two guards — all of which fail open, and so would never announce themselves.

Fork PRs still cannot deploy themselves

workflow_run runs with a write token even when a fork’s pull request triggered it — the classic pwn-request shape, and a real hazard rather than a theoretical one. The caller therefore requires head_repository.full_name == github.repository, so a fork’s build never reaches the engine automatically. A maintainer publishes a preview by dispatching publish.yml with the PR number, which records approval against that exact head SHA; a later push to the PR drops the preview until re-approved.

The inline-bash / JS split inside assemble

The assemble logic is split between publish-gh-pages.yml inline steps and assemble.mjs (sparse-checked-out at job.workflow_sha):

Pure bash is ruled out — semver ordering, prerelease detection and JSON rendering are not unit-testable in bash. Bash never parses JSON itself: every extraction uses gh’s built-in -q/--jq (it embeds real jq), never a piped standalone jq — a gh … | jq pipe would mask an API failure as empty output. Gather order encodes priority — releases first, then branch CI overwrites them; all version-ordering and prerelease logic lives in generate.

Fork-PR previews: per-commit maintainer opt-in

The risk with a fork PR is not the build (it never holds a write token) but serving fork-authored HTML/JS under the canonical *.github.io domain — phishing/defacement under a trusted URL, and free arbitrary-content hosting. So a fork preview is never automatic and is pinned to a specific commit:

Rejected alternatives: pull_request_target (privileged but checks out base code — building PR-head content under it is the classic RCE footgun, since a MyST build runs PR-authored plugins); auto-publishing every fork PR (unattended untrusted content on the canonical domain); the fork’s own Pages (required all-branch push triggers and gave contributors no canonical preview).

Stable alias

Other projects fetch this site’s objects.inv for cross-references, so they need a stable URL that always points at the latest release — not a version number that changes every release. The site therefore publishes a stable/ alias.

MyST writes base-relative URIs into objects.inv, so a consumer pointing intersphinx at …/repo/stable/ resolves every target under /stable/ — the links stay stable rather than pinning to a concrete version.

The widget keeps switcher.json listing real versions only (no stable entry), with preferred: true on the latest release. Visiting /stable/ selects the concrete release it aliases (so the dropdown shows e.g. v2.0, not a separate “stable” item), and switching to a pinned version preserves the page path onto it. The stable segment name is a fixed convention, hardcoded in the widget.

Edge cases

The release-layer cache

Re-downloading every release’s docs.zip on every deploy is the one recurring cost that scales with the number of releases, and it is not theoretical: at 131 releases it was 114 seconds per deploy, on every event, fetching the same immutable bytes.

So the engine caches them — actions/cache, keyed on the exact set of asset ids it intends to publish, files named by asset id so a re-cut release cannot be served from a stale entry, and pruned to the published set so a capped site keeps a capped cache. Steady state is a total hit; cutting a release downloads exactly one zip.

Caches are written only when the default branch was what got built. Entries are scoped to the ref that wrote them, so one written by a deploy dispatched from some other ref is invisible to every ordinary deploy — dead weight against the 10 GB quota. (Under workflow_run the writing ref is always the default branch whatever triggered the run, which is why the guard has to read head_branch, not github.ref.)

Neither key contains a branch or a commit. The branch is already in the scope, and both keys are content hashes: the release cache on the set of asset ids it intends to publish, the default-branch cache on the bytes of docs.zip itself. A commit SHA would mint a new multi-megabyte entry on every push even when the built docs are identical. The default-branch restore therefore uses a bare key: mvs-default-v1- that can never match exactly, letting restore-keys return the most recent entry — you cannot name a content hash before you have the content.

That cache is also why the release downloads are serial while the PR-artifact downloads are parallel. With the cache in place the release gather fetches nothing on almost every deploy, so parallelising it buys a rare, off-critical-path cold start at the cost of a three-pass download-and-stage structure. PR artifacts cannot be cached at all — a PR’s head SHA changes with every push — so that gather pays one download per open PR on every deploy, which is where xargs -P 8 earns its keep.

Key resolved decisions

myst-version-switcher-plugin
Explanation
myst-version-switcher-plugin
Reference