[phillip-kruger]

smallrye-docs-mutiny.mp4

Every SmallRye project's documentation is now published on this site at /docs/<project>/<version>/, from the project's own repository, as AsciiDoc laid out like the Quarkus guides. Antora leaves the build, and the per-project MkDocs/mike sites are replaced.

Project	Versions	Today's source
GraphQL	latest (3.0.0.Beta9), 2.0–2.18, 1, main	MkDocs
Config	latest (4.1.0), 4.0, 3.0–3.18, 2, main	MkDocs
Fault Tolerance	latest (7.0.0), 6.0–6.11, 5, main	Antora
Health	latest (4.3.0), 4.0–4.2, main	Antora
JWT	latest (4.6.4), 4.0–4.5, main	Antora
Metrics (archived)	3, 2	Antora
Reactive Messaging	latest (5.0.0-RC2), 4.0–4.37, 3, main	MkDocs
Mutiny	latest (3.3.1), 3.0–3.2, 2.0–2.9, main	MkDocs
Stork	latest (3.0.3), 2.0–2.7, 1, main	MkDocs
Mutiny Zero	latest (1.3.1), 1.0–1.2, 0.1–0.2, main	MkDocs
Mutiny Vert.x bindings	latest (4.1.0), 4.0, 3.0–3.23, 2, main	MkDocs
Jandex	latest (3.6.0), 3.0–3.5, main	Antora (own site)

Versions. 200 versions in all:

• every minor of the latest two major lines gets its own version, from its newest release (/docs/smallrye-mutiny/2.9/ is 2.9.5);
• older major lines keep one version each (/docs/smallrye-graphql/1/);
• the major line before the latest release's is followed without any setup: when 4.0.0 is released, the 3.x minors stay online;
• a release without documentation is skipped, and not tried again until its tag moves;
• per project, hidden keeps versions published but out of the picker, and from stops publishing releases older than a given one (their old addresses go to the next newer version).

The old MkDocs sites can therefore go entirely: old addresses redirect, see Cutover.

/docs/ and /projects/.

• Both pages show a card per project.
• On /docs/, each card shows the latest release and its date, and links to the documentation. Within each group, the newest release comes first; the sync writes that order to data/docversions.yml.
• On /projects/, each card has a Documentation link (when the docs are here) and a GitHub link. The page is now generated from data/projects.yml:
  • it adds Stork, Mutiny Vert.x bindings and Mutiny Zero;
  • the renamed Reactive Utils is now the Mutiny Vert.x bindings;
  • archived projects are marked.
• Each project's start page opens with its description as a heading.
• Search in every docs header (button, / or Ctrl/Cmd+K). It searches the project and version being read, using a static Pagefind index built after the site (tools/docs/search-index.sh, one index per project, about 35 s for all 5,915 pages). One index across all projects dropped pages for common words; per-project indexes match a plain text count.
• GitHub in every docs header: the repository link with its latest release. On wide screens it also shows stars and forks, fetched by the browser from GitHub's API. The CSP now allows 'wasm-unsafe-eval' (Pagefind) and api.github.com.
• Mutiny, Reactive Messaging and Stork show their logos in the docs header and on the cards. Each logo is a single-colour mask in the text colour, so one file works in both themes.

Picker and notices.

• The version picker lists latest, then the minors grouped by major line (3.x, 2.x), then main.
• Every page shows its release date.
• A page that isn't the latest release says so and links to the same page in latest, when that page exists.

Each project gets a PR moving its sources to AsciiDoc (smallrye/smallrye-graphql#2732 first). The site does not wait for those PRs: a release whose docs are still MkDocs or Antora is converted on the fly, so all current releases are published as soon as this is merged.

Reviewing: why 11,910 files

Almost all of the diff is documentation copied in from the projects, not code.

Part	Files	Lines added
Code to review: tools/docs/, templates, CSS/JS, workflows, data/docs.yml, README, blog links	55	~7,000
Synced docs: content/docs/ (200 versions of 12 projects) and data/docversions.yml	11,938	~594,000

• What the synced docs are:
  • 1,870 .adoc pages: an older version stores only the pages that differ from the next newer one, see "Shared pages" below;
  • 6,867 example sources (only the ones pages include);
  • 1,708 images, about half of them generated dark versions;
  • 726 generated tables.
• It's mostly copies. Example sources and images repeat from one minor to the next, so the 11,862 files hold only 1,578 distinct contents: git stores each once.
• The biggest share. Reactive Messaging is 8,026 files: 41 versions.
• Why it is committed. The site builds from this copy, so the build needs nothing from the projects, and every change to the published docs shows in git history.
• To review only the code, hide content/docs/ and data/docversions.yml in the PR's file filter.
• Keeping the copy up to date. No PRs and no manual steps:
  • every night, Sync project documentation re-syncs each version whose main commit or release tag moved;
  • it adds new minors and removes versions no longer followed;
  • it commits to develop as the CI GitHub App, which triggers the normal build and deploy;
  • it can also be run by hand for one project or one version;
  • docs changes are reviewed in each project's own PRs, where Check project documentation renders them with this site's tools.

How it works

• Registry. data/docs.yml lists every project with docs: repository, docs path, the versions followed, and the addresses published before. It also drives the /docs/ hub.
• Tools. One Java program run with JBang, tools/docs/Docs.java: Java and JBang are all it needs. Each subcommand was checked byte for byte against the Python scripts it replaced, including the full nightly sync (28 versions of 13 projects).
• Sync. Docs.java sync copies a project's docs from a branch, tag or local checkout into content/docs/<project>/<version>/, and records versions and navigation in data/docversions.yml. It also:
  • copies only the example sources pages include;
  • renders diagram blocks to SVG through Kroki;
  • takes build-generated files from the released jars (Reactive Messaging's connector tables).
• Shared pages. Roq records every page in one generated class and fails beyond roughly 5,000 pages; the 200 versions have 5,992.
  • So the sync stores a page in an older version only when it renders differently from the next newer version. It compares the page text, the files it includes, and the attributes it uses (a page showing {project-version} stays).
  • The others are listed in the version's _inherited.yml: 1,870 pages are stored and 4,045 inherited.
  • After Roq, Docs.java inherit target/roq writes each inherited page from the stored one:
    • links and Javadoc versions are moved to the page's own version;
    • the version picker, sidebar, notice and source link are taken from that version's own start page.
  • The result is a real page per version, not a redirect.
• Converters. Docs.java mkdocs2adoc and Docs.java antora2adoc do the one-off conversion of a project, and the on-the-fly one. The MkDocs converter covers the docissimo macros (insert, javadoc, image, attributes), admonitions, tabs, PlantUML and nested pages. An HTML page kept next to the Markdown (Mutiny Vert.x bindings' 3.x to 4.x API compatibility report) becomes a page of its own, so the migration guide's links to it keep working. Javadoc links to other projects' artifacts (Kafka's and Pulsar's clients) use those projects' versions, read from each release's pom (maven-attributes, javadoc-versions).
• Workflows
  • Sync project documentation: nightly (main branches, new release tags) and on demand. It commits to develop.
  • Check project documentation: reusable. Project PRs render their docs with this site and fail on a broken include or a page that does not render.
  • Build: Antora and its Kroki services are gone. One step writes the inherited pages; another writes redirect pages for the 1,524 old Antora addresses (/docs/<project>/<release>/<page>.html) to the same page in the version replacing them.
• Layout.
  • Sidebar, version picker (pre-move versions as "Older versions…"), on-this-page list.
  • Tabs, and "main"/"archived" notices.
  • Old #_snake_case anchors are mapped to the new Quarkus-style ids.
  • Dark theme: diagrams and drawings show a dark version (Docs.java dark-images, run by the sync): the project's own x.dark.png, or the same hues with lightness inverted and white made transparent. Screenshots and photos keep a light background; roles no-dark/dark-invert override.
• Blog. Posts link to the new addresses. Docs.java rewrite-links checked each page and anchor against the generated site, including sections renamed in later releases.

Try this PR locally

try-smallrye-docs.sh checks out this PR, builds the site the way CI does, and serves it. With --project, it also checks out that project's docs PR and previews it live.

curl -sLO https://gist.githubusercontent.com/phillip-kruger/c22164fd8ecc2f56acbf97d116cfd37b/raw/try-smallrye-docs.sh
chmod +x try-smallrye-docs.sh
./try-smallrye-docs.sh                    # the whole site at http://localhost:8094 (about 5 minutes)
./try-smallrye-docs.sh --project mutiny   # also Mutiny's docs PR, live at /docs/smallrye-mutiny/main/

• Needs: git, Java 21+, JBang, and Node (for the search index).
• Platforms: Linux and macOS (bash 3.2 included). No gh login is needed; the PRs are fetched as pull/<n>/head.
• Projects: graphql, config, fault-tolerance, health, reactive-messaging, mutiny, mutiny-vertx-bindings, mutiny-zero, stork, jandex.
• Stopping: Ctrl+C stops it and undoes what the preview wrote into the site checkout.

Preview a project's docs locally (once this is merged)

In the project's checkout:

./mvnw -Pdocs-preview

Then open http://localhost:8080/docs/<project>/main/, for example http://localhost:8080/docs/smallrye-fault-tolerance/main/.

• Save a page or an included example, wait a few seconds, then refresh the page.
• Ctrl+C stops it.

What it does:

• No smallrye.github.io checkout, no paths, no Python: only Java and Maven. The docs-preview profile (added by each project PR) runs jbang docs-preview@smallrye/smallrye.github.io, and installs JBang if needed.
• It finds the project from its checkout, downloads this site's sources from develop once a day, and publishes the project's docs as the main version.
• The first run downloads the site's Maven dependencies, so it takes a few minutes. After that it starts in about 20 seconds.
• Options (where JBang is installed, jbang docs-preview@smallrye/smallrye.github.io --help):
  • --port <port> for another port;
  • --site ../smallrye.github.io to preview against a local checkout of this site, for example to try layout changes.
• Generated tables, such as Reactive Messaging's and Stork's config tables, come from the project's own local build when one exists, otherwise from the latest release.

It only works once this PR is merged, because it runs the tools and the site from develop.

Security

The PR check renders untrusted pull-request content, so:

• sync and converters never follow symbolic links;
• every read stays inside the checkout, and every write inside the target;
• ids, refs and nav entries are validated (nav links must be http(s));
• Asciidoctor's SAFE-mode jail is the site root.

Checked locally

The full site was built with all 200 versions, then inherit and redirects were run on it:

• 5,992 docs pages render with no errors.
• Asciidoctor warns only about includes that don't exist in the projects' own repositories:
  • Reactive Messaging's new RabbitMQ (og) examples and one Kafka example;
  • examples outside the docs directory in early Stork 2.x releases.
• Every internal link and image (7,583 pages, redirects included) points at a file that exists. The converters fixed the few links that were broken on the old sites too: GraphQL 1.x's /client_configuration, Fault Tolerance 6.0's link:core.adoc, and the links to pages that never existed (Reactive Messaging's JMS serde up to 4.24, and 5.0.0-RC2's RabbitMQ page), which become plain text.
• A second sync with nothing new changes nothing.

After merge

• Repo settings: the CI GitHub App (CI_APP_ID/CI_APP_PRIVATE_KEY) is reused by the sync workflow.
• Project PRs: merge them one by one; the nightly sync picks up each AsciiDoc main.
• Cutover: the old MkDocs sites go entirely. On each project's gh-pages, Docs.java gh-pages-redirects turns every page of every old version into a redirect to the same page of the version replacing it here: its minor, or its major line when older (e.g. /smallrye-graphql/2.8.0/federation/ → /docs/smallrye-graphql/2.8/federation/), or the version's start page if the page no longer exists. Tried on all seven branches when versions were per major line: 12,739 redirect pages, none broken, 85% to the same page. Javadoc (apidocs/) stays as it is. Jandex's old site gets the same treatment.

[phillip-kruger]

Screenshots of each project's documentation before (its MkDocs or Antora site today) and after (the project's AsciiDoc from its PR, rendered by this branch locally), updated for the header and sidebar changes. On smallrye.io the sidebar starts closed (the ☰ button opens it, and it stays open on the next pages for that reader); its groups are folded except the one holding the current page. The project and version picker sit in the header next to the logo, so a start page has no separate title; further down, a start page lists the documentation, from its navigation. Each project PR has more pages, including one with the sidebar open.

Before	After
jandex · smallrye/jandex#733 · old page
smallrye-config · smallrye/smallrye-config#1603 · old page
smallrye-fault-tolerance · smallrye/smallrye-fault-tolerance#1351 · old page
smallrye-graphql · smallrye/smallrye-graphql#2732 · old page
smallrye-health · smallrye/smallrye-health#713 · old page
smallrye-jwt · smallrye/smallrye-jwt#935 · old page
smallrye-mutiny · smallrye/smallrye-mutiny#2220 · old page
smallrye-mutiny-vertx-bindings · smallrye/smallrye-mutiny-vertx-bindings#1475 · old page
smallrye-mutiny-zero · smallrye/smallrye-mutiny-zero#376 · old page
smallrye-reactive-messaging · smallrye/smallrye-reactive-messaging#3577 · old page
smallrye-stork · smallrye/smallrye-stork#1385 · old page
Sidebar open (smallrye-reactive-messaging, nested groups)
Further down the start page: its contents, listed from the navigation
Dark theme (smallrye-reactive-messaging): light paper behind the drawing (before) and its dark version (after)
Dark theme (smallrye-mutiny): light paper behind the drawing (before) and its dark version (after)
Dark theme (smallrye-stork): light paper behind the drawing (before) and its dark version (after)
Dark theme (smallrye-mutiny-zero): light paper behind the drawing (before) and its dark version (after)

Dark theme: the same screenshots with the dark theme

The old Antora sites and the GraphQL, Vert.x bindings and Mutiny Zero MkDocs sites have no dark theme, so their "before" stays light; the other old MkDocs sites were switched to their own dark palette.

Before (dark theme)	After (dark theme)
jandex · smallrye/jandex#733 · old page
smallrye-config · smallrye/smallrye-config#1603 · old page
smallrye-fault-tolerance · smallrye/smallrye-fault-tolerance#1351 · old page
smallrye-graphql · smallrye/smallrye-graphql#2732 · old page
smallrye-health · smallrye/smallrye-health#713 · old page
smallrye-jwt · smallrye/smallrye-jwt#935 · old page
smallrye-mutiny · smallrye/smallrye-mutiny#2220 · old page
smallrye-mutiny-vertx-bindings · smallrye/smallrye-mutiny-vertx-bindings#1475 · old page
smallrye-mutiny-zero · smallrye/smallrye-mutiny-zero#376 · old page
smallrye-reactive-messaging · smallrye/smallrye-reactive-messaging#3577 · old page
smallrye-stork · smallrye/smallrye-stork#1385 · old page
Sidebar open (smallrye-reactive-messaging, nested groups)
Further down the start page: its contents, listed from the navigation

[Ladicek]

Hey @phillip-kruger, this looks amazing! One question: are we supposed to merge your PRs in the individual projects before this one is merged, or after? Or it doesn't matter?

[Ladicek]

Oh one more thing: the way we preview individual project docs locally is not good. I mean, it's probably the best we can do at the moment, and I usually don't mind because IntelliJ's built-in Asciidoc renderer is good enough for most cases, but when it's not... Well, I guess I'll get used to it. At least it's documented :-)

[phillip-kruger]

Thanks @Ladicek! Best is this one first, then the project PRs in any order.

• Each project PR adds a "Documentation" check that calls a workflow from this PR, so that check fails until this one is merged.
• The current site still builds the Antora docs, and for JWT it reads doc/ on main. Move the documentation from Antora to AsciiDoc pages published on smallrye.io smallrye-jwt#935 is merged already, so the next build of the current site would fail on JWT until this PR (which drops Antora) is in. Fault Tolerance and Health are fine either way, as the current site reads their tags and the 3.0.x branch.
• Nothing waits for the project PRs: once this is merged, smallrye.io converts every project's current release on the fly, and the nightly sync picks up each project's own AsciiDoc on main the night after its PR is merged.

[Ladicek]

OK, thanks! I'll review the PRs but wait with merging until this is done.

[phillip-kruger]

@Ladicek I'll add something that make local preview easier.

[phillip-kruger]

@Ladicek about previewing locally: that should be much better now.

In a project's checkout, ./mvnw -Pdocs-preview shows its documentation on a local smallrye.io at http://localhost:8080/docs/<project>/main/, and publishes it again whenever a page or an included example is saved (a few seconds, then refresh). No smallrye.github.io checkout, no paths, no Python: only Java. The profile runs the docs tools with JBang (and installs JBang if needed); with JBang installed, jbang docs-preview@smallrye/smallrye.github.io does the same.

To make that possible, the tools in this PR are now Java (one JBang program, tools/docs/Docs.java) instead of Python; each tool was checked byte for byte against the Python version, including the whole nightly sync (28 versions of 13 projects). Every project PR has the profile and an updated "Previewing" section in its docs README (I folded your README fixes in: the path is gone). It works once this PR is merged, since it runs the tools from develop.

[Ladicek]

Sounds much better, thanks!

[phillip-kruger]

I merged this by accident, sorry. It was reverted (#139), and the deployed site is back to what it was before the merge. The change is open again as #140, from the same branch, for review there.