Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 15 additions & 6 deletions .github/workflows/deploy-pages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -118,14 +118,23 @@ jobs:
# Verified with the exact published command this lane runs, fetched
# through the proxy:
#
# $ go run github.com/go-embedded-ruby/docs/tools/conformancefigure@v0.6.0 --root .
# 5 total claim(s) and 2 version claim(s) checked against the gate's 23511 at Ruby 3.4.1 (two copies, agreeing)
# $ go run github.com/go-embedded-ruby/docs/tools/conformancefigure@v0.7.0 --root .
# 6 total claim(s) and 1 version claim(s) checked against the gate's 23672 at Ruby 4.0.9 (derived from RUBY_VERSION)
#
# "two copies, agreeing" is the interpreter repository's state today;
# go-embedded-ruby/ruby#845 collapses it to one and the same line will
# then read "derived from RUBY_VERSION", with nothing here to change.
# Two numbers in that line moved for reasons worth keeping apart. "1
# version claim … derived from RUBY_VERSION" is what the earlier comment
# here predicted: go-embedded-ruby/ruby#845 collapsed the two copies of
# the version into one, and nothing on this page had to change.
#
# SIX total claims, where v0.6.0 counted five, is a defect that version
# could not report. Its vocabulary spelled out "the sum is", "its sum is"
# and "whose sum is", so "The sum of those rows is <b>23,673</b>" in
# layouts/_default/list.html was never a candidate: the lane printed "5
# checked" and passed while that line held a stale figure. v0.7.0 reads
# it. Keep this pin moving forward -- a gate pinned to a version that
# cannot see a line is indistinguishable from a page with nothing wrong.
- name: The stated conformance total, and the version it was counted at, match rbgo's gate
run: go run github.com/go-embedded-ruby/docs/tools/conformancefigure@v0.6.0 --root .
run: go run github.com/go-embedded-ruby/docs/tools/conformancefigure@v0.7.0 --root .

- name: Build landing page
run: hugo --minify --baseURL "${{ steps.pages.outputs.base_url }}/"
Expand Down
2 changes: 1 addition & 1 deletion hugo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -71,4 +71,4 @@ disableKinds = ["taxonomy", "term", "RSS", "sitemap", "404"]
[[params.phases]]
name = "Phase 8 — Conformance & performance"
status = "active"
blurb = "Since July the work has been **runtime** conformance, measured against [ruby/spec](https://github.com/ruby/spec)'s `language/` + `core/` suites: **23,511 examples pass** — the total that `BASELINE` sums to on `main` (2026-10-30; 2,202 of 2,206 spec files produce a result, the other 4 recorded as FILEFAIL), gated in CI by a **per-file baseline** — one record per spec file, so a file that regresses is named rather than absorbed. It replaced a single shrink-only floor, which needed a margin wider than the largest spec file and was therefore blind to any regression smaller than that: the floor had climbed from 6,000 when the ratchet first landed on 2026-08-03 to 22,475, while sitting 211 below the measured total. **Only `language/` and `core/` are swept** — ruby/spec's `library/`, `optional/` and `security/` trees are not run at all, so this is not a percentage of Ruby. Two of the recent jumps were fixes to the **measurement** rather than the interpreter — the mspec shim, and a parser upgrade that made 13 previously-unparseable spec files visible — and those are gains in what the measurement can *see*, not in what the VM can *do*. Earlier, a conformance campaign brought the pure-Go **front-end** to ~**100% of all valid Ruby** parsed across Rails + Puppet (99.96% of files; 2 misses are syntax-error fixtures MRI rejects) and **parse + compile** of **Rails 99.82% / Puppet 100%**, with **0 over-permissive** — and RuboCop 99.7%, Sinatra/Jekyll/Thor/Kramdown/dry-struct 100%, RSpec DSL 10/10. On the runtime side, that road is now underway: **the real `puppet apply` CLI runs end-to-end** on a pure-Go CGO=0 `rbgo` — `require \"puppet\"` boots Puppet 8.11.0 (all pure-Ruby gem deps on the path), and the genuine `Puppet::Util::CommandLine` → `Puppet::Application::Apply` path (real `OptionParser`, all types + providers, settings catalog applied to disk, then the user catalog via the transaction/RAL) converges `notify`, `file` and `exec` resources (the `exec` provider runs commands via pure-Go process execution with `onlyif`/`unless`/`creates`/`path` guards), emits real Puppet output (`Notice: hi from rbgo cli`) and exits 0 with the YAML run report round-tripped. Broader resource **providers** are the active next milestone (`user`/`group` provider-ready; `package`/`service` need a host package manager/systemd + root). rbgo is also increasingly composed from the standalone pure-Go **go-ruby-* family** — **178 modules across 176 orgs are now bound into rbgo as native modules** (parser, regexp, erb, marshal, yaml, format, optparse, strscan, json, date, … a selection is on the landing page), each its own 100%-coverage 6-arch org. (`Set` is not among them: it is implemented in-tree, over rbgo's own Hash.) Of the six 64-bit arches, amd64 and arm64 are tested on every pull request, as is 32-bit 386 (since v0.8.2; 32-bit arm runs nightly under qemu), and since v0.8.3 every pull request also compiles rbgo for every GOOS/GOARCH Go supports, plan9 included since v0.8.5 (vetted, not run: there is no Plan 9 kernel in CI); riscv64, loong64, ppc64le and s390x are validated on a **nightly schedule** — under qemu, and natively on the GCC Compile Farm plus an IBM LinuxONE s390x host — not on the pull-request path. On performance, a 6-runtime suite (rbgo/rbgo+AOT vs MRI/YJIT/JRuby/TruffleRuby): **rbgo+AOT beats MRI+YJIT on integer-bound kernels — measured here at 8.7× on the tight-loop benchmark and 2.0× on recursive `fib(35)`** (darwin/arm64, whole-process wall time); plus inline method caches, small-integer interning (a small-int loop drops from ~245k allocations to 1) and capture-tracked frame recycling (~14% faster on fib)."
blurb = "Since July the work has been **runtime** conformance, measured against [ruby/spec](https://github.com/ruby/spec)'s `language/` + `core/` suites: **23,672 examples pass** — the total that `BASELINE` sums to on `main` (re-recorded 2026-11-03 at Ruby 4.0.9, on the pinned runner that enforces it; 2,202 of 2,206 spec files produce a result, the other 4 recorded as FILEFAIL). One file, `core/random/bytes_spec.rb`, draws 8 or 9 passing examples from run to run; on 2026-11-04 its row was pinned to **8**, its floor, rather than to the 9 that re-recording happened to observe — a baseline recorded at a high draw turns the next ordinary run red, gated in CI by a **per-file baseline** — one record per spec file, so a file that regresses is named rather than absorbed. It replaced a single shrink-only floor, which needed a margin wider than the largest spec file and was therefore blind to any regression smaller than that: the floor had climbed from 6,000 when the ratchet first landed on 2026-08-03 to 22,475, while sitting 211 below the measured total. **Only `language/` and `core/` are swept** — ruby/spec's `library/`, `optional/` and `security/` trees are not run at all, so this is not a percentage of Ruby. Two of the recent jumps were fixes to the **measurement** rather than the interpreter — the mspec shim, and a parser upgrade that made 13 previously-unparseable spec files visible — and those are gains in what the measurement can *see*, not in what the VM can *do*. Earlier, a conformance campaign brought the pure-Go **front-end** to ~**100% of all valid Ruby** parsed across Rails + Puppet (99.96% of files; 2 misses are syntax-error fixtures MRI rejects) and **parse + compile** of **Rails 99.82% / Puppet 100%**, with **0 over-permissive** — and RuboCop 99.7%, Sinatra/Jekyll/Thor/Kramdown/dry-struct 100%, RSpec DSL 10/10. On the runtime side, that road is now underway: **the real `puppet apply` CLI runs end-to-end** on a pure-Go CGO=0 `rbgo` — `require \"puppet\"` boots Puppet 8.11.0 (all pure-Ruby gem deps on the path), and the genuine `Puppet::Util::CommandLine` → `Puppet::Application::Apply` path (real `OptionParser`, all types + providers, settings catalog applied to disk, then the user catalog via the transaction/RAL) converges `notify`, `file` and `exec` resources (the `exec` provider runs commands via pure-Go process execution with `onlyif`/`unless`/`creates`/`path` guards), emits real Puppet output (`Notice: hi from rbgo cli`) and exits 0 with the YAML run report round-tripped. Broader resource **providers** are the active next milestone (`user`/`group` provider-ready; `package`/`service` need a host package manager/systemd + root). rbgo is also increasingly composed from the standalone pure-Go **go-ruby-* family** — **178 modules across 176 orgs are now bound into rbgo as native modules** (parser, regexp, erb, marshal, yaml, format, optparse, strscan, json, date, … a selection is on the landing page), each its own 100%-coverage 6-arch org. (`Set` is not among them: it is implemented in-tree, over rbgo's own Hash.) Of the six 64-bit arches, amd64 and arm64 are tested on every pull request, as is 32-bit 386 (since v0.8.2; 32-bit arm runs nightly under qemu), and since v0.8.3 every pull request also compiles rbgo for every GOOS/GOARCH Go supports, plan9 included since v0.8.5 (vetted, not run: there is no Plan 9 kernel in CI); riscv64, loong64, ppc64le and s390x are validated on a **nightly schedule** — under qemu, and natively on the GCC Compile Farm plus an IBM LinuxONE s390x host — not on the pull-request path. On performance, a 6-runtime suite (rbgo/rbgo+AOT vs MRI/YJIT/JRuby/TruffleRuby): **rbgo+AOT beats MRI+YJIT on integer-bound kernels — measured here at 8.7× on the tight-loop benchmark and 2.0× on recursive `fib(35)`** (darwin/arm64, whole-process wall time); plus inline method caches, small-integer interning (a small-int loop drops from ~245k allocations to 1) and capture-tracked frame recycling (~14% faster on fib)."
4 changes: 2 additions & 2 deletions layouts/_default/list.html
Original file line number Diff line number Diff line change
Expand Up @@ -70,10 +70,10 @@ <h2>C · Application conformance &amp; performance</h2>
<h3>Runtime conformance — ruby/spec</h3>
<p><b>22,488 examples pass</b> in ruby/spec's <code>language/</code> + <code>core/</code> suites, measured 2026-09-26 on <code>main</code> (<code>3e8e3cc</code>). Three independent runs — two through the ratchet at different timeouts, one through a separate sweep — reported the same total.</p>
<ul>
<li>CI gates <b>file by file</b>, not on a total: no spec file may pass fewer examples than its recorded row, and none may stop loading. The sum of those rows is <b>23,511</b> at rbgo v0.3.0 (2026-10-12) — it is what CI guarantees, while the measured total above is what one dated run did. This replaced a single shrink-only floor of 22,475, which needed a margin wider than the largest spec file and so could not see a regression smaller than that.</li>
<li>CI gates <b>file by file</b>, not on a total: no spec file may pass fewer examples than its recorded row, and none may stop loading. The sum of those rows is <b>23,672</b>, re-recorded at Ruby 4.0.9 (2026-11-03) on the pinned runner that enforces it — it is what CI guarantees, while the measured total above is what one dated run did. One file (<code>core/random/bytes_spec.rb</code>) is unstable between 8 and 9 passing examples; since 2026-11-04 its row holds <b>8</b>, its floor, because a row recorded at the high draw makes the next ordinary run a red build. This replaced a single shrink-only floor of 22,475, which needed a margin wider than the largest spec file and so could not see a regression smaller than that.</li>
<li><b>Only <code>language/</code> and <code>core/</code> are swept.</b> ruby/spec's <code>library/</code>, <code>optional/</code> and <code>security/</code> trees are not run at all, so this figure is <b>not</b> a percentage of Ruby, and most of the standard library sits outside it.</li>
<li>Of the examples that ran, <b>93.3%</b> pass (22,488 of 24,116); 477 more are skipped because our shim stubs the matcher they need, and 10 of 2,206 spec files produce no result. Skips and unreadable files sit outside the ratio entirely.</li>
<li>The guarantee has climbed from a floor of <b>6,000</b> on 2026-08-03 to a per-file baseline summing <b>23,511</b>. Two earlier jumps were fixes to the <b>measurement</b> — the mspec shim, and a parser upgrade that made 13 previously-unparseable spec files visible — which are gains in what the measurement can <i>see</i>, not in what the VM can <i>do</i>.</li>
<li>The guarantee has climbed from a floor of <b>6,000</b> on 2026-08-03 to a per-file baseline summing <b>23,672</b>. Two earlier jumps were fixes to the <b>measurement</b> — the mspec shim, and a parser upgrade that made 13 previously-unparseable spec files visible — which are gains in what the measurement can <i>see</i>, not in what the VM can <i>do</i>.</li>
</ul>
</div>
<div class="appcard">
Expand Down
6 changes: 3 additions & 3 deletions layouts/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ <h1 class="head">A Ruby you can embed,<br><span class="accent">written in Go.</s
<a class="btn btn-primary" href="https://github.com/go-embedded-ruby/ruby">{{ partial "icons/rocket.html" . }}Get started →</a>
<a class="btn btn-ghost" href="/performance/">{{ partial "icons/graph.html" . }}See the benchmarks</a>
</div>
<div class="trust"><span><b>23,511</b> ruby/spec <code>language/</code>+<code>core/</code> examples passing</span><span><b>~100%</b> of real Ruby parsed</span><span><b>6</b> 64-bit + <b>2</b> 32-bit arches</span></div>
<div class="trust"><span><b>23,672</b> ruby/spec <code>language/</code>+<code>core/</code> examples passing</span><span><b>~100%</b> of real Ruby parsed</span><span><b>6</b> 64-bit + <b>2</b> 32-bit arches</span></div>
</div>
<div class="term">
<div class="bar"><i style="background:#ff5f57"></i><i style="background:#febc2e"></i><i style="background:#28c840"></i><span class="t">rbgo — zsh</span></div>
Expand Down Expand Up @@ -59,7 +59,7 @@ <h2>Everything a Ruby runtime gives you — as a Go library.</h2>
<div class="card"><div class="ico">🧩</div><h3>Embeddable</h3><p>A real object model — classes, mixins, <code>method_missing</code>, metaprogramming, Fiber. The Go API is deliberately small today: <code>ruby.Run(src, out)</code> runs a program on a fresh VM and writes its output to one writer. Calling a Ruby method from Go is not exposed yet.</p></div>
<div class="card"><div class="ico">📦</div><h3>One static binary</h3><p>CGO=0 by default. Tested on amd64, arm64, riscv64, loong64, ppc64le, s390x, 386 and arm; compiles for every GOOS/GOARCH Go supports — 47 targets, the BSDs, AIX, Solaris, mips and Plan 9 included (Plan 9 is vetted, not run).</p></div>
<div class="card"><div class="ico">⚡</div><h3>AOT for integer kernels</h3><p>Ahead-of-time compilation of integer-bound methods to unboxed Go. Measured on this project’s own micro-benchmarks (darwin/arm64, whole-process wall time): <b>8.7×</b> MRI+YJIT on the tight-loop case, <b>2.0×</b> on recursive <code>fib(35)</code>. Only methods the compiler can lower benefit.</p></div>
<div class="card"><div class="ico">✅</div><h3>Conformance, measured</h3><p><b>23,511</b> ruby/spec <code>language/</code> + <code>core/</code> examples pass, gated in CI <b>file by file</b>: no spec file may pass fewer examples than its recorded row, so a regression is named rather than absorbed in a total. <b>Only <code>language/</code> and <code>core/</code> are swept:</b> <code>library/</code>, <code>optional/</code> and <code>security/</code> are not run, so this is not a percentage of Ruby. Those guards resolve at Ruby <b>3.4.1</b> — the version this runtime reports — so every example behind a <code>ruby_version_is "4.0"</code> guard is removed from the suite before anything is counted: 61 block guards and 5 inline sites sit at that boundary, while the latest MRI release is 4.0.9. Front-end: ~100% of valid Ruby parsed across Rails + Puppet (2026-06).</p></div>
<div class="card"><div class="ico">✅</div><h3>Conformance, measured</h3><p><b>23,672</b> ruby/spec <code>language/</code> + <code>core/</code> examples pass, gated in CI <b>file by file</b>: no spec file may pass fewer examples than its recorded row, so a regression is named rather than absorbed in a total. <b>Only <code>language/</code> and <code>core/</code> are swept:</b> <code>library/</code>, <code>optional/</code> and <code>security/</code> are not run, so this is not a percentage of Ruby. Those guards now resolve at Ruby <b>4.0.9</b> — the version this runtime reports, and the latest MRI release — so the examples behind a <code>ruby_version_is "4.0"</code> guard are counted, and those behind <code>ruby_version_is ""..."4.0"</code> are no longer asked for. The move was measured before it was made, both engines built from one commit minus the version string: <b>+144 passing</b>, and <b>27 examples that went red</b> — a figure no pass total shows, found by comparing the two runs' failure descriptions. Front-end: ~100% of valid Ruby parsed across Rails + Puppet (2026-06).</p></div>
<div class="card"><div class="ico">🔒</div><h3>Closed-world builds</h3><p><code>rbgo build --closed</code> bakes the program to bytecode and drops the front-end for a smaller binary.</p></div>
<div class="card"><div class="ico">🌿</div><h3>A real ecosystem</h3><p>Built from the <b>go-ruby-*</b> family — each a 100%-coverage, 6-arch stdlib module, bound natively.</p></div>
</div>
Expand All @@ -68,7 +68,7 @@ <h2>Everything a Ruby runtime gives you — as a Go library.</h2>
<section id="perf" style="padding-top:10px"><div class="wrap">
<div class="stats">
<div><div class="n">8.7×</div><div class="l">rbgo+AOT vs MRI+YJIT, tight-loop benchmark (2.0× on <code>fib(35)</code>)</div></div>
<div><div class="n">23,511</div><div class="l">ruby/spec <code>language/</code>+<code>core/</code> examples passing · gated per file, not by a total</div></div>
<div><div class="n">23,672</div><div class="l">ruby/spec <code>language/</code>+<code>core/</code> examples passing · gated per file, not by a total</div></div>
<div><div class="n">178</div><div class="l">go-ruby-* modules bound into rbgo</div></div>
<div><div class="n">CGO=0</div><div class="l">across 6 arches · 3 OSes</div></div>
</div>
Expand Down
Loading