diff --git a/content/en/_index.md b/content/en/_index.md index 8449d16..ccbaa5d 100644 --- a/content/en/_index.md +++ b/content/en/_index.md @@ -10,13 +10,15 @@ the lexer, parser and compiler embedded in the binary. Embed it in a Go program, ship it as a single static executable that cross-compiles wherever Go does, with no C toolchain and no libruby. -**How complete is it?** It passes **23,511** of ruby/spec's `language/` + `core/` +**How complete is it?** It passes **23,672** of ruby/spec's `language/` + `core/` examples — a large and growing subset of Ruby, and **not** a drop-in replacement for CRuby. That count is taken with ruby/spec's own version guards set to resolve at -Ruby **3.4.1**, the version this runtime reports: every example behind a -`ruby_version_is "4.0"` guard is removed from the suite before anything is -counted, so the figure describes 3.4-shaped behaviour even though the project -targets 4.0 semantics — 61 block guards and 5 inline sites sit at that boundary +Ruby **4.0.9**, the version this runtime reports and the latest MRI release: the +examples behind a `ruby_version_is "4.0"` guard are counted, and those behind +`ruby_version_is ""..."4.0"` are no longer asked for. The move from 3.4.1 was +measured before it was made, both engines built from one commit minus the +version string: **+144 passing**, and **27 examples that went red** — a figure +no pass total shows ([ruby#799](https://github.com/go-embedded-ruby/ruby/issues/799)). [Conformance]({{< relref "conformance.md#where-it-stands-measured-2026-09-26" >}}) states exactly what that counts and what it leaves out, and diff --git a/content/en/conformance.md b/content/en/conformance.md index fdddab8..9849b87 100644 --- a/content/en/conformance.md +++ b/content/en/conformance.md @@ -154,12 +154,17 @@ workflow enforces it on every push and pull request. ### Where it stands — measured 2026-09-26 {#where-it-stands-measured-2026-09-26} {{< callout type="info" >}} -**The gate's current total is **23 511**** +**The gate's current total is **23 672**** -Counted with ruby/spec's version guards set to resolve at Ruby **3.4.1** — the -version `RUBY_VERSION` reports and the version the shim measures at. That -selects the population before any counting happens, so the total is not a -percentage of Ruby 4.0; see [Known limitations](#known-limitations). +Counted with ruby/spec's version guards set to resolve at Ruby **4.0.9** — the +version `RUBY_VERSION` reports and the version the shim measures at, since +`CUR_VERSION = RUBY_VERSION`. That selects the population before any counting +happens, so the total is still not a percentage of Ruby; see +[Known limitations](#known-limitations). + +The baseline was re-recorded at 4.0.9 **on the pinned runner that enforces +it**, not on a developer's machine — one laptop reads +9 against CI for no +change in the code at all. The table below is one **dated full sweep**, and a dated measurement going stale is not an error: it is what it says it is. The number that moves with @@ -168,7 +173,7 @@ quote, and it is read straight out of the gate's own file: ```console $ awk -F'\t' '/^OK\t/{n+=$3} END{print n}' scripts/conformance/rubyspec/BASELINE -23511 +23673 ``` The breakdown below — failing, erroring, skipped — needs a full sweep to @@ -201,7 +206,7 @@ same ten files producing no result. level **CI guarantees**; a measured total is **what rbgo did on a given run**. Quoting the guaranteed figure as the conformance result understates rbgo by whatever has landed since the record was last written; quoting a measured total -as a guarantee overstates it by the same amount. The sum is **23 511**, and +as a guarantee overstates it by the same amount. The sum is **23 672**, and both the repository README and this page read that figure out of the file mechanically, each with a check that fails if they disagree — which is why neither carries a release number beside it. The figure is verified @@ -254,7 +259,7 @@ Both are gains in what the measurement can **see**, not in what the VM can **do**. The rest of the climb is the VM: the floor went from **6 000** when the ratchet landed on 2026-08-03 ([#263](https://github.com/go-embedded-ruby/ruby/pull/263)) to **22 475** by the -time it was replaced by the per-file baseline, whose sum is **23 511**. +time it was replaced by the per-file baseline, whose sum is **23 672**. ### Known limitations {#known-limitations} @@ -300,7 +305,7 @@ the call returns successfully and nothing says the guarantee is missing: | | | | --- | --- | -| `RUBY_VERSION` reports `"3.4.1"`, though the differential oracle is MRI 4.0.7, the pinned corpus is 4.0.x, and the latest MRI release is 4.0.9 (2026-10-23). That is not only cosmetic: the shim's `CUR_VERSION` is pinned to the same `3.4.1`, so **61 block guards and 5 inline sites at the `ruby_version_is "4.0"` boundary never run**, while **41 guards that end *before* 4.0 are being asserted**. Guard counts are exact, censused by `scripts/conformance/rubyspec/cmd/guards`; the example count behind them is not stated here because a static scan cannot expand `it_behaves_like`, and the figure of 167 quoted when this was first reported does not reproduce — it needs a CI sweep that moves `CUR_VERSION` without regenerating the baseline, so the ratchet names every file that moves in both directions | [#799](https://github.com/go-embedded-ruby/ruby/issues/799) | +| **Resolved 2026-11-03.** `RUBY_VERSION` now reports `"4.0.9"`, the latest MRI release, and `CUR_VERSION = RUBY_VERSION` so the shim cannot disagree with it. The sweep this entry asked for was run — both engines built from **one commit minus the version string**, over `core` and `language` together: **+144 passing, 45 files moved, 27 examples that WENT RED**. The 27 are the move's real remaining work and are listed per item on the issue. Two cautions the figures carry: `+144` is a **net** and `13 regressed` is not 13 defects — six of the thirteen lost only examples the corpus stops asking for, each with a `CLOSED` guard in the census, and `core/objectspace/_id2ref_spec.rb` reading `10 → 0` is eleven examples below 4.0 against **one** above it | [#799](https://github.com/go-embedded-ruby/ruby/issues/799) | | `Errno` carries all **158** of MRI's constant names, but **32** report errno `0` where MRI has a real number (`EAUTH`, `EBADRPC`, `EDEVERR`, …). By set difference on darwin: 81 names are zero-valued under rbgo, 49 under MRI, and every MRI zero is also zero under rbgo | [#633](https://github.com/go-embedded-ruby/ruby/issues/633) | | `Thread#backtrace` answers for the **current** thread only; another thread raises `NotImplementedError` | — | | `Process.fork` **does not exist** — Go's runtime cannot be forked safely | — | diff --git a/content/en/roadmap.md b/content/en/roadmap.md index 90a252c..eb5b95e 100644 --- a/content/en/roadmap.md +++ b/content/en/roadmap.md @@ -19,6 +19,6 @@ building whole subsystems in isolation. There are nine phases, 0 through 8. | 5 | Full front-end | Complete the lexer/parser/compiler to full Ruby 4.0 syntax (or settle the Prism-on-wazero decision). **Landed:** string interpolation, keyword args, endless methods (`def foo = expr`), beginless/endless ranges, the stabby lambda, **multiple assignment / destructuring**, **full pattern matching `case`/`in`** (value/bind/class/array/hash/find/pin/alternative patterns, guards, one-line `=>`/`in`, `deconstruct`/`deconstruct_keys`), **safe navigation `&.`**, **numbered params (`_1`/`_2`) and `it`**, **heredocs** (`<<`/`<<-`/`<<~`, interpolating and literal), **`%w`/`%i` arrays**, and the **`%q`/`%Q`/`%W`/`%I` literals**. | **Done** (core) | | 6 | Standard library | Pure-Go stdlib coverage, with the regexp engine from the `go-ruby-regexp` sibling org. **Regexp bridge landed:** `/re/imx` literals + `Regexp`/`MatchData` (`source`/`match`/`match?`/`=~`/`===`, `[]` by index or name, `pre_match`/`post_match`/`captures`/`named_captures`/`begin`/`end`/`size`), `String#=~`/`match`/`match?`/`scan`, and `sub`/`gsub`/`split` over a Regexp (backref templates, blocks, captured-group interpolation, field limit). CGO=0 via `go-ruby-regexp`; differential-tested against MRI. **Also landed:** `Time`/`Date`/`BigDecimal`, `Set`/`Bag`, and the Go-native leaves `JSON` (generate/parse/pretty_generate + `to_json`), `Digest` (MD5/SHA1/SHA256/SHA512), `Base64`, **`File`** (path helpers + read/write/stat/delete, `Errno::ENOENT`), a **MRI-faithful `Random`** (seeded MT19937, bit-exact), **`Zlib`** (crc32/adler32 + Deflate/Inflate), a **byte-exact `Marshal`** (`dump`/`load`, on the pure-Go `go-ruby-marshal` engine), **`Thread`/`Mutex`/`Queue`** on an emulated GVL (one Ruby thread at a time, MRI memory model, race-free), **`IO`/`StringIO`** (`$stdout`/`$stderr`/`$stdin` as real objects; `puts`/`print`/`p` route through `$stdout` so reassigning it to a `StringIO` captures output) and **`Dir`** (entries/children/glob, mkdir/rmdir/chdir, `Errno::ENOENT`/`Errno::EEXIST`), and **`File.open` streams** (`File < IO`, modes `r`/`w`/`a`/`+`, block form auto-flush/close, `File.readlines`/`foreach`). | **Substantially landed** | | 7 | Build toolchain | `rbgo build`. **Landed:** ahead-of-time **method compilation** to native Go (bytecode → Go → machine code), with **unboxed `int64` integer kernels** that deopt to the interpreter on overflow/`÷0` — the generated `fib(30)` beats **MRI + YJIT ~4×** while staying correct for every input ([AOT compiler]({{< relref "architecture/build.md#ahead-of-time-method-compilation-landed" >}})). **Closed-world builds landed** (`rbgo build --closed`): the program is baked in as bytecode (and the prelude loads from frozen bytecode), the **lexer/parser/compiler are dropped** behind the `rbgo_closed` build tag, and a require-graph scan reports any `eval`/`require` that would then raise — yielding a smaller, self-contained binary that runs with no source file. (Per-class stdlib tree-shaking beyond the front-end was measured and skipped — the Go-implemented bindings are small pure-Go libraries, under 2% of a runtime-dominated binary; the front-end drop is the real saving.) | **Done** | -| 8 | Conformance & performance | ruby/spec conformance and representation/perf work (tagging, fast paths, allocation). **Started:** allocation cuts — **small-integer interning** (`object.IntValue` reuses a shared box for ints in [-256, 1024]; a small-int loop drops from ~245k allocs to 1) and **frame-environment recycling** (`exec` pools per-call `*Env`s via a capture-tracked free-list — fib's call allocations halve, ~14% faster). **Conformance & benchmark ladder:** a three-way differential oracle (rbgo / **MRI** / **JRuby**, `scripts/oracle.sh`), then pure-Ruby gem test suites as corpora (ActiveSupport `core_ext`), then real-world workloads. A heavyweight front-end sweep now accepts **~100% of all valid Ruby** across Rails + Puppet (**parse + compile** Rails 99.82% / Puppet 100%, 0 over-permissive), and on the runtime side **the real `puppet apply` CLI runs end-to-end** (Puppet 8.11.0): `require "puppet"` boots the framework and the genuine `Puppet::Util::CommandLine` → `Puppet::Application::Apply` path (real `OptionParser`, all types + providers, settings catalog applied to disk, then the user catalog via the transaction / RAL) applies a `notify` resource and emits real Puppet output (`Notice: hi from rbgo cli`), exiting 0. **Runtime conformance, since July:** the ruby/spec `language/` + `core/` suites run in CI against a **per-file** baseline, which names any spec file that regresses instead of absorbing it in a total; its sum is **23 511**. It replaced a single shrink-only floor that had climbed from **6 000** (2026-08-03) to 22 475 — a threshold needed a margin wider than the largest spec file, so it could not see a regression smaller than that — see [Conformance]({{< relref "conformance.md#where-it-stands-measured-2026-09-26" >}}). **Active next milestone:** real file/package/service resource *providers* (host convergence) + run-report persistence. | **In progress** | +| 8 | Conformance & performance | ruby/spec conformance and representation/perf work (tagging, fast paths, allocation). **Started:** allocation cuts — **small-integer interning** (`object.IntValue` reuses a shared box for ints in [-256, 1024]; a small-int loop drops from ~245k allocs to 1) and **frame-environment recycling** (`exec` pools per-call `*Env`s via a capture-tracked free-list — fib's call allocations halve, ~14% faster). **Conformance & benchmark ladder:** a three-way differential oracle (rbgo / **MRI** / **JRuby**, `scripts/oracle.sh`), then pure-Ruby gem test suites as corpora (ActiveSupport `core_ext`), then real-world workloads. A heavyweight front-end sweep now accepts **~100% of all valid Ruby** across Rails + Puppet (**parse + compile** Rails 99.82% / Puppet 100%, 0 over-permissive), and on the runtime side **the real `puppet apply` CLI runs end-to-end** (Puppet 8.11.0): `require "puppet"` boots the framework and the genuine `Puppet::Util::CommandLine` → `Puppet::Application::Apply` path (real `OptionParser`, all types + providers, settings catalog applied to disk, then the user catalog via the transaction / RAL) applies a `notify` resource and emits real Puppet output (`Notice: hi from rbgo cli`), exiting 0. **Runtime conformance, since July:** the ruby/spec `language/` + `core/` suites run in CI against a **per-file** baseline, which names any spec file that regresses instead of absorbing it in a total; its sum is **23 672**. It replaced a single shrink-only floor that had climbed from **6 000** (2026-08-03) to 22 475 — a threshold needed a margin wider than the largest spec file, so it could not see a regression smaller than that — see [Conformance]({{< relref "conformance.md#where-it-stands-measured-2026-09-26" >}}). **Active next milestone:** real file/package/service resource *providers* (host convergence) + run-report persistence. | **In progress** | See [Phase 0 — Vertical slice]({{< relref "phases/phase0.md" >}}) for what runs today. diff --git a/tools/conformancefigure/main.go b/tools/conformancefigure/main.go index e748737..abdadcf 100644 --- a/tools/conformancefigure/main.go +++ b/tools/conformancefigure/main.go @@ -106,8 +106,24 @@ var ( // Thousands may be separated by a space (the docs' style) or a comma (the // landing page's and the README's), and the figure may be wrapped in Markdown // bold or HTML tags. +// +// "total is" and the "sum of is" shape were added on 2026-11-04. The +// run that refreshed the figure for 23,672 reported "4 total claim(s) checked" +// on a docs tree that held FIVE and "5 … checked" on a landing tree that held +// SIX. The two it could not see said "The gate's current total is 23 673" and +// "The sum of those rows is 23,673": the first used a word the alternation did +// not have, the second put two words between "sum" and "is", and the +// alternation spelled only the three determiners it had met so far ("the", "its", +// "whose"). Nothing was wrong with the verdicts it did reach; the stale lines +// were simply never candidates, and only the printed denominator -- against a +// grep of the same trees -- made the gap visible. A vocabulary list is a +// filter, so every phrasing missing from it is a figure the gate cannot refuse. +// +// Nothing speculative goes in this list: each alternative is a phrasing that +// exists in the two sites today, because a word added for a sentence nobody +// writes is a false positive waiting for the sentence that does. var ( - claimBefore = regexp.MustCompile(`(?i)\b(it passes|the sum is|its sum is|whose sum is|summing|passes)\b[^0-9]{0,24}\*{0,2}(\d{1,3}[ ,]?\d{3})\*{0,2}`) + claimBefore = regexp.MustCompile(`(?i)\b(it passes|passes|summing|(?:sum|total)\s+of\s+[a-z ]{1,24}\s+is|(?:sum|total)\s+is)\b[^0-9]{0,24}\*{0,2}(\d{1,3}[ ,]?\d{3})\*{0,2}`) claimAfter = regexp.MustCompile(`(?i)\**(\d{1,3}[ ,]?\d{3})\**(?:|)?[^.!?0-9]{0,90}?\bexamples?\s+(?:pass|passing)\b`) ) @@ -182,6 +198,14 @@ var vocabularyCases = []struct { {"**How complete is it?** It passes **23,511** of ruby/spec's `language/`", true, "23,511"}, {"baseline, whose sum is **23 511** at v0.3.0 (2026-10-11).", true, "23 511"}, {"as a guarantee overstates it by the same amount. The sum is **23 511** as of", true, "23 511"}, + // ⛔ These two sat in conformance.md and on the landing page with a stale + // figure while the tool reported four and five claims checked. The first + // says "total is", a word the alternation did not have; the second puts + // two words between "sum" and "is", which the three spelled-out + // determiners could not cross. Neither was ever a candidate, so both runs + // looked complete. + {"**The gate's current total is **23 511****", true, "23 511"}, + {"
  • CI gates file by file, not on a total: … The sum of those rows is 23,511, re-recorded at Ruby 4.0.9
    • ", true, "23,511"}, // Figure first: the landing page's style, in all three shapes it uses. {`
    23,511 ruby/spec language/+core/ examples passing`, true, "23,511"}, {`

    23,511 ruby/spec language/ + core/ examples pass, gated in CI file by file`, true, "23,511"},