From 76a2b9072c7006edacaacff6ce217b4e0450e254 Mon Sep 17 00:00:00 2001
From: tannevaled <tannevaled@users.noreply.github.com>
Date: Tue, 10 Nov 2026 15:29:49 +0100
Subject: [PATCH] Say what the unswept trees hold, now that they have been
 measured
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The page said the unswept trees are "not run at all" and stopped there,
which leaves a reader unable to tell whether the unmeasured half of
ruby/spec is a rounding error or the bulk of the work.

Measured once on 2026-11-10 against MRI 4.0.7 through the same shim — a
dated run, said to be one, and not a gate. The gap across the 970 files
where the two engines differ is 2,341 examples, and the shape matters
more than the size:

  - 78 files cannot be measured at all: every command_line/ spec is built
    on ruby_exe and optional/capi needs a compiled C extension, so MRI
    itself scores zero in them;
  - the largest single debt is a HANG rather than a missing feature —
    Net::FTP is 442 examples behind with 39 of its 56 files killed by the
    clock on rbgo and none on MRI;
  - the outright absences are comparatively small (Coverage 53, WeakRef 9).

Verified with the pinned command this lane runs:

    $ go run github.com/go-embedded-ruby/docs/tools/conformancefigure@v0.8.0 --root .
    6 total claim(s), 1 version claim(s) and 3 stubbed-helper mention(s) checked
    against the gate's 23672 at Ruby 4.0.9 (derived from RUBY_VERSION);
    the shim stubs output_to_fd, ruby_cmd, ruby_exe

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---
 layouts/_default/list.html | 1 +
 1 file changed, 1 insertion(+)

diff --git a/layouts/_default/list.html b/layouts/_default/list.html
index bbea461..7303141 100644
--- a/layouts/_default/list.html
+++ b/layouts/_default/list.html
@@ -72,6 +72,7 @@ <h3>Runtime conformance — ruby/spec</h3>
         <ul>
           <li>CI gates <b>file by file</b>, not on a total: no spec file may pass fewer examples than its recorded row, and none may stop loading. The sum of those rows is <b>23,672</b>, re-recorded at Ruby 4.0.9 (2026-11-03) on the pinned runner that enforces it — it is what CI guarantees, while the measured total above is what one dated run did. One file (<code>core/random/bytes_spec.rb</code>) is unstable between 8 and 9 passing examples; since 2026-11-04 its row holds <b>8</b>, its floor, because a row recorded at the high draw makes the next ordinary run a red build. This replaced a single shrink-only floor of 22,475, which needed a margin wider than the largest spec file and so could not see a regression smaller than that.</li>
           <li><b>Only <code>language/</code> and <code>core/</code> are swept.</b> ruby/spec's <code>library/</code>, <code>optional/</code>, <code>command_line/</code> and <code>security/</code> trees are not run at all, so this figure is <b>not</b> a percentage of Ruby, and most of the standard library sits outside it.</li>
+          <li>Those unswept trees were measured <b>once</b>, on 2026-11-10, against MRI 4.0.7 through the same shim — a dated run, not a gate. Across the 970 files where the two engines differ the gap is <b>2,341</b> examples, and the shape matters more than the size: <b>78 files cannot be measured at all</b> (every <code>command_line/</code> spec is built on <code>ruby_exe</code>, and <code>optional/capi</code> needs a compiled C extension — MRI itself scores zero in them), the <b>largest single debt is a hang rather than a missing feature</b> (<code>Net::FTP</code> is 442 examples behind, with 39 of its 56 files killed by the clock on rbgo and none on MRI), and the outright absences are comparatively small (<code>Coverage</code> 53 examples, <code>WeakRef</code> 9).</li>
           <li>Of the examples that ran, <b>93.3%</b> pass (22,488 of 24,116); 477 more are skipped because the shim stubs <code>ruby_exe</code>, <code>ruby_cmd</code> and <code>output_to_fd</code> — three helpers that would need a subprocess or real file-descriptor redirection — and 10 of 2,206 spec files produce no result. Skips and unreadable files sit outside the ratio entirely.</li>
           <li>The guarantee has climbed from a floor of <b>6,000</b> on 2026-08-03 to a per-file baseline summing <b>23,672</b>. Two earlier jumps were fixes to the <b>measurement</b> — the mspec shim, and a parser upgrade that made 13 previously-unparseable spec files visible — which are gains in what the measurement can <i>see</i>, not in what the VM can <i>do</i>.</li>
         </ul>