Repository navigation
Conversation
|
@codex review |
The entry names #9191, so the changelog points at this PR.
Changelog💅 Polish
Semantic Diff3 commits · Full page Let a build system start a request before its dependencies publish485eea8 · JonoPrest · Nov 09, 2026, 08:02 UTC
A request now takes hooks from the build system. interface_computed receives the contents of the .cmi an implementation is about to write, once no warning reported as an error is pending; the contents are computed once (Cmi_format.cmi_contents) and also written. redirect_cmi hands Bs_cmi_load the contents of a .cmi that is not published yet, which it reads instead of the build file store; the store never sees them. before_effect is called before the request writes a .cmi, .cmt, .cmti, binary AST or a PPX's input, runs a command such as a PPX, or takes a command line action that writes a file (-e, -format); a build system that started the request early waits there, or raises to abandon it before any effect. Build_file_store.key is exported so that a build system matches paths as the store does. bsc attaches no hooks and behaves as before. Review
Start dependents once a module's interface is final5a62ac0 · JonoPrest · Nov 09, 2026, 08:02 UTC
When a slot would otherwise idle, a module can now start as soon as each unfinished dependency's interface is final: an implementation without .resi has computed the .cmi it is about to write, or an interface file has compiled. The early request reads those .cmi files from memory and waits at a gate before writing a file, even one the build holds in memory until publishing, running a command or returning its result. The scheduler adopts it at the point it would have started it (dependencies complete, module still dirty), so its code generation sees the dependencies' .cmj files, and artifacts, diagnostics and compiler.log are unchanged. Otherwise it is abandoned without effects, together with anything that started on top of it. Redirected paths are matched as the build file store keys them. The directories in which the build publishes interfaces are spelled from canonical package roots, both in the redirected paths and on the load path, so a lookup there that matches no redirected path names another file. A lookup of a redirected file name in any other directory, which may be an alias from user flags (with -nostdlib those come first), or one that differs only in case, waits for adoption instead. Whether an announced interface changes the published one is read from disk, so that an abandoned request never fills the store. Modules with js-post-build commands or lowercase .cmi names don't release their dependents early, and with one domain nothing changes. Review
Build a .cmi's contents only when something reads them7e033ce · JonoPrest · Nov 09, 2026, 13:36 UTC
Review
|
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Codex Review: Didn't find any major issues. What shall we delve into next? Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
The entry names #9191, so the changelog points at this PR. Signed-off-by: Jono Prest <[email protected]>
d681ed4 to
93e6375
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #9191 +/- ##
==========================================
+ Coverage 82.07% 82.10% +0.02%
==========================================
Files 455 455
Lines 65356 65570 +214
==========================================
+ Hits 53643 53837 +194
- Misses 11713 11733 +20
🚀 New features to boost your workflow:
|
rescript
@rescript/belt
@rescript/darwin-arm64
@rescript/linux-arm64
@rescript/linux-x64
@rescript/runtime
@rescript/win32-x64
commit: |
|
Developer playground preview: https://rescript-lang.github.io/rescript/dev-playground/?version=pr-9191 |
|
@codex review |
|
Codex Review: Didn't find any major issues. Hooray! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
04dca39 to
7bcd9d0
Compare
|
@codex review |
7bcd9d0 to
8a61aaa
Compare
|
@codex review |
|
Codex Review: Didn't find any major issues. You're on a roll. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
8a61aaa to
4203498
Compare
✅ Compiler benchmarksAllocated wordsIn total: 683,546,137 → 683,702,634 (+0.02%). None of the 261 benchmarks changed by 0.1% or more. InstructionsIn total: 9,664,671,518 → 9,665,343,822 (+0.01%).
Output sizeIn total: 25,569,598 → 25,570,078 (±0.00%). None of the 13 benchmarks changed by 0.1% or more. Pull request 7e033ce compared with base b82d944 in this run. |
b6806c0 to
63af9ae
Compare
7e033ce to
4bf9701
Compare
Changelog💅 Polish
Semantic Diff3 commits · Full page Let a build system start a request before its dependencies publishbe2672a · JonoPrest · Nov 09, 2026, 08:02 UTC
A request now takes hooks from the build system. interface_computed receives the contents of the .cmi an implementation is about to write, once no warning reported as an error is pending; the contents are computed once (Cmi_format.cmi_contents) and also written. redirect_cmi hands Bs_cmi_load the contents of a .cmi that is not published yet, which it reads instead of the build file store; the store never sees them. before_effect is called before the request writes a .cmi, .cmt, .cmti, binary AST or a PPX's input, runs a command such as a PPX, or takes a command line action that writes a file (-e, -format); a build system that started the request early waits there, or raises to abandon it before any effect. Build_file_store.key is exported so that a build system matches paths as the store does. bsc attaches no hooks and behaves as before. Review
Start dependents once a module's interface is finald3293e6 · JonoPrest · Nov 09, 2026, 08:02 UTC
When a slot would otherwise idle, a module can now start as soon as each unfinished dependency's interface is final: an implementation without .resi has computed the .cmi it is about to write, or an interface file has compiled. The early request reads those .cmi files from memory and waits at a gate before writing a file, even one the build holds in memory until publishing, running a command or returning its result. The scheduler adopts it at the point it would have started it (dependencies complete, module still dirty), so its code generation sees the dependencies' .cmj files, and artifacts, diagnostics and compiler.log are unchanged. Otherwise it is abandoned without effects, together with anything that started on top of it. Redirected paths are matched as the build file store keys them. The directories in which the build publishes interfaces are spelled from canonical package roots, both in the redirected paths and on the load path, so a lookup there that matches no redirected path names another file. A lookup of a redirected file name in any other directory, which may be an alias from user flags (with -nostdlib those come first), or one that differs only in case, waits for adoption instead. Whether an announced interface changes the published one is read from disk, so that an abandoned request never fills the store. Modules with js-post-build commands or lowercase .cmi names don't release their dependents early, and with one domain nothing changes. Review
Build a .cmi's contents only when something reads themb763790 · JonoPrest · Nov 09, 2026, 13:36 UTC
Review
|
✅ Compiler benchmarksAllocated wordsIn total: 640,948,928 → 641,107,650 (+0.02%). Generated projects: 920,891,835 → 921,156,462 (+0.03%).
InstructionsIn total: 9,209,713,560 → 9,209,971,389 (±0.00%). None of the 33 benchmarks changed by 0.1% or more. Output sizeIn total: 25,843,406 → 25,843,913 (±0.00%). None of the 13 benchmarks changed by 0.1% or more. Pull request b763790 compared with base ed08a8c in this run. |
|
Developer playground preview: https://rescript-lang.github.io/rescript/dev-playground/?version=pr-9191 |
6ab7643 to
dc515a8
Compare
Why:
A dependent only needs a module's .cmi to type check, but a build
system that runs requests in process can only start it after that
module has also produced its lambda, JavaScript, .cmj and .cmt.
A request now takes hooks from the build system. interface_computed
receives the contents of the .cmi an implementation is about to write,
once no warning reported as an error is pending; the contents are
computed once (Cmi_format.cmi_contents) and also written. redirect_cmi
hands Bs_cmi_load the contents of a .cmi that is not published yet,
which it reads instead of the build file store; the store never sees
them. before_effect is called before the request writes a .cmi, .cmt,
.cmti, binary AST or a PPX's input, runs a command such as a PPX, or
takes a command line action that writes a file (-e, -format); a build
system that started the request early waits there, or raises to abandon
it before any effect. Build_file_store.key is exported so that a build
system matches paths as the store does. bsc attaches no hooks and
behaves as before.
Review: Can a redirected interface reach a later request through the
store or the decoded interface cache? Only as the same bytes. The
dependency writes the very string it announced through the store, which
holds it until the build publishes it, so a domain's cache may reuse an
interface it decoded from the redirect for the published file. A request
that is abandoned or fails drops its domain's cache.
Review: Why gate writes that the build file store already holds? The
store is shared by the requests of a build, and a request that is
abandoned may still run beside the request that replaces it, so a held
.cmi or .cmt it wrote could replace its successor's before publication.
Signed-off-by: Jono Prest <[email protected]>
Why:
On builds limited by a long dependency chain most job slots sit idle
on the critical path: a 136 module library gets only 1.6x from 7
domains.
When a slot would otherwise idle, a module can now start as soon as each
unfinished dependency's interface is final: an implementation without
.resi has computed the .cmi it is about to write, or an interface file
has compiled. The early request reads those .cmi files from memory and
waits at a gate before writing a file, even one the build holds in
memory until publishing, running a command or returning its result. The
scheduler adopts it at the point it would have started it (dependencies
complete, module still dirty), so its code generation sees the
dependencies' .cmj files, and artifacts, diagnostics and compiler.log
are unchanged. Otherwise it is abandoned without effects, together with
anything that started on top of it.
Redirected paths are matched as the build file store keys them. The
directories in which the build publishes interfaces are spelled from
canonical package roots, both in the redirected paths and on the load
path, so a lookup there that matches no redirected path names another
file. A lookup of a redirected file name in any other directory, which
may be an alias from user flags (with -nostdlib those come first), or
one that differs only in case, waits for adoption instead. Whether an
announced interface changes the published one is read from disk, so
that an abandoned request never fills the store. Modules with
js-post-build commands or lowercase .cmi names don't release their
dependents early, and with one domain nothing changes.
Review: Can a dependent see an interface that differs from master's?
It reads the exact contents the dependency then writes, and it is only
adopted after the dependency completed. If the dependency fails later,
for example in code generation, the dependent is abandoned before it
wrote, ran or reported anything.
Signed-off-by: Jono Prest <[email protected]>
Why:
Since the interface is announced before it is written, every .cmi
write built the whole file (the signature plus the marshalled CRCs and
flags), even with no build system listening and when `check_exists`
then found the same file on disk and wrote nothing. That is about
5,000 more words per runtime module in the compiler benchmarks, over
1% for the typed array modules.
`interface_computed` is now optional in `Compiler_request_state.hooks`,
and rewatch passes it only when a dependent may start early (as
`on_interface_computed` already was). Without it, `create_cmi` digests
the signature as before and builds the rest of the file only when it
writes it, as on master.
Review: Can a .cmi or a speculation change? No: the bytes are built by
the same functions in the same order, and a listener receives them
exactly as before. Only when nobody listens is the concatenation
skipped, and then only when the file is not written.
Signed-off-by: Jono Prest <[email protected]>
dc515a8 to
b763790
Compare
A dependent only needs a module's
.cmito type check, but OCaml rewatch starts it only after that module has also produced its lambda, JavaScript,.cmjand.cmt. On builds limited by a long dependency chain most job slots sit idle on the critical path.Rebased on the flat
lib/rescriptlayout (#9245).How it works
.resihas computed the.cmiit is about to write (and has no pending warning as error), or.cmicontents from memory (Compiler_request_state.redirect_cmi, checked inBs_cmi_loadbefore the build file store) and waits at a gate before it writes any file, runs any command (PPX) or returns a result..cmjfiles, so artifacts, diagnostics andcompiler.logare the same as on master.Held writes
The build file store now holds ASTs, interfaces and typed trees until their request succeeded. That doesn't replace the gate:
.cmjfiles, which aren't held and exist only once a dependency completed..cmior partial.cmtit wrote could replace its successor's before publication.The redirected path is the file the dependency writes and the store holds, so both serve the same string and the decoded interface cache stays valid after publication. Dating back the AST of a failed compile and publishing its partial typed tree happen only after adoption.
Redirected paths are matched as the store keys them (
Build_file_store.key). Each package'slib/rescriptis spelled from its canonical package root, on the load path too (-I .from the request's directory), so a lookup there that matches no redirected path is another file. A lookup of a redirected name anywhere else (an alias from user-Iflags), or one that differs only in case, waits for adoption instead of reading the disk. Whether an announced interface changes the published one is read from disk, so an abandoned request never fills the store. Modules withjs-post-buildcommands or lowercase.cminames don't release their dependents early. With one domain nothing changes.Performance
Clean builds, OCaml 5.5.2, 8 vCPU Linux VM, interleaved pairs against master. Median B/A with bootstrap 95% CI, a library of 132 modules with a deep dependency chain:
Projects bound by total CPU (a 552 module React app, a 391 module backend service) showed no change before the rebase.
Testing
.cmi,.cmjand.astare byte identical to master on three production codebases at default and 1 domain..cmtis byte identical at 1 domain and semantically equivalent at default (master's own.cmtbytes vary between builds on OCaml 5.5).Process(gate, adopt, abandon, failed dependency, interrupt), the compiler hooks (gate before every file and command, including-bs-ast -o,-e,-format, PPXs), redirects vs the store and its interface cache, scheduler eligibility and exclusions, redirect matching inlib/rescript, andcheck_pipelining.sh, which compares 1 and 4 domain builds byte for byte and checks speculation happened.REWATCH_SPECULATION_LOG(test only) records start, adopt and abandon.make test,make test-rewatch, the rewatch unit tests andrewatch-ocaml/tests/run.shpass.