Repository navigation
Net::FTP.default_passive, and the options a nil host was throwing away - #928
Merged
Merged
Conversation
…hrinks #921 bounded the control connection and #924 released the GVL around it. Both left the DATA connection holding it, and the gate #924 added said so in its seeded backlog -- four entries I wrote down rather than fixed, so the gate could land without a sprawling change. This is the change. dialData, acceptData, sendData, retrlines and retrbinary now run their blocking call inside ftpIO, and the four entries are DELETED from testdata/gvl-blocking-backlog.txt. That list may only shrink, and an entry that stops reporting fails the gate too, so a fix cannot leave its allowance behind. ⛔ acceptData is the one #916 is actually about. In ACTIVE mode the client listens and the server dials back, so a Ruby program that serves itself has both ends in one process -- and holding the GVL in that Accept parks every other Ruby thread, INCLUDING the one that would make the call it is waiting for. ruby/spec's net-ftp fixtures are exactly that shape, and their spec_helper.rb puts Net::FTP into active mode (see #916's comment: rbgo has no Net::FTP.default_passive, so today that helper is skipped in silence and the suite never reaches this path at all). Only the blocking call goes inside each closure. ftpConnErr raises, and a panic crossing a released GVL would unwind without re-acquiring it, so every raise stays outside. ⛔ NO BEHAVIOURAL TEST, AND THE REASON IS NOT LAZINESS. The in-repo FTP test server is Go-side, so it cannot reproduce this deadlock at all: nothing about rbgo's emulated GVL blocks a Go goroutine, and a test built on it would pass whether or not the fix is present. A real regression test needs BOTH ends in Ruby -- a dummy FTP server in a Ruby Thread, which is what ruby/spec's fixture is -- and that is worth doing properly rather than approximating with something that looks like proof and is not. What does guard the property is the gate: TestEveryBlockingSocketCallReleasesTheGVL fails in about a second if any of these five is unwrapped again, and the backlog deletion means it also fails if one silently stops being reported. The measurement that would show this moving -- the library/net-ftp sweep -- is deferred: this machine has been carrying a 1-minute load average between 23 and 86, and a per-file timeout measured under that cannot tell a hang from a slow file. Refs #916 Co-Authored-By: Claude Opus 5 <[email protected]>
ruby/spec's library/net-ftp/spec_helper.rb — which all 56 files require —
opens with:
if defined?(Net::FTP.default_passive)
Net::FTP.default_passive = false
end
rbgo had no such accessor, so the guard was skipped in SILENCE and the whole
suite ran against a connection rbgo kept in PASSIVE mode, while the fixture
server implements PORT/EPRT and has no `pasv` method at all. That is the
`451 Exception: undefined method 'pasv'` those specs report (#916).
Implemented against the reference, not against expectations: net-ftp 0.3.9
lib/net/ftp.rb:96 (`@@default_passive = true`), :159-167 (the accessors) and
:286-290 (`if options[:passive].nil?` → the default). The value lives in the
CLASS VARIABLE MRI uses, so Net::FTP.class_variable_get(:@@default_passive)
answers here as it does there and there is one place holding it, not two that
can disagree.
Two defects the reference comparison turned up that reading our own code would
not have:
- ⛔ `Net::FTP.new(nil, passive: true)` answered FALSE. ftpInit returned
before reading the options Hash whenever the host was nil, so EVERY option
was dropped on that path — binary, timeouts, credentials, all of it. MRI
applies the options first and connects separately, and ruby/spec drives
exactly that shape: `@ftp.send(:initialize, nil, options)` then asserts
`passive` (initialize_spec.rb:201-226). A mechanism wired at only some of
its doors, again.
- ⛔ `passive: nil` meant FALSE. ftpApplyOpts tested the KEY's presence, but
MRI tests the VALUE: nil is "no opinion" and falls back to the default.
Note this is the OPPOSITE of read_timeout in the same Hash three lines
away, where nil IS a request (for no bound) — easy to conflate, so both
now say which they are in a comment.
Tests: a 12-line table run through rbgo, each line annotated with the
reference line it comes from, plus ftpDefaultPassive's fallbacks (nil class,
absent or nil class variable → true), which no Ruby program can reach and
which decide the transfer mode of every connection.
⛔ Measured before pushing with the tool added in #925, which caught my own
gap: `covratchet -require ftpDefaultPassive,ftpApplyOpts,ftpInit` first
reported ftpDefaultPassive at 66.7%, and reports all three at 100% now.
Refs #916
Co-Authored-By: Claude Opus 5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Net::FTP.default_passive, and the options a nil host was throwing away
Stacked on #927 — the data-connection GVL release has to land first, and the
reason is in that pull request: this change moves the net-ftp specs into
active mode, which is where the
Accept#927 wraps actually runs.The gap
library/net-ftp/spec_helper.rb, which all 56 files require, opens with:rbgo had no such accessor, so that guard was skipped in silence and the
whole suite ran against a connection rbgo kept in passive mode — while the
fixture server implements
PORT/EPRTand has nopasvmethod at all.That is the
451 Exception: undefined method 'pasv'those specs report.Implemented against the reference, not against expectations
net-ftp 0.3.9
lib/net/ftp.rb:96 (@@default_passive = true), :159-167 (theaccessors), :286-290 (
if options[:passive].nil?→ the default). The valuelives in the class variable MRI uses, so
Net::FTP.class_variable_get(:@@default_passive)answers here as it doesthere, and one place holds it rather than two that can disagree.
⛔ Two defects the comparison found that reading our own code would not
Net::FTP.new(nil, passive: true)answeredfalse.ftpInitreturnedbefore reading the options Hash whenever the host was nil — so on that path
every option was dropped: binary, the timeouts, the credentials, all of it.
MRI applies the options first and connects separately, and ruby/spec drives
exactly that shape:
@ftp.send(:initialize, nil, options)and then assertspassive(initialize_spec.rb:201-226). A mechanism wired at only some of itsdoors, again — the ninth this week.
passive: nilmeantfalse.ftpApplyOptstested the key's presence;MRI tests the value, and
nilis "no opinion" falling back to the default.This is the opposite of
read_timeoutin the same Hash three lines away,where
nilgenuinely is a request (for no bound). Both now say which they are,in a comment, because that is precisely the pair someone will conflate next.
Tests and measurement
A 12-line table driven through rbgo, every line annotated with the reference
line it comes from — including that a connection built before
default_passive =keeps its mode, and that the writer returns its argument.Plus
ftpDefaultPassive's fallbacks (nil class, absent or nil class variable →true), which no Ruby program can reach and which decide the transfer mode ofevery connection.
⛔ Coverage measured before pushing, with the tool added in #925 — which
caught my own gap:
covratchet -require ftpDefaultPassive,ftpApplyOpts,ftpInitfirst reported
ftpDefaultPassiveat 66.7%, and reports all three at100% now. That tool exists because twice in one day a commit written to
answer the coverage ratchet did not move the measurement.
Refs #916, #925, #927
(#916 stays open on purpose: the sweep that would show this moving is still
deferred — this machine has been carrying a 1-minute load average between 23
and 86, and a per-file timeout measured under that cannot tell a hang from a
slow file. #925 and #927 are the pull requests this one builds on.)
🤖 Generated with Claude Code