Skip to content

Redesign v1 pyconvert dispatch and native conversions - #849

Open
cjdoris wants to merge 7 commits into
v1from
codex/v1-pyconvert-redesign
Open

cjdoris wants to merge 7 commits into
v1from
codex/v1-pyconvert-redesign

Conversation

@cjdoris

@cjdoris cjdoris commented Nov 8, 2026 •

Copy link
Copy Markdown
Member

Redesign pyconvert around a global Python source-type DAG, destination scopes, stable rule IDs, cached selections, and explicit conversion contexts. Canonical conversions prefer native materialized containers while generic iterators remain lazy. Add source-specific preferences, lazy public type aliases and shorthand such as "list" => PyList and "None" => Missing.

Rules and public pytryconvert share a normalized result convention: nothing means failure, ordinary successes are raw values, and successful nothing/Some values are wrapped once. After checking for failure, something(result) recovers the value. This avoids unnecessary result allocations while preserving concrete and union return bounds. Exceptions propagate.

Restrict generic floating-point coercions to Python float/complex sources so integer overflow cannot succeed after rounding through a float. Integral/rational sources retain exact intermediates; supported NumPy scalars retain dedicated rules. Update conversion documentation, migration guidance, and invariant tests.

Validation: 1,812 focused conversion/wrapper/Julia-wrapper checks passed with NumPy installed (one existing optional-dependency skip); 1,694 passed with bare system Python (four expected optional-dependency skips). Full-suite and supported-version coverage run in CI. NumPy-specific tests respect its optional status in external-interpreter jobs.

Reproducible benchmark scripts are included; generated outputs were removed. The latest direct allocation check saves 16 bytes/one allocation for public String, BigInt and array try-conversion; concrete Int/Float64 remain allocation-free. Earlier original-v1 comparisons showed inferred large-vector conversion improving by roughly one third, and array allocation totals matching original v1 after minimal rule-result wrapping. The remaining major measured regression is cold registry rebuilding (~2.6 ms versus ~28 μs in the synthetic fixture), plus slower simple selection misses. Warm calls do not rebuild the DAG.

Related to #661.

Comment thread src/JlWrap/JlWrap.jl Resolved
Comment thread src/Wrap/Wrap.jl Resolved
Comment thread src/PythonCall.jl Resolved
Comment thread src/Wrap/Wrap.jl Resolved

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant