Skip to content

Add zero-copy PyString wrapper - #831

Open
cjdoris wants to merge 1 commit into
mainfrom
codex/add-pystring-wrapper-type
Open

cjdoris wants to merge 1 commit into
mainfrom
codex/add-pystring-wrapper-type

Conversation

@cjdoris

@cjdoris cjdoris commented Oct 3, 2026

Copy link
Copy Markdown
Member

Motivation

  • Provide a zero-copy wrapper for Python str values so Julia code can access UTF-8 bytes without allocating or copying Python data.
  • Reuse and generalize the existing UTF-8 string iteration/validation logic so both the new wrapper and StaticString share correct behaviour.

Description

  • Add exported PyString <: AbstractString (stores the underlying Py object, a Ptr{UInt8} to the UTF-8 data and its byte length) in src/API/types.jl and implement the wrapper in src/Wrap/PyString.jl.
  • Expose the C API binding PyUnicode_AsUTF8AndSize in src/C/pointers.jl and use it to construct PyString with a pointer+length without copying.
  • Generalise the UTF-8 helpers in src/Utils/Utils.jl by introducing utf8_isvalid and utf8_iterate and adapt StaticString helpers to reuse them, and add bounds-checked codeunit for PyString.
  • Register a normal-priority pyconvert rule mapping builtins:str → PyString in src/Wrap/Wrap.jl so PyString conversion is available while preserving the existing canonical String conversion rules; export PyString via src/API/exports.jl.
  • Add tests exercising ASCII, multi-byte Unicode, embedded NULs, empty strings, bounds, invalid-UTF8 detection, iteration and conversion priority in test/Convert.jl and test/Utils.jl.

Testing

  • Ran package tests with julia --project -e 'using Pkg; Pkg.test()' and the test suite completed with PythonCall tests passing (test summary shows package tests passed while two pre-existing tests were marked broken in the environment).
  • Ran Python tests with cp pysrc/juliacall/juliapkg-dev.json pysrc/juliacall/juliapkg.json && uv run pytest -s --nbval ./pytest and all Python tests passed (21 passed).
  • Ran coverage with julia --project -e 'using Pkg; Pkg.test(coverage=true)' and the coverage run completed; the new PyString implementation and the edited UTF-8 utility code have executable lines covered by tests.
  • Added unit tests in test/Convert.jl and test/Utils.jl that passed during the above runs.

Codex Task

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant