Skip to content

FilterList.apply_to_* truncate output at the first NUL byte #1507

Closed

Description

@Grinv

FilterList.apply_to_buffer, apply_to_file and apply_to_blob return ffi.string(buf.ptr), which stops at the first NUL byte, so binary data comes back truncated.

It shows up for any binary file that gets a filter list, e.g. with core.autocrlf=true or * text=auto in .gitattributes. The crlf filter passes binary data through unchanged, and pygit2 then cuts it at the first NUL:

import tempfile

import pygit2

repo = pygit2.init_repository(tempfile.mkdtemp())
repo.config['core.autocrlf'] = True
fl = repo.load_filter_list('image.png')
data = b'\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR'
print(fl.apply_to_buffer(data))  # b'\x89PNG\r\n\x1a\n'

Seen in 1.19.2 through 1.21.0 (libgit2 1.9.7). In GitFourchette, the preview of an unstaged image in such a repo shows "0 × 0 pixels, 8 bytes".

Reading the buffer with its length fixes it: ffi.buffer(buf.ptr, buf.size)[:] in all three methods. With that change and a test that runs NUL bytes through all three, the suite passes (677 passed, 6 skipped). I can open a PR if you'd like.

Repository.raw_message (repository.py:1298) has the same ffi.string(buf.ptr) and cuts MERGE_MSG at a NUL. That one hardly matters, since git refuses to commit a message containing NUL.

Prepared with Claude Code; I reviewed the changes.

Activity

  1. jorio commented on Nov 5, 2026

    @jorio
    Contributor

    @Grinv: Nice find. I wrote these filter functions, so that was my mistake. I would love to see the PR.

  2. added a commit that references this issue on Nov 5, 2026
    06f8788
  3. Grinv commented on Nov 5, 2026

    @Grinv
    ContributorAuthor

    Opened #1510. Besides the three FilterList methods it fixes Repository.raw_message, which had the same truncation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions