Skip to content

Commit 1e92aac

Browse files
codexByron
authored andcommitted
fix(gix): preserve Git compatibility on Windows
The Windows suite with `GixPython 0.1.0` had 17 failures that the CLI-only package matrix did not exercise. Format paths returned by `Repository.common_dir()`, `workdir()` and `git_dir()` with the existing Windows separator adapter, including repository properties and worktree listings. Gix still resolves the locations, and POSIX filenames retain their literal backslashes. Recover from `RuntimeError("native worker panicked")` only during repository discovery, letting Git validate malformed metadata and provide its normal error. Propagate unrelated errors and errors after a native mutation starts. Keep the upstream discovery bug open as `GIX-14` in `doc/gix-backend.md`. Correct the gitfile and symlink controls to use the platform's Git behavior. Exercise status with portable filenames on Windows while retaining POSIX newline coverage. Compare direct-branch and symbolic-alias blob updates with Git, which rejects the former and accepts the latter in the tested versions. Add a Windows/Python 3.12 Gix job while preserving all 28 existing CLI jobs and check names. Prepare Rust for the released `GixPython` source build, cache `pip` wheels, verify backend selection and retain JUnit results plus backend operation counts for every job. Validation: - Full Windows Gix suite: 1,760 passed, 40 subtests passed, 58 skipped, nine expected failures and three unexpected passes in 2,242.66 seconds. The baseline reproduced all 17 failures; the unexpected passes and the path-deprecation warning also occurred there. Overlapping runs do not establish a performance improvement for these compatibility fixes. - Windows CLI regressions: 138 passed and 14 subtests passed. - Ubuntu WSL Gix regressions: 43 passed and 14 subtests passed, including both hash formats and POSIX newline filenames. - `ruff`, `mypy`, `basedpyright` and pinned `pre-commit` checks passed. Matrix expansion and existing GitHub check names were verified.
1 parent 63912db commit 1e92aac

5 files changed

Lines changed: 197 additions & 39 deletions

File tree

‎.github/workflows/pythonpackage.yml‎

Lines changed: 35 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,10 +14,12 @@ permissions:
1414

1515
jobs:
1616
test:
17+
name: test (${{ matrix.os-type }}, ${{ matrix.python-version }}${{ matrix.backend == 'gix' && ', gix' || '' }})
1718
strategy:
1819
matrix:
1920
os-type: [ubuntu, macos, windows]
2021
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12", "3.13", "3.14", "3.14t", "3.15", "3.15t"]
22+
backend: [cli]
2123
exclude:
2224
- os-type: macos
2325
python-version: "3.14t"
@@ -32,6 +34,11 @@ jobs:
3234
experimental: true
3335
- python-version: "3.15t"
3436
experimental: true
37+
- os-type: windows
38+
python-version: "3.12"
39+
backend: gix
40+
build-docs: false
41+
experimental: false
3542

3643
fail-fast: false
3744

@@ -51,6 +58,20 @@ jobs:
5158
with:
5259
python-version: ${{ matrix.python-version }}
5360
allow-prereleases: ${{ matrix.experimental }}
61+
cache: ${{ matrix.backend == 'gix' && 'pip' || '' }}
62+
cache-dependency-path: |
63+
pyproject.toml
64+
requirements.txt
65+
test-requirements.txt
66+
gix-requirements.txt
67+
68+
- name: Prepare Rust for GixPython's Windows source build
69+
if: matrix.backend == 'gix'
70+
run: |
71+
# GixPython 0.1.0 publishes macOS wheels only and needs Rust >= 1.89.
72+
rustup toolchain install stable --profile minimal
73+
rustup default stable
74+
rustc --version
5475
5576
- name: Set up WSL (Windows)
5677
if: matrix.os-type == 'windows'
@@ -89,7 +110,11 @@ jobs:
89110
90111
- name: Install project and test dependencies
91112
run: |
92-
pip install ./smmap ./gitdb '.[test]'
113+
pip install ./smmap ./gitdb ".[${EXTRAS}]"
114+
python -c 'import os; from git import _backend; print("Backend:", _backend.name); assert _backend.name == os.environ["EXPECTED_BACKEND"]'
115+
env:
116+
EXTRAS: ${{ matrix.backend == 'gix' && 'test,gix' || 'test' }}
117+
EXPECTED_BACKEND: ${{ matrix.backend }}
93118

94119
- name: Show POSIX file ownership
95120
# Linux and macOS only. On Windows, Git Bash's `ls -ld` reports a
@@ -172,9 +197,17 @@ jobs:
172197
173198
- name: Test with pytest
174199
run: |
175-
pytest --color=yes -p no:sugar --instafail -vv --durations=30
200+
pytest --color=yes -p no:sugar --instafail -vv --durations=30 \
201+
--junitxml=test-results/pytest.xml --backend-report=test-results/backend.json
176202
continue-on-error: false
177203

204+
- name: Retain test results and backend operation counts
205+
if: always()
206+
uses: actions/upload-artifact@v7
207+
with:
208+
name: tests-${{ matrix.os-type }}-${{ matrix.python-version }}-${{ matrix.backend }}
209+
path: test-results/
210+
178211
- name: Documentation
179212
if: matrix.build-docs
180213
run: |

‎doc/gix-backend.md‎

Lines changed: 74 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,11 @@ uv pip install --python .tox/gix/bin/python --editable '.[test,gix]'
1616

1717
`tox -e gix` also resolves the published release from the package index.
1818

19+
GixPython 0.1.0 publishes macOS wheels. On Windows and Linux, installation
20+
builds the released source distribution and requires Rust 1.89 or newer and
21+
a platform C/C++ toolchain. Windows CI uses the runner's MSVC tools and a
22+
stable Rust toolchain, with pip's built-wheel cache enabled.
23+
1924
Python extras add dependencies; they do not leave a runtime feature bit.
2025
Set `GIT_PYTHON_BACKEND=cli` or `GIT_PYTHON_BACKEND=gix` before importing
2126
GitPython. Explicit Gix selection fails if `gix` cannot be imported; an invalid
@@ -70,7 +75,8 @@ uv --cache-dir .cache/uv pip install --offline --no-index \
7075
For another machine, populate that directory from PyPI or its package cache
7176
before running the offline installation tests. Select wheels compatible with
7277
the interpreter and platform; ordinary and free-threaded CPython use different
73-
wheels. A local GixPython source build is no longer required.
78+
wheels. On platforms without a published wheel, first build one from the
79+
released source distribution and retain it in the wheelhouse.
7480

7581
## Conversion coverage
7682

@@ -683,13 +689,13 @@ identify the historical fixture where needed.
683689
| GIX-21 | Bug; missing API | `Repository.committer()` preserves a process identity of ` Name. ` / ` email@example.invalid ` while `git var GIT_COMMITTER_IDENT` returns `Name.` / `email@example.invalid`. Native snapshot overrides for name, email and date work, but modifying the retained snapshot would leak per-command identity across threads. GixPython 0.1.0 has no general independent repository clone: Python copying fails and internal `RepoHandle::clone()` shares its `Arc` state. | Expose Git-compatible identity cleanup and a cheap independent native clone with isolated config, including per-command removal semantics. Reopening isolates state but repeats setup; `with_object_memory()` changes object-write behavior and is not a general clone. Keep `_checked_signature` validation and CLI fallback for cleanup/overrides. Never clean identity fields in Python or mutate shared snapshots/process environment to handle a command. |
684690
| GIX-22 | Missing API | The native Git directory is sufficient for the two fixed metadata leaves used here: `modules` and `COMMIT_EDITMSG`. Git's `path.c` applies no special relocation to either, including in linked worktrees. The adapter appends only these names to the canonical native Git directory. | No new binding is needed for these ordinary paths. Other `--git-path` names and symlinked metadata leaves retain CLI resolution; a general resolver must handle config/environment overrides, common/private storage and canonical targets. `Repository.modules_path()` means `.gitmodules`, not the `modules` storage directory. |
685691
| GIX-23 | Missing API | No hook lookup or execution API is bound in GixPython 0.1.0. The removed absence fast path read config through Gix but used Python `os.stat()` to declare success. | Bind native hook lookup with configured/default path, linked-worktree, missing/nonexecutable-hook and diagnostic semantics, plus execution where needed. Until then all managed hook calls use Git, including `--ignore-missing` no-ops. A Python filesystem check is not a Gix implementation. |
686-
| GIX-24 | Bug | In both object formats, `new_commit_as()` accepts a blob as the tree or a parent where `git commit-tree` rejects it. Tree-editor `upsert()`/`write()` accepts object IDs whose actual kinds disagree with blob/tree/gitlink modes; `git mktree --missing` rejects all three tested mismatches. `edit_references_as()` accepts a blob target under `refs/heads/`, rejected by `git update-ref`. | Provide checked commit/tree construction and reference edits, at least in Git-strict mode. Validate kinds and branch targets, including symbolic aliases, before writing and under the required ref locks. Existing `_commit_tree`, `_write_tree` and `_update_ref` guards query Gix headers and select CLI on invalid inputs; they do not replace Gix writes with Python. |
692+
| GIX-24 | Bug | In both object formats, `new_commit_as()` accepts a blob as the tree or a parent where `git commit-tree` rejects it. Tree-editor `upsert()`/`write()` accepts object IDs whose actual kinds disagree with blob/tree/gitlink modes; `git mktree --missing` rejects all three tested mismatches. `edit_references_as()` accepts a blob target under `refs/heads/`, rejected by `git update-ref`. | Provide checked commit/tree construction and reference edits, at least in Git-strict mode. Match Git's kind and direct-branch target validation before writing and under the required ref locks, including its distinct handling of symbolic aliases. Existing `_commit_tree`, `_write_tree` and `_update_ref` guards query Gix headers and select CLI on invalid inputs; they do not replace Gix writes with Python. |
687693
| GIX-25 | Bug; missing API | After `git symbolic-ref refs/heads/dangling refs/heads/missing`, `Repository.references().all()` enumerates the dangling name, while `git for-each-ref --format=%(refname)` omits it. Valid symbolic aliases must remain present. Both SHA-1 and SHA-256 probes reproduce this difference. | Expose Git-compatible enumeration with the same dangling-reference and diagnostic behavior, retaining the raw iterator for callers that need it. `_for_each_ref` forces Gix to resolve symbolic targets and falls back to Git on failure; keep that fallback until a compatible Gix API/mode is verified. |
688694

689695
Windows reproduction for GIX-14 uses CPython 3.12.13, Git
690696
2.55.0.windows.3 and the released GixPython 0.1.0 source distribution with
691697
the same pinned Gitoxide revision above. Both `105114db` and the CLI metadata
692-
batching change reproduce the following existing failures:
698+
batching change (`f6779dbb`) reproduce the following existing failures:
693699

694700
- `test_native_command_queries_match_cli` and
695701
`test_native_worktree_inventory_includes_bare_main` return backslashes in
@@ -701,13 +707,30 @@ batching change reproduce the following existing failures:
701707
`InvalidGitRepositoryError`. The binding must reject this input through its
702708
normal native error contract so discovery can take the existing fallback.
703709

704-
These remain open compatibility bugs. The optional Windows selection also
705-
has tests that fail before a native/Git comparison: newline-containing
706-
filenames, a raw backslash gitfile operand, and a POSIX-specific symlink
707-
canonicalization assertion. Its symbolic-alias blob assertion is not evidence
708-
of a Gix mismatch on this Git version: direct Git controls in both object
709-
formats also accept the alias update and leave the branch pointing to a blob.
710-
This does not invalidate GIX-24's separate direct-branch validation comparison.
710+
The adapters now format locations returned by `Repository.common_dir()`,
711+
`workdir()` and `git_dir()` with the existing Windows-only path separator
712+
conversion, including `Repo.common_dir` and gitfile worktree paths. Gix still
713+
resolves every location. Ordinary, bare and linked repository metadata and
714+
worktree-list queries compare exactly against Git without additional CLI
715+
calls. POSIX filenames containing literal backslashes are unaffected.
716+
717+
Discovery catches only the binding's exact
718+
`RuntimeError("native worker panicked")` and selects Git's existing validation
719+
path. The malformed metadata regression then raises `InvalidGitRepositoryError`.
720+
Unrelated runtime
721+
errors still propagate. Both normal native write errors and worker panics
722+
after a mutation begins are tested to ensure they never trigger a CLI retry.
723+
GIX-14 remains open: these adapter changes do not fix the upstream panic or
724+
the other discovery compatibility bugs.
725+
726+
The tests now pass gitfile operands in Git's Windows-compatible spelling and
727+
compare symlink resolution against the current platform's Git output. Status
728+
and ignore parity runs with a portable filename everywhere; only the extra
729+
newline-filename variant is skipped on Windows. Blob-reference updates also
730+
compare against Git: a direct branch rejects the blob, while a symbolic alias
731+
outside `refs/heads/` accepts it on Git 2.55.0.windows.3, in both object
732+
formats. That alias behavior is not evidence of a Gix mismatch and does not
733+
invalidate GIX-24's separate direct-branch comparison.
711734

712735
For GIX-8, at fixture commit `44e0a8ec55c42559dfcdf5117710b26261a7c937`,
713736
compare `git rev-list HEAD` with
@@ -754,7 +777,7 @@ probe and captured results are in
754777
`.cache/gix-compatibility-ledger/{probe-contracts.py,contracts.json}`.
755778
Existing [backend regressions](../test/test_gix_backend.py) protect the adapter
756779
fallbacks, including `test_tree_writes_validate_child_object_kinds`,
757-
`test_symbolic_alias_cannot_point_a_branch_at_a_blob`,
780+
`test_branch_blob_updates_match_git`,
758781
`test_commit_identity_cleanup_matches_git` and
759782
`test_reference_enumeration_preserves_aliases_and_skips_dangling_refs`.
760783
This ledger update changes no runtime code or CLI budgets; the earlier
@@ -816,8 +839,9 @@ reproduction or regression. A Gix-based workaround does not resolve that bug;
816839
require an upstream fix or a verified mode as strict as Git before closing it.
817840

818841
Decide whether to fall back before mutating anything. Read/preparation failures
819-
can use Git to preserve its public diagnostics. Once `_write()` begins, native
820-
errors become `GitCommandError` and must never trigger a second CLI mutation.
842+
can use Git to preserve its public diagnostics. Once `_write()` begins,
843+
`gix.Error` failures become `GitCommandError`; other exceptions propagate.
844+
Neither may trigger a second CLI mutation.
821845
Access native repository state through `Repo._get_gix_repository()` for bound
822846
commands. Reuse is the default, with explicit recreation available; keep the
823847
snapshot deviation and refresh behavior above documented. Close native iterators
@@ -827,6 +851,43 @@ Add each conversion with a no-subprocess check and a Git parity check where
827851
practical, in its own commit. Update this ledger when a limitation changes;
828852
the runtime report's reason should point to the corresponding entry.
829853

854+
## Windows validation
855+
856+
The [Python package workflow](../.github/workflows/pythonpackage.yml) adds a
857+
Windows/Python 3.12 Gix job while preserving all 28 CLI combinations and their
858+
existing check names. It installs `.[test,gix]`, verifies the selected backend
859+
and runs the full suite
860+
with the same coverage and pytest options. Every job retains JUnit results
861+
and `--backend-report` operation counts under its `tests-OS-PYTHON-BACKEND`
862+
artifact, and prints the 30 slowest test durations. The extra Gix job does not
863+
duplicate the documentation build.
864+
865+
Local Windows checks use CPython 3.12.13, Git 2.55.0.windows.3 and GixPython
866+
0.1.0 built from the released source distribution. The installed extension
867+
matches the cached Windows ABI3 wheel byte-for-byte; that wheel's SHA-256 is
868+
`1a0addd5f569e2ff7fb2b45c38be1093f6cabbead176c779533b983b86307dc2`.
869+
870+
The full baseline at `f6779dbb` reproduced all 17 failures described above:
871+
16 ordinary failures and the subtest with malformed `commondir`. The full
872+
patched suite passed with **1,760 tests and 40 subtests**, 58 skipped, nine
873+
expected failures and three unexpected passes in 2,242.66 seconds. The three
874+
unexpected passes and the path-deprecation warning also appeared in the
875+
baseline. Ruff, mypy, basedpyright and the pinned pre-commit checks passed.
876+
877+
The baseline recorded 21,598 CLI launches in 2,279.10 seconds; the patched
878+
run recorded 21,826 CLI launches and 86,936 native operations. Totals include the additional
879+
regressions and tests that previously stopped at failing assertions. No CLI
880+
ceilings were lowered. Both runs used isolated snapshots and CI's coverage,
881+
Git configuration and pytest options. They overlapped, so the elapsed times
882+
are not a controlled speed comparison. Logs, JUnit results and operation
883+
reports are retained locally in `.cache/gix-windows/`.
884+
885+
The CLI repository/discovery regressions passed with 138 tests, 14 subtests,
886+
nine skips and one expected failure. The changed native path, discovery,
887+
reference and mutation-error cases also passed under Ubuntu WSL: 43 tests
888+
and 14 subtests, using CPython 3.14.4, Git 2.53.0 and a separate released-source
889+
GixPython build. This includes the POSIX newline-filename cases.
890+
830891
## Published release validation
831892

832893
The official Apple Silicon ABI3 wheel for GixPython 0.1.0 has SHA-256

‎git/_backend.py‎

Lines changed: 14 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -101,6 +101,13 @@ def discover_repository(path: str, environment: Dict[str, Any]) -> Any:
101101
# Retain Git's rejection/diagnostics for layouts Gix cannot open yet.
102102
_logger.debug("native discovery: %s", exc)
103103
return _fallback("Repo.open", "native discovery diagnostics")
104+
except RuntimeError as exc:
105+
if str(exc) != "native worker panicked":
106+
raise
107+
# Undecodable Windows commondir paths panic in GixPython 0.1.0.
108+
# Discovery is read-only; keep Git's validation and error contract.
109+
_logger.debug("native discovery: %s", exc)
110+
return _fallback("Repo.open", "native discovery worker panic (GIX-14)")
104111

105112

106113
def _repository(command: Any, env: Dict[str, Any], *, query_config: bool = False) -> Any:
@@ -261,9 +268,9 @@ def revision_info(command: Any, ref: str) -> Any:
261268

262269

263270
def _rev_parse(repo: Any, args: List[str], kwargs: Dict[str, Any]) -> bytes:
264-
if args[:2] == ["--path-format=absolute", "--git-path"]:
265-
from git.util import to_native_path_linux
271+
from git.util import to_native_path_linux
266272

273+
if args[:2] == ["--path-format=absolute", "--git-path"]:
267274
if len(args) != 3 or args[2] not in ("modules", "COMMIT_EDITMSG"):
268275
raise _Unsupported("Git metadata path resolution (GIX-22)")
269276
# Git keeps these fixed leaves in the private Git directory, including
@@ -273,7 +280,7 @@ def _rev_parse(repo: Any, args: List[str], kwargs: Dict[str, Any]) -> bytes:
273280
raise _Unsupported("symlinked metadata path (GIX-22)")
274281
return os.fsencode(to_native_path_linux(path)) + b"\n"
275282
if args == ["--path-format=absolute", "--git-common-dir"]:
276-
return os.fsencode(os.path.abspath(repo.common_dir())) + b"\n"
283+
return os.fsencode(to_native_path_linux(os.path.abspath(repo.common_dir()))) + b"\n"
277284
if args == ["--is-bare-repository"]:
278285
return b"true\n" if repo.is_bare() and repo.workdir() is None else b"false\n"
279286
if args == ["--show-object-format"]:
@@ -283,7 +290,7 @@ def _rev_parse(repo: Any, args: List[str], kwargs: Dict[str, Any]) -> bytes:
283290
# neither validates unknown repository extensions as Git's query does.
284291
raise _Unsupported("reference storage format query (GIX-1)")
285292
if args == ["--show-toplevel"] and repo.workdir() is not None:
286-
return os.fsencode(os.path.abspath(repo.workdir())) + b"\n"
293+
return os.fsencode(to_native_path_linux(os.path.abspath(repo.workdir()))) + b"\n"
287294
if args[:1] != ["--verify"] or args[-2:-1] != ["--end-of-options"]:
288295
raise _Unsupported("discovery or revision options")
289296
if args[:-2] not in (["--verify"], ["--verify", "--quiet"]):
@@ -437,6 +444,8 @@ def _config(repo: Any, args: List[str], kwargs: Dict[str, Any]) -> bytes:
437444

438445

439446
def _worktree(repo: Any, args: List[str], kwargs: Dict[str, Any]) -> bytes:
447+
from git.util import to_native_path_linux
448+
440449
if args != ["list", "--porcelain", "-z"] or not hasattr(repo, "worktrees"):
441450
raise _Unsupported("worktree operation or feature disabled")
442451
proxies = repo.worktrees()
@@ -446,7 +455,7 @@ def _worktree(repo: Any, args: List[str], kwargs: Dict[str, Any]) -> bytes:
446455

447456
def describe(local: Any) -> List[bytes]:
448457
workdir = local.workdir()
449-
fields = [b"worktree " + os.fsencode(workdir or local.git_dir())]
458+
fields = [b"worktree " + os.fsencode(to_native_path_linux(os.fspath(workdir or local.git_dir())))]
450459
# Gix's is_bare() reflects configuration, even for a linked worktree.
451460
if local.is_bare() and workdir is None:
452461
fields.append(b"bare")

‎git/repo/base.py‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -387,9 +387,12 @@ def __init__(
387387
except GitCommandError as exc:
388388
raise InvalidGitRepositoryError(epath) from exc
389389
if native is not NotImplemented:
390-
self._common_dir = osp.abspath(native.common_dir())
390+
self._common_dir = to_native_path_linux(osp.abspath(native.common_dir()))
391391
self.object_format = str(native.object_hash())
392-
self._working_tree_dir = environment.get("GIT_WORK_TREE") or native.workdir()
392+
workdir = native.workdir()
393+
self._working_tree_dir = environment.get("GIT_WORK_TREE") or (
394+
to_native_path_linux(os.fspath(workdir)) if workdir is not None else None
395+
)
393396
# Gix's configured bare flag also applies to linked worktrees.
394397
self._bare = native.is_bare() and self._working_tree_dir is None
395398
else:

0 commit comments

Comments
 (0)