Skip to content

Commit f437d84

Browse files
codexByron
authored andcommitted
perf(gix): open repositories with supported native metadata
Use `gix.open_opts()` and native accessors for storage, worktree, object-format and empty-tree metadata. Retain the native handle and leave parent traversal under `search_parent_directories`. Every candidate reaches Gix before a discovery decision; do not fabricate rejection from Python `HEAD` checks. Gix already supplies worktree metadata for linked worktrees of bare main repositories. Its `is_bare()` reflects inherited configuration, so combine it with the native worktree's presence for GitPython's bare flag and managed `--is-bare-repository`. Remove the unnecessary linked-bare fallback. Cover opening through the worktree, its gitfile, and its administrative directory. Keep the real format query for Git-compatible repository-extension validation (`GIX-1`), not because storage cannot be identified. Force native HEAD decoding and retain common-directory/layout guards and Git diagnostics (`GIX-14`). In particular, strict native opening still ignores dangling `commondir` symlinks. Each linked-worktree bare query now drops from one CLI call to zero. Opening those worktrees uses the normal native path: only the one format-validation query and executable-version preflight remain. Nested invalid candidates still need Git diagnostics because GixPython flattens the native error kinds. Validation: affected opening/discovery/worktree/format/lifetime tests passed with Gix first (25 passed, 1 skipped, 14 subtests in 4.86s), then CLI (12 passed, 2 skipped, 14 subtests in 3.20s). Ruff lint/format and `git diff --check` passed. Replay preserves all opening and safety changes.
1 parent 9d283af commit f437d84

11 files changed

Lines changed: 236 additions & 101 deletions

File tree

‎doc/gix-backend.md‎

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -107,7 +107,7 @@ the call phase, so that column can still include their fixture work. Counts
107107
cover the current Python process; they are not aggregated across xdist workers.
108108
For a stable CI assertion, the pinned repository benchmark uses per-measurement
109109
ceilings in `test/performance/cli-budget.json`: the warm Gix journey launches
110-
one CLI process, opening launches five, and nested discovery launches seven.
110+
one CLI process, opening launches two, and nested discovery launches four.
111111
Reduce these ceilings as conversions land. The other operation rows have zero
112112
ceilings except the one-process patch diff.
113113

@@ -121,16 +121,17 @@ accounting. The same run recorded 21,409 fallback decisions. These totals
121121
include explicit CLI tests and fixture commands, including non-Git commands
122122
passed directly to `Git.execute`; they are a local baseline, not a count of
123123
only fallback calls. The fixed warm benchmark instead isolates a user journey:
124-
23 CLI launches versus one Gix launch, opening 11 versus five, and nested
125-
discovery 13 versus seven after restoring the reference-format fallback.
124+
23 CLI launches versus one Gix launch, opening 11 versus two, and nested
125+
discovery 13 versus four after restoring the reference-format fallback.
126126
The historical full-suite counts above predate this correction. A constant
127127
`files` answer was not a Gix query; `GIX-1` records the missing binding.
128128

129129
| GitPython entry point / managed command | Native implementation | Remaining CLI cases |
130130
| --- | --- | --- |
131131
| `Git.get_object_header`, ODB `info` | Object resolution and native header lookup | Custom storage, unsupported revision grammar |
132132
| `Git.stream_object_data`, ODB `stream` | Object bytes in an independent stream | Objects larger than 8 MiB; GIX-2 |
133-
| `Repo.rev_parse`, `rev_parse` metadata queries | Revision lookup, common directory, object format, bare flag, worktree root | Reference-storage format query; initial repository validation; message searches and describe forms; GIX-1/14/17 |
133+
| `Repo` opening/discovery | Native storage, worktree, object format and empty-tree metadata | Repository-format validation, unsupported layouts/environment and native validation gaps; GIX-1/14/19 |
134+
| `Repo.rev_parse`, `rev_parse` metadata queries | Revision lookup, common directory, object format, bare flag, worktree root | Repository-format validation, message searches and describe forms; GIX-1/14/17 |
134135
| Tree enumeration, `ls_tree` | Native tree entries with modes, names, and object IDs | Other command options |
135136
| Symbolic reference lookup, `symbolic_ref` | Nonrecursive target lookup | Mutation, missing-reference diagnostics |
136137
| Reference enumeration, `for_each_ref` | Sorted reference names and literal prefixes, preserving symbolic aliases | Dangling symbolic refs, root refs, glob patterns, other formats/options |
@@ -172,8 +173,9 @@ only an absent top-level `gix` selects CLI mode.
172173
operations and their complete journey on one already-open `git.Repo`, plus
173174
separate direct opening and discovery from `git/objects`. Fresh high-level
174175
wrappers preserve the cost of actual operations; imports, fixture preparation
175-
and parity preflight are outside timing. Native repository refresh remains inside operation timing, so these
176-
measurements include the cost of keeping retained state current.
176+
and parity preflight are outside timing. Native repository refresh remains
177+
inside operation timing, so these measurements include the cost of keeping
178+
retained state current.
177179

178180
At `244e418da6cc43de129cbe2908d11be4c5ad457a`, using official GixPython
179181
0.1.0 and the same existing CPython 3.12.14/macOS arm64 interpreter for both
@@ -343,7 +345,7 @@ below use the supplied `pygix` source and its pinned Gitoxide revision.
343345
| GIX-11 | Inexact and ambiguous rename pairing/scores have not been established as Git-compatible. This is a conservative guard, not a claimed native bug. | Verified pairing, scoring, tie-breaking and option parity. Exact unambiguous renames are native. |
344346
| GIX-12 | Config bindings lack standalone parsing, ordered section/key enumeration, multivars, unset/remove-section and source-scoped queries. | Those operations for `GitConfigParser` and remote/branch configuration. Native merged getters cannot replace repository-only config readers. |
345347
| GIX-13 | Native index writing normalizes v4 to v2, offers no version setter, and expands split indexes. | Version and split-index preservation. Tests now assert that a split index remains split after an update. |
346-
| GIX-14 | Gix correctly exposes linked worktree paths, including those of bare main repositories. Its documented `is_bare()` reflects configuration and can be true alongside a worktree; inventory now uses both pieces of native metadata. Actual gaps remain: discovery tolerates undecodable HEADs and ignores dangling `commondir` symlinks, while Python flattens native failure kinds into `gix.Error`. | Explicit native HEAD decoding and CLI fallback for remaining layout/diagnostic gaps. Strict trust checks do not replace layout validation. Canonical paths can be supplied when reopening through Gix, as already done for the inventory's main repository; no classification fix is required upstream. Initial `--resolve-git-dir` stays with Git at this stage. |
348+
| GIX-14 | Gix correctly exposes linked worktree paths, including those of bare main repositories. Its documented `is_bare()` reflects configuration and can be true alongside a worktree; opening and inventory now use both pieces of native metadata. Actual gaps remain: discovery tolerates undecodable HEADs and ignores dangling `commondir` symlinks, while Python flattens native failure kinds into `gix.Error`. | Opening forces native HEAD decoding and retains CLI validation for remaining layout/diagnostic gaps. Strict trust checks do not replace layout validation. Canonical inputs can be reopened through Gix, as already done for the inventory's main repository; no classification fix is required upstream. Do not fabricate rejection from Python `HEAD` existence checks or infer a missing worktree root. |
347349
| GIX-15 | Native blame disagrees with Git even with Myers and rewrite tracking selected. In the fixture's `README.md`, lines 150 and 158 are attributed to the opposite commits. Incremental order also differs. | Attribution and incremental-output parity before replacing `Repo.blame` / `blame_incremental`. |
348350
| GIX-16 | Native archive streaming takes a tree rather than a commit. Its TAR omits Git's global PAX commit comment, leaves `export-subst` placeholders literal, and writes ordinary modes as `0644` where Git uses `0664`. | Commit-aware export substitution, metadata and permission parity. Both engines respected `export-ignore` in the probe. |
349351
| GIX-17 | Revision parsing accepts abbreviated IDs with `-dirty`, prefers the OID suffix over an exact describe-shaped tag, and treats escaped braces in message searches differently. | Git-compatible revision grammar and regex semantics. Existing `test_rev_parse.py` cases reproduce all three. |

‎git/_backend.py‎

Lines changed: 43 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,47 @@ def _fallback(method: str, reason: str) -> Any:
5858
return NotImplemented
5959

6060

61+
def discover_repository(path: str, environment: Dict[str, Any]) -> Any:
62+
"""Open one discovery candidate; the caller controls parent traversal."""
63+
if gix is None:
64+
return NotImplemented
65+
effective = {**os.environ, **environment}
66+
if any(
67+
effective.get(key)
68+
for key in ("GIT_COMMON_DIR", "GIT_OBJECT_DIRECTORY", "GIT_ALTERNATE_OBJECT_DIRECTORIES", "GIT_NAMESPACE")
69+
) or any(value != os.environ.get(key) for key, value in environment.items() if key != "GIT_WORK_TREE"):
70+
return _fallback("Repo.open", "storage or command environment")
71+
options = gix.OpenOptions().open_path_as_is(True).bail_if_untrusted(True).strict_config(True)
72+
options = options.config_overrides(
73+
["core.fsmonitor=false", "gc.auto=0", "maintenance.auto=false", "core.hooksPath=" + os.devnull]
74+
)
75+
try:
76+
repo = gix.open_opts(path, options)
77+
snapshot = repo.config_snapshot()
78+
if snapshot.string("extensions.refStorage") == b"reftable":
79+
return _fallback("Repo.open", "reftable (GIX-1)")
80+
if snapshot.string("extensions.compatObjectFormat") is not None:
81+
return _fallback("Repo.open", "compatibility object format (GIX-19)")
82+
# Discovery accepts undecodable HEADs; force the native reference decoder.
83+
repo.head()
84+
if not repo.is_bare() and repo.workdir() is None and not effective.get("GIT_WORK_TREE"):
85+
return _fallback("Repo.open", "missing native worktree metadata (GIX-14)")
86+
commondir = os.path.join(repo.git_dir(), "commondir")
87+
if os.path.lexists(commondir):
88+
# Gix ignores invalid commondir files on common repositories. A common
89+
# directory must have been resolved and contain actual shared storage.
90+
if os.path.realpath(repo.common_dir()) == os.path.realpath(repo.git_dir()) or not all(
91+
os.path.isdir(os.path.join(repo.common_dir(), entry)) for entry in ("objects", "refs")
92+
):
93+
return _fallback("Repo.open", "common-directory validation (GIX-14)")
94+
record("Repo.open", "native")
95+
return repo
96+
except gix.Error as exc:
97+
# Retain Git's rejection/diagnostics for layouts Gix cannot open yet.
98+
_logger.debug("native discovery: %s", exc)
99+
return _fallback("Repo.open", "native discovery diagnostics")
100+
101+
61102
def _repository(command: Any, env: Dict[str, Any], *, query_config: bool = False) -> Any:
62103
owner = command._repo() if command._repo is not None else None
63104
if owner is not None:
@@ -118,7 +159,7 @@ def _open_repository(command: Any, env: Dict[str, Any], *, query_config: bool =
118159
stamps.append(None)
119160
state = (path, tuple(sorted(effective.items())), tuple(stamps))
120161
repo = owner._gix_repository
121-
if repo is not None and os.path.abspath(repo.git_dir()) != os.path.abspath(path):
162+
if repo is not None and os.path.realpath(repo.git_dir()) != os.path.realpath(path):
122163
repo = None
123164
if repo is not None and state != owner._gix_state:
124165
repo.reload()
@@ -166,7 +207,6 @@ def object_data(command: Any, ref: bytes, *, stream: bool = False) -> Any:
166207
if gix is None:
167208
return NotImplemented
168209
method = "stream_object_data" if stream else "get_object_header"
169-
command._require_version()
170210
try:
171211
repo = _repository(command, {})
172212
oid = _oid(repo, ref)
@@ -192,7 +232,7 @@ def _rev_parse(repo: Any, args: List[str], kwargs: Dict[str, Any]) -> bytes:
192232
if args == ["--path-format=absolute", "--git-common-dir"]:
193233
return os.fsencode(os.path.abspath(repo.common_dir())) + b"\n"
194234
if args == ["--is-bare-repository"]:
195-
return b"true\n" if repo.is_bare() else b"false\n"
235+
return b"true\n" if repo.is_bare() and repo.workdir() is None else b"false\n"
196236
if args == ["--show-object-format"]:
197237
return str(repo.object_hash()).encode("ascii") + b"\n"
198238
if args == ["--show-ref-format"]:
@@ -319,7 +359,6 @@ def is_dirty(command: Any, index: bool, working_tree: bool, untracked: bool, sub
319359
if gix is None:
320360
return NotImplemented
321361
method = "Repo.is_dirty"
322-
command._require_version()
323362
try:
324363
repo = _repository(command, {})
325364
_worktree_root(command, repo)
@@ -365,7 +404,6 @@ def untracked_files(command: Any, args: Tuple[Any, ...], options: Dict[str, Any]
365404
if gix is None:
366405
return NotImplemented
367406
method = "Repo.untracked_files"
368-
command._require_version()
369407
try:
370408
if options.keys() - {"ignore_submodules"}:
371409
raise _Unsupported("status options")
@@ -401,7 +439,6 @@ def ignored(command: Any, paths: Sequence[Any]) -> Any:
401439
if gix is None:
402440
return NotImplemented
403441
method = "Repo.ignored"
404-
command._require_version()
405442
try:
406443
repo = _repository(command, {})
407444
root = _worktree_root(command, repo)
@@ -495,7 +532,6 @@ def history(command: Any, rev: str, paths: Any, options: Dict[str, Any], *, coun
495532
if gix is None:
496533
return NotImplemented
497534
method = "Commit.count" if count else "Commit.iter_items"
498-
command._require_version()
499535
try:
500536
if paths:
501537
raise _Unsupported("history path filtering")
@@ -624,7 +660,6 @@ def materialize_index(command: Any, source: Any, destination: str, desired: Dict
624660
if gix is None:
625661
return NotImplemented
626662
method = "IndexFile.write"
627-
command._require_version()
628663
try:
629664
if any(stage for _path, stage in desired):
630665
raise _Unsupported("unmerged index editing")
@@ -761,7 +796,6 @@ def _write_tree(repo: Any, entries: Sequence[Tuple[bytes, int, str]]) -> str:
761796
def write_tree(command: Any, entries: Sequence[Tuple[bytes, int, str]]) -> Any:
762797
if gix is None:
763798
return NotImplemented
764-
command._require_version()
765799
try:
766800
result = _write_tree(_repository(command, {}), entries)
767801
except _Unsupported as exc:
@@ -780,7 +814,6 @@ def tree_diff(repository: Any, left: Any, right: Any, paths: Any, patch: bool, o
780814
from git.diff import Diff, DiffIndex, Lit_change_type
781815

782816
method = "Diffable.diff"
783-
repository.git._require_version()
784817
try:
785818
if not hasattr(left, "hexsha") or not (hasattr(right, "hexsha") or isinstance(right, str)):
786819
raise _Unsupported("index/worktree/root diff")
@@ -867,7 +900,6 @@ def commit_stats(commit: Any) -> Any:
867900
from git.util import Stats
868901

869902
method = "Commit.stats"
870-
commit.repo.git._require_version()
871903
try:
872904
repo = _repository(commit.repo.git, {})
873905
if not hasattr(repo, "diff_tree_to_tree") or not hasattr(repo, "attributes_only"):

‎git/cmd.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1185,7 +1185,6 @@ def _call_process_safe(
11851185
):
11861186
if "\0" in arg:
11871187
raise UnsafeOptionError("Git arguments cannot contain NUL bytes")
1188-
self._require_version()
11891188
options = ["--no-pager", "--no-optional-locks"]
11901189
# These settings would become visible as user configuration in `config`.
11911190
if method != "config":
@@ -1199,6 +1198,7 @@ def _call_process_safe(
11991198
native = _backend.dispatch(self, method, args, kwargs, _config)
12001199
if native is not NotImplemented:
12011200
return native
1201+
self._require_version()
12021202
env = dict(kwargs.pop("env", {}) or {})
12031203
env.update(LC_ALL="C", LANGUAGE="C")
12041204
if not _allow_network:

0 commit comments

Comments
 (0)