Skip to content

Commit efee6e4

Browse files
codexByron
authored andcommitted
perf(gix): open repositories with supported native metadata
Use `gix.open_opts()` and native accessors for storage, worktree, object-format and empty-tree metadata. Retain the native handle and leave parent traversal under `search_parent_directories`. Every candidate reaches Gix before a discovery decision; do not fabricate rejection from Python `HEAD` checks. Gix already supplies worktree metadata for linked worktrees of bare main repositories. Its `is_bare()` reflects inherited configuration, so combine it with the native worktree's presence for GitPython's bare flag and managed `--is-bare-repository`. Remove the unnecessary linked-bare fallback. Cover opening through the worktree, its gitfile, and its administrative directory. Keep the real format query for Git-compatible repository-extension validation (`GIX-1`), not because storage cannot be identified. Force native HEAD decoding and retain common-directory/layout guards and Git diagnostics (`GIX-14`). In particular, strict native opening still ignores dangling `commondir` symlinks. Each linked-worktree bare query now drops from one CLI call to zero. Opening those worktrees uses the normal native path: only the one format-validation query and executable-version preflight remain. Nested invalid candidates still need Git diagnostics because GixPython flattens the native error kinds. Validation: affected opening/discovery/worktree/format/lifetime tests passed with Gix first (25 passed, 1 skipped, 14 subtests in 4.86s), then CLI (12 passed, 2 skipped, 14 subtests in 3.20s). Ruff lint/format and `git diff --check` passed. Replay preserves all opening and safety changes.
1 parent 887a0cf commit efee6e4

11 files changed

Lines changed: 236 additions & 101 deletions

File tree

‎doc/gix-backend.md‎

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -105,7 +105,7 @@ the call phase, so that column can still include their fixture work. Counts
105105
cover the current Python process; they are not aggregated across xdist workers.
106106
For a stable CI assertion, the pinned repository benchmark uses per-measurement
107107
ceilings in `test/performance/cli-budget.json`: the warm Gix journey launches
108-
one CLI process, opening launches five, and nested discovery launches seven.
108+
one CLI process, opening launches two, and nested discovery launches four.
109109
Reduce these ceilings as conversions land. The other operation rows have zero
110110
ceilings except the one-process patch diff.
111111

@@ -119,16 +119,17 @@ accounting. The same run recorded 21,409 fallback decisions. These totals
119119
include explicit CLI tests and fixture commands, including non-Git commands
120120
passed directly to `Git.execute`; they are a local baseline, not a count of
121121
only fallback calls. The fixed warm benchmark instead isolates a user journey:
122-
23 CLI launches versus one Gix launch, opening 11 versus five, and nested
123-
discovery 13 versus seven after restoring the reference-format fallback.
122+
23 CLI launches versus one Gix launch, opening 11 versus two, and nested
123+
discovery 13 versus four after restoring the reference-format fallback.
124124
The historical full-suite counts above predate this correction. A constant
125125
`files` answer was not a Gix query; `GIX-1` records the missing binding.
126126

127127
| GitPython entry point / managed command | Native implementation | Remaining CLI cases |
128128
| --- | --- | --- |
129129
| `Git.get_object_header`, ODB `info` | Object resolution and native header lookup | Custom storage, unsupported revision grammar |
130130
| `Git.stream_object_data`, ODB `stream` | Object bytes in an independent stream | Objects larger than 8 MiB; GIX-2 |
131-
| `Repo.rev_parse`, `rev_parse` metadata queries | Revision lookup, common directory, object format, bare flag, worktree root | Reference-storage format query; initial repository validation; message searches and describe forms; GIX-1/14/17 |
131+
| `Repo` opening/discovery | Native storage, worktree, object format and empty-tree metadata | Repository-format validation, unsupported layouts/environment and native validation gaps; GIX-1/14/19 |
132+
| `Repo.rev_parse`, `rev_parse` metadata queries | Revision lookup, common directory, object format, bare flag, worktree root | Repository-format validation, message searches and describe forms; GIX-1/14/17 |
132133
| Tree enumeration, `ls_tree` | Native tree entries with modes, names, and object IDs | Other command options |
133134
| Symbolic reference lookup, `symbolic_ref` | Nonrecursive target lookup | Mutation, missing-reference diagnostics |
134135
| Reference enumeration, `for_each_ref` | Sorted reference names and literal prefixes, preserving symbolic aliases | Dangling symbolic refs, root refs, glob patterns, other formats/options |
@@ -169,8 +170,9 @@ an import error; only an absent top-level `gix` selects CLI mode.
169170
operations and their complete journey on one already-open `git.Repo`, plus
170171
separate direct opening and discovery from `git/objects`. Fresh high-level
171172
wrappers preserve the cost of actual operations; imports, fixture preparation
172-
and parity preflight are outside timing. Native repository refresh remains inside operation timing, so these
173-
measurements include the cost of keeping retained state current.
173+
and parity preflight are outside timing. Native repository refresh remains
174+
inside operation timing, so these measurements include the cost of keeping
175+
retained state current.
174176

175177
At `244e418da6cc43de129cbe2908d11be4c5ad457a`, using official GixPython
176178
0.1.0 and the same existing CPython 3.12.14/macOS arm64 interpreter for both
@@ -340,7 +342,7 @@ below use the supplied `pygix` source and its pinned Gitoxide revision.
340342
| GIX-11 | Inexact and ambiguous rename pairing/scores have not been established as Git-compatible. This is a conservative guard, not a claimed native bug. | Verified pairing, scoring, tie-breaking and option parity. Exact unambiguous renames are native. |
341343
| GIX-12 | Config bindings lack standalone parsing, ordered section/key enumeration, multivars, unset/remove-section and source-scoped queries. | Those operations for `GitConfigParser` and remote/branch configuration. Native merged getters cannot replace repository-only config readers. |
342344
| GIX-13 | Native index writing normalizes v4 to v2, offers no version setter, and expands split indexes. | Version and split-index preservation. Tests now assert that a split index remains split after an update. |
343-
| GIX-14 | Gix correctly exposes linked worktree paths, including those of bare main repositories. Its documented `is_bare()` reflects configuration and can be true alongside a worktree; inventory now uses both pieces of native metadata. Actual gaps remain: discovery tolerates undecodable HEADs and ignores dangling `commondir` symlinks, while Python flattens native failure kinds into `gix.Error`. | Explicit native HEAD decoding and CLI fallback for remaining layout/diagnostic gaps. Strict trust checks do not replace layout validation. Canonical paths can be supplied when reopening through Gix, as already done for the inventory's main repository; no classification fix is required upstream. Initial `--resolve-git-dir` stays with Git at this stage. |
345+
| GIX-14 | Gix correctly exposes linked worktree paths, including those of bare main repositories. Its documented `is_bare()` reflects configuration and can be true alongside a worktree; opening and inventory now use both pieces of native metadata. Actual gaps remain: discovery tolerates undecodable HEADs and ignores dangling `commondir` symlinks, while Python flattens native failure kinds into `gix.Error`. | Opening forces native HEAD decoding and retains CLI validation for remaining layout/diagnostic gaps. Strict trust checks do not replace layout validation. Canonical inputs can be reopened through Gix, as already done for the inventory's main repository; no classification fix is required upstream. Do not fabricate rejection from Python `HEAD` existence checks or infer a missing worktree root. |
344346
| GIX-15 | Native blame disagrees with Git even with Myers and rewrite tracking selected. In the fixture's `README.md`, lines 150 and 158 are attributed to the opposite commits. Incremental order also differs. | Attribution and incremental-output parity before replacing `Repo.blame` / `blame_incremental`. |
345347
| GIX-16 | Native archive streaming takes a tree rather than a commit. Its TAR omits Git's global PAX commit comment, leaves `export-subst` placeholders literal, and writes ordinary modes as `0644` where Git uses `0664`. | Commit-aware export substitution, metadata and permission parity. Both engines respected `export-ignore` in the probe. |
346348
| GIX-17 | Revision parsing accepts abbreviated IDs with `-dirty`, prefers the OID suffix over an exact describe-shaped tag, and treats escaped braces in message searches differently. | Git-compatible revision grammar and regex semantics. Existing `test_rev_parse.py` cases reproduce all three. |

‎git/_backend.py‎

Lines changed: 43 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,47 @@ def _fallback(method: str, reason: str) -> Any:
5454
return NotImplemented
5555

5656

57+
def discover_repository(path: str, environment: Dict[str, Any]) -> Any:
58+
"""Open one discovery candidate; the caller controls parent traversal."""
59+
if gix is None:
60+
return NotImplemented
61+
effective = {**os.environ, **environment}
62+
if any(
63+
effective.get(key)
64+
for key in ("GIT_COMMON_DIR", "GIT_OBJECT_DIRECTORY", "GIT_ALTERNATE_OBJECT_DIRECTORIES", "GIT_NAMESPACE")
65+
) or any(value != os.environ.get(key) for key, value in environment.items() if key != "GIT_WORK_TREE"):
66+
return _fallback("Repo.open", "storage or command environment")
67+
options = gix.OpenOptions().open_path_as_is(True).bail_if_untrusted(True).strict_config(True)
68+
options = options.config_overrides(
69+
["core.fsmonitor=false", "gc.auto=0", "maintenance.auto=false", "core.hooksPath=" + os.devnull]
70+
)
71+
try:
72+
repo = gix.open_opts(path, options)
73+
snapshot = repo.config_snapshot()
74+
if snapshot.string("extensions.refStorage") == b"reftable":
75+
return _fallback("Repo.open", "reftable (GIX-1)")
76+
if snapshot.string("extensions.compatObjectFormat") is not None:
77+
return _fallback("Repo.open", "compatibility object format (GIX-19)")
78+
# Discovery accepts undecodable HEADs; force the native reference decoder.
79+
repo.head()
80+
if not repo.is_bare() and repo.workdir() is None and not effective.get("GIT_WORK_TREE"):
81+
return _fallback("Repo.open", "missing native worktree metadata (GIX-14)")
82+
commondir = os.path.join(repo.git_dir(), "commondir")
83+
if os.path.lexists(commondir):
84+
# Gix ignores invalid commondir files on common repositories. A common
85+
# directory must have been resolved and contain actual shared storage.
86+
if os.path.realpath(repo.common_dir()) == os.path.realpath(repo.git_dir()) or not all(
87+
os.path.isdir(os.path.join(repo.common_dir(), entry)) for entry in ("objects", "refs")
88+
):
89+
return _fallback("Repo.open", "common-directory validation (GIX-14)")
90+
record("Repo.open", "native")
91+
return repo
92+
except gix.Error as exc:
93+
# Retain Git's rejection/diagnostics for layouts Gix cannot open yet.
94+
_logger.debug("native discovery: %s", exc)
95+
return _fallback("Repo.open", "native discovery diagnostics")
96+
97+
5798
def _repository(command: Any, env: Dict[str, Any], *, query_config: bool = False) -> Any:
5899
owner = command._repo() if command._repo is not None else None
59100
if owner is not None:
@@ -114,7 +155,7 @@ def _open_repository(command: Any, env: Dict[str, Any], *, query_config: bool =
114155
stamps.append(None)
115156
state = (path, tuple(sorted(effective.items())), tuple(stamps))
116157
repo = owner._gix_repository
117-
if repo is not None and os.path.abspath(repo.git_dir()) != os.path.abspath(path):
158+
if repo is not None and os.path.realpath(repo.git_dir()) != os.path.realpath(path):
118159
repo = None
119160
if repo is not None and state != owner._gix_state:
120161
repo.reload()
@@ -162,7 +203,6 @@ def object_data(command: Any, ref: bytes, *, stream: bool = False) -> Any:
162203
if gix is None:
163204
return NotImplemented
164205
method = "stream_object_data" if stream else "get_object_header"
165-
command._require_version()
166206
try:
167207
repo = _repository(command, {})
168208
oid = _oid(repo, ref)
@@ -188,7 +228,7 @@ def _rev_parse(repo: Any, args: List[str], kwargs: Dict[str, Any]) -> bytes:
188228
if args == ["--path-format=absolute", "--git-common-dir"]:
189229
return os.fsencode(os.path.abspath(repo.common_dir())) + b"\n"
190230
if args == ["--is-bare-repository"]:
191-
return b"true\n" if repo.is_bare() else b"false\n"
231+
return b"true\n" if repo.is_bare() and repo.workdir() is None else b"false\n"
192232
if args == ["--show-object-format"]:
193233
return str(repo.object_hash()).encode("ascii") + b"\n"
194234
if args == ["--show-ref-format"]:
@@ -315,7 +355,6 @@ def is_dirty(command: Any, index: bool, working_tree: bool, untracked: bool, sub
315355
if gix is None:
316356
return NotImplemented
317357
method = "Repo.is_dirty"
318-
command._require_version()
319358
try:
320359
repo = _repository(command, {})
321360
_worktree_root(command, repo)
@@ -361,7 +400,6 @@ def untracked_files(command: Any, args: Tuple[Any, ...], options: Dict[str, Any]
361400
if gix is None:
362401
return NotImplemented
363402
method = "Repo.untracked_files"
364-
command._require_version()
365403
try:
366404
if options.keys() - {"ignore_submodules"}:
367405
raise _Unsupported("status options")
@@ -397,7 +435,6 @@ def ignored(command: Any, paths: Sequence[Any]) -> Any:
397435
if gix is None:
398436
return NotImplemented
399437
method = "Repo.ignored"
400-
command._require_version()
401438
try:
402439
repo = _repository(command, {})
403440
root = _worktree_root(command, repo)
@@ -491,7 +528,6 @@ def history(command: Any, rev: str, paths: Any, options: Dict[str, Any], *, coun
491528
if gix is None:
492529
return NotImplemented
493530
method = "Commit.count" if count else "Commit.iter_items"
494-
command._require_version()
495531
try:
496532
if paths:
497533
raise _Unsupported("history path filtering")
@@ -620,7 +656,6 @@ def materialize_index(command: Any, source: Any, destination: str, desired: Dict
620656
if gix is None:
621657
return NotImplemented
622658
method = "IndexFile.write"
623-
command._require_version()
624659
try:
625660
if any(stage for _path, stage in desired):
626661
raise _Unsupported("unmerged index editing")
@@ -757,7 +792,6 @@ def _write_tree(repo: Any, entries: Sequence[Tuple[bytes, int, str]]) -> str:
757792
def write_tree(command: Any, entries: Sequence[Tuple[bytes, int, str]]) -> Any:
758793
if gix is None:
759794
return NotImplemented
760-
command._require_version()
761795
try:
762796
result = _write_tree(_repository(command, {}), entries)
763797
except _Unsupported as exc:
@@ -776,7 +810,6 @@ def tree_diff(repository: Any, left: Any, right: Any, paths: Any, patch: bool, o
776810
from git.diff import Diff, DiffIndex, Lit_change_type
777811

778812
method = "Diffable.diff"
779-
repository.git._require_version()
780813
try:
781814
if not hasattr(left, "hexsha") or not (hasattr(right, "hexsha") or isinstance(right, str)):
782815
raise _Unsupported("index/worktree/root diff")
@@ -863,7 +896,6 @@ def commit_stats(commit: Any) -> Any:
863896
from git.util import Stats
864897

865898
method = "Commit.stats"
866-
commit.repo.git._require_version()
867899
try:
868900
repo = _repository(commit.repo.git, {})
869901
if not hasattr(repo, "diff_tree_to_tree") or not hasattr(repo, "attributes_only"):

‎git/cmd.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1185,7 +1185,6 @@ def _call_process_safe(
11851185
):
11861186
if "\0" in arg:
11871187
raise UnsafeOptionError("Git arguments cannot contain NUL bytes")
1188-
self._require_version()
11891188
options = ["--no-pager", "--no-optional-locks"]
11901189
# These settings would become visible as user configuration in `config`.
11911190
if method != "config":
@@ -1199,6 +1198,7 @@ def _call_process_safe(
11991198
native = _backend.dispatch(self, method, args, kwargs, _config)
12001199
if native is not NotImplemented:
12011200
return native
1201+
self._require_version()
12021202
env = dict(kwargs.pop("env", {}) or {})
12031203
env.update(LC_ALL="C", LANGUAGE="C")
12041204
if not _allow_network:

0 commit comments

Comments
 (0)