Skip to content

Commit 51f5c88

Browse files
codexByron
authored andcommitted
docs(perf): record measured CLI and GixPython repository journeys
Document the new `pyperf` workload and local measurements independently of full-suite setup/coverage timings. Record all eleven means and standard deviations, fixture/source revisions and environment details: direct opening and nested discovery favor GixPython, but the already-open journey is about 13% slower because history metadata and commit statistics offset other gains. Explain retained `Repo` ownership during operation measurements, separate open/discovery lifecycle costs, fresh high-level wrappers, parity checks and the extensible CI artifact workflow. Native handle reuse remains future work; these measurements expose its potential benefit without implementing it. Both complete calibrated runs finished on existing CPython 3.12.14 with GixPython first and CLI second. All eleven result digests match; the comparison and `pyperf` significance table completed successfully. Record the stability warnings rather than treating one machine's warm-cache values as universal. The initial dedicated CI benchmark job also passed. `git diff --check` passed.
1 parent 6d3fec3 commit 51f5c88

1 file changed

Lines changed: 48 additions & 0 deletions

File tree

‎doc/gix-backend.md‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -118,6 +118,54 @@ only an absent top-level `gix` selects CLI mode.
118118

119119
## Performance work
120120

121+
### Existing-repository benchmark
122+
123+
[`test/performance/README.md`](../test/performance/README.md) describes the
124+
`pyperf` harness and fixed GitPython 3.1.45 fixture. It measures eight public-API
125+
operations and their complete journey on one already-open `git.Repo`, plus
126+
separate direct opening and discovery from `git/objects`. Fresh high-level
127+
wrappers preserve the cost of actual operations; imports, fixture preparation
128+
and parity preflight are outside timing. Native repository reopening remains
129+
inside operation timing, so retaining native state in the future can improve
130+
these measurements without changing the workload.
131+
132+
At `244e418da6cc43de129cbe2908d11be4c5ad457a`, using official GixPython
133+
0.1.0 and the same existing CPython 3.12.14/macOS arm64 interpreter for both
134+
installations, with Git 2.54.0 (Apple Git-157), six worker processes with three values each produced the
135+
following means and standard deviations. GixPython ran first, then CLI;
136+
all eleven result digests matched. The fixture has one branch, one untracked
137+
file and an ignored directory, with ambient Git configuration disabled.
138+
139+
| Measurement | CLI (ms) | GixPython (ms) | CLI / Gix |
140+
| --- | ---: | ---: | ---: |
141+
| Already-open journey | 230.90 ± 6.87 | 260.14 ± 13.37 | 0.89× |
142+
| Reference inventory | 22.77 ± 1.15 | 1.17 ± 0.26 | 19.52× |
143+
| 25 first-parent commits with metadata | 9.24 ± 0.46 | 73.61 ± 7.24 | 0.13× |
144+
| Root tree and README blob | 16.02 ± 1.07 | 2.40 ± 0.46 | 6.66× |
145+
| Index entries | 8.43 ± 0.57 | 2.00 ± 0.38 | 4.21× |
146+
| Commit count and ancestry queries | 82.28 ± 2.27 | 36.19 ± 0.84 | 2.27× |
147+
| Latest diff and commit statistics | 29.98 ± 1.66 | 60.98 ± 2.45 | 0.49× |
148+
| Patch diff (includes CLI fallback) | 17.87 ± 1.28 | 10.86 ± 1.01 | 1.64× |
149+
| Dirty/untracked/ignored paths | 68.56 ± 3.18 | 64.74 ± 2.00 | 1.06× |
150+
| Direct opening and close | 95.08 ± 4.64 | 35.84 ± 2.02 | 2.65× |
151+
| Nested discovery and close | 112.58 ± 5.38 | 49.20 ± 4.16 | 2.29× |
152+
153+
Ratios above one favor GixPython. This workload's complete journey is about
154+
13% slower with GixPython: metadata reads and commit statistics offset gains
155+
elsewhere. These are warm-cache measurements on one machine, with `pyperf`
156+
stability warnings for several samples; they do not establish a general
157+
speedup. Full-suite times below also include setup and coverage.
158+
159+
The separate `Backend benchmark` CI job runs both installations on one runner,
160+
publishes every measurement and `pyperf` significance reporting, and retains
161+
raw JSON and comparison artifacts. Results include revisions and per-operation
162+
native/fallback decisions. Adding a `MEASUREMENTS` entry extends the journey,
163+
individual timings and parity checks together. Opening/discovery remain
164+
separate lifecycle measurements. CI fails on execution/parity errors; timing
165+
ratios are observational on shared runners.
166+
167+
### Test-suite setup measurements
168+
121169
Native repository reuse remains future work. The fixture optimizations below
122170
are implemented as separate commits, each validated with GixPython before CLI.
123171
The original local measurements use official GixPython 0.1.0 and existing

0 commit comments

Comments
 (0)