Repository navigation
fix(observe): retain failed-call prices without token metadata - #143
Conversation
Shashankss1205
left a comment
There was a problem hiding this comment.
Evaluation of 7e820f4: known USD is now attributed independently of optional token counts on failed executions. Explicit zero remains zero; no price remains missing. Subcall events are still a breakdown of the enclosing terminal, so the change does not add their prices twice.
Three regression cases fail before the guard correction; all seven new checks pass afterward. Focused cost/replay/tenant/documentation checks: 169 passed, one skipped only for the explicit figure refresh. The final integrated default suite passes: 2,395 passed, 13 skipped and 13 paid tests deselected. Ruff and diff checks pass. The final wheel builds and all 133 modules import from a clean installed environment with all extras.
No blocking code findings. Merge after #140 and #141; GitHub runner checks are still pending.
|
Final evaluation for unchanged head 7e820f4: no blocking review findings, unresolved threads or requested changes. #140 and #141 are merged, so the remaining diff is the cost-reader guard, its regressions and documentation. GitHub CI has passed the full Python 3.13 and 3.14 jobs and distribution checks. Python 3.12, lint and the live-test guard remain queued for runners. Their equivalent local checks passed: the complete final suite has 2,395 passed, 13 skipped and 13 paid tests deselected on both Python 3.12 and 3.13; ruff and diff checks are clean; explicit selection verification keeps all 13 paid tests out of the default suite. The final installed wheel imports all 133 modules. Proceeding through the normal GitHub merge endpoint with the reviewed head SHA, without an administrative override. This decision relies on the completed local verification for the queued jobs; their remote results are still pending. |
A failed execution with a provider-reported USD price but zero or absent token usage was omitted from cost attribution, even though replay retained the recorded price. Read known dollars independently of token availability, including an explicit zero price; leave missing prices missing.
Add seven checks for success/error terminals, zero/positive/missing prices, subcall deduplication, and a real scripted priced node that fails without token metadata. Assert run/thread/tenant totals and replay agree. Three cases fail before the fix; all seven pass afterward. Update the pricing documentation and selected-test figure.
Validation: focused cost/replay/tenant/documentation checks passed 169 tests, with one skipped only during the explicit figure refresh. The final integrated suite passed 2,395 tests on both Python 3.12 and 3.13 locally, with 13 skipped and 13 paid tests excluded. Ruff/diff checks pass; the wheel builds and all 133 modules import from a clean environment with all extras. GitHub CI results are recorded in the review before merge. No paid model calls.
Fixes #142.