Skip to content

fix(observe): retain failed-call prices without token metadata - #143

Merged
Shashankss1205 merged 3 commits into
mainfrom
fix/cost-without-token-usage
Oct 15, 2026
Merged

Shashankss1205 merged 3 commits into
mainfrom
fix/cost-without-token-usage

Conversation

@Shashankss1205

@Shashankss1205 Shashankss1205 commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator

A failed execution with a provider-reported USD price but zero or absent token usage was omitted from cost attribution, even though replay retained the recorded price. Read known dollars independently of token availability, including an explicit zero price; leave missing prices missing.

Add seven checks for success/error terminals, zero/positive/missing prices, subcall deduplication, and a real scripted priced node that fails without token metadata. Assert run/thread/tenant totals and replay agree. Three cases fail before the fix; all seven pass afterward. Update the pricing documentation and selected-test figure.

Validation: focused cost/replay/tenant/documentation checks passed 169 tests, with one skipped only during the explicit figure refresh. The final integrated suite passed 2,395 tests on both Python 3.12 and 3.13 locally, with 13 skipped and 13 paid tests excluded. Ruff/diff checks pass; the wheel builds and all 133 modules import from a clean environment with all extras. GitHub CI results are recorded in the review before merge. No paid model calls.

Fixes #142.

@Shashankss1205 Shashankss1205 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Evaluation of 7e820f4: known USD is now attributed independently of optional token counts on failed executions. Explicit zero remains zero; no price remains missing. Subcall events are still a breakdown of the enclosing terminal, so the change does not add their prices twice.

Three regression cases fail before the guard correction; all seven new checks pass afterward. Focused cost/replay/tenant/documentation checks: 169 passed, one skipped only for the explicit figure refresh. The final integrated default suite passes: 2,395 passed, 13 skipped and 13 paid tests deselected. Ruff and diff checks pass. The final wheel builds and all 133 modules import from a clean installed environment with all extras.

No blocking code findings. Merge after #140 and #141; GitHub runner checks are still pending.

Copy link
Copy Markdown
Collaborator Author

Final evaluation for unchanged head 7e820f4: no blocking review findings, unresolved threads or requested changes. #140 and #141 are merged, so the remaining diff is the cost-reader guard, its regressions and documentation.

GitHub CI has passed the full Python 3.13 and 3.14 jobs and distribution checks. Python 3.12, lint and the live-test guard remain queued for runners. Their equivalent local checks passed: the complete final suite has 2,395 passed, 13 skipped and 13 paid tests deselected on both Python 3.12 and 3.13; ruff and diff checks are clean; explicit selection verification keeps all 13 paid tests out of the default suite. The final installed wheel imports all 133 modules.

Proceeding through the normal GitHub merge endpoint with the reviewed head SHA, without an administrative override. This decision relies on the completed local verification for the queued jobs; their remote results are still pending.

@Shashankss1205
Shashankss1205 merged commit e27b9e8 into main Oct 15, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

observe: failed executions lose recorded USD when token usage is absent

1 participant