Skip to content

observe: failed executions lose recorded USD when token usage is absent #142

Closed

Description

@Shashankss1205

Summary

After retaining provider prices on failed terminal events (#140), the cost reader still discards a failed execution’s real USD when its token total is zero or absent. A provider can supply a price without token usage; known dollars must not depend on a separate optional measurement.

Reproduction

Use a scripted model that returns llm_output={"token_usage": {"cost": 0.002}} but no message usage_metadata; invoke it in a node and then raise. The terminal trace event has cost_usd=0.002, tokens=0, while attribute(trace, run_id).recorded_cost_usd is None. replay(...).recorded_cost_usd retains the price, so the two readers disagree.

Cause

grapharc/observe/cost.py:_price_run handles a failed node’s price only inside if execution.tokens:. Successful node terminals already preserve prices without requiring tokens.

Fix boundary

Process an explicitly recorded failure price even when tokens are zero. Keep absent prices absent and estimates reader-side; retain failed-agent subcall prices as a breakdown rather than adding them again.

Verification

Cover absent, zero and positive prices on success/error events; include a real scripted failed node without token metadata and run/thread/tenant attribution. Run the replay/cost and full suites with paid tests excluded.

Acceptance

  • Known provider USD survives run/thread/tenant attribution without token metadata.
  • A zero price is recorded zero; a missing price stays missing.
  • Replay and cost attribution agree, without counting subcall prices twice.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions