fix(observe): retain failed-call prices without token metadata

ROADMAP.md

@@ -454,7 +454,7 @@ allowances; the agent records document-driven denials and approval decisions.
 - [x] **10.4 — Cost attribution** per run, thread (session), node and explicit tenant. The
       price is now recorded, not guessed: every gateway publishes a
       `cost_usd` through the same `llm_output` envelope, the runtime's usage
-      callback accumulates it per node and writes it onto the `end` event, and
+      callback accumulates it per node and writes it onto the `end` or `error` event, and
       an `AgentNode` writes the per-call figure onto each `model` event. A
       backend that reports no price still falls back to a `RateCard` estimate,
       and `recorded_cost_usd` and `estimated_cost_usd` never mix — a recorded
@@ -467,9 +467,11 @@ allowances; the agent records document-driven denials and approval decisions.
       `grapharc cost --tenant NAME` select whole labelled runs, refuse conflicting
       labels, and leave old unlabelled runs unassigned. Planning tokens without
       a recorded price are estimated or marked unpriced, including in tenant
-      totals. **Producer limit:** planner turns and failed node terminal events
-      still omit the provider price ([#138](https://github.com/CodeGraphContext/GraphARC/issues/138));
-      a rate-card estimate is not a recorded bill.
+      totals. Planner turns retain the call-specific price and model from the
+      returned message envelope, including parse failures. Every node terminal
+      error path retains its own usage scope, excluding concurrent siblings'
+      spend. A failed execution's reported USD is retained even without token
+      metadata. A rate-card estimate is not a recorded bill.
       Tenant labels provide attribution, not authorization.
 - [ ] **10.2 — Rollback** and versioned graph/prompt configs.
 - [ ] **10.5 — Alerting** on budget, failure, and verifier-drift.

docs/cookbook/07-slack.md

@@ -166,11 +166,22 @@ status message is posted to the channel (a `respond()`-style reply would allow
 only five updates), so a live run is visible to everyone in it — the mention
 path threads it under your message as before.
 
-Because the trace now lands inside the working directory, the run is also
-inspectable afterwards from Slack itself: `/grapharc metrics
-slack-runs/<stamp>/trace.jsonl <run-id>`, `viz` for the finished diagram,
-`replay` for the reconstruction. The `slack-runs/` directories are the audit
-trail and are never cleaned up automatically; prune them like any other logs.
+Because the trace lands inside the working directory, the run is also
+inspectable afterwards from Slack itself: `/grapharc metrics`, `viz` for the
+finished diagram, `replay` for the reconstruction, or `trace` for its events.
+Bare readers use the latest completed trace remembered for that channel. In a
+new channel or after a bot restart, they fall back to the newest trace under
+`slack-runs/`. Each reply names the selected path, run id and whether it used
+that workdir fallback. A trace containing zero or multiple runs needs explicit
+selection; the bot does not guess. With no previous trace, it suggests
+`demo stage0` instead of returning an argparse usage error.
+
+Explicit commands such as `/grapharc metrics
+slack-runs/<stamp>/trace.jsonl <run-id>` keep working. `diff` still needs an
+explicit path and two run ids. Inferred paths pass the same confinement gate
+as typed paths. Channel history is process-local; the traces are durable. The
+`slack-runs/` directories are the audit trail and are never cleaned up
+automatically; prune them like any other logs.
 
 ## Watching it live in a browser
 

docs/deep-dive.md

@@ -94,7 +94,7 @@ OpenRouter also carries routing: model-level `fallback_models` chains, provider
 
 **Cost ceilings are enforced, on both sides of a call.** A `SpendMeter` refuses before a call once the ceiling is reached, and after a call it charges the cost and *then* raises if that crossed the line — so overspend is bounded by the single call that crossed it rather than discovered a node later. The limit worth stating: a call whose cost the provider does not report cannot be charged, and those land in `unpriced_calls` rather than being guessed at. `unpriced_calls > 0` means the ceiling saw less than the whole bill.
 
-Caveats each backend accepts openly. **Claude CLI:** `bind_tools` and `with_structured_output` raise `NotImplementedError` — the adapter implements neither, so what you get is LangChain's `BaseChatModel` default, and `claude -p` in the tool-free mode GraphARC drives it in offers nothing to implement them with. GraphARC also has no cache control on this path — the CLI decides and the usage envelope reports what it did — and calls spend subscription quota. **OpenRouter:** credit is reserved against `max_tokens`, so the default is deliberately modest. **OpenAI:** the API returns token counts and no price, so `cost_usd` is `None` and a dollar ceiling counts calls instead of enforcing — pass `price_per_million=` or price the trace afterwards with `observe.cost.RateCard`; token budgets are unaffected. **Ollama:** free by definition, so calls are charged `0.0` rather than counted as unpriced, and whether tool-calling works depends on the model you pulled rather than on the adapter. **All of them:** a reported per-call `cost_usd` is captured and budgeted against. Successful node events and agent model events carry it into the trace; planner turns and failed node terminal events currently omit it. See [ROADMAP.md](../ROADMAP.md) §10.4 for explicit tenant attribution and producer pricing limits.
+Caveats each backend accepts openly. **Claude CLI:** `bind_tools` and `with_structured_output` raise `NotImplementedError` — the adapter implements neither, so what you get is LangChain's `BaseChatModel` default, and `claude -p` in the tool-free mode GraphARC drives it in offers nothing to implement them with. GraphARC also has no cache control on this path — the CLI decides and the usage envelope reports what it did — and calls spend subscription quota. **OpenRouter:** credit is reserved against `max_tokens`, so the default is deliberately modest. **OpenAI:** the API returns token counts and no price, so `cost_usd` is `None` and a dollar ceiling counts calls instead of enforcing — pass `price_per_million=` or price the trace afterwards with `observe.cost.RateCard`; token budgets are unaffected. **Ollama:** free by definition, so calls are charged `0.0` rather than counted as unpriced, and whether tool-calling works depends on the model you pulled rather than on the adapter. **All of them:** a reported per-call `cost_usd` is captured and budgeted against. Successful and failed node terminal events, planner turns and agent model events carry it into the trace. See [ROADMAP.md](../ROADMAP.md) §10.4 for explicit tenant attribution and producer pricing limits.
 
 
 ## Tools and the harness
@@ -248,15 +248,15 @@ A stable system is not one that claims to have no edges — it is one whose edge
 - **`run_command` is not confined.** Argv-only and never a shell, but the child is an ordinary process with your privileges.
 - **`interrupt()` suspends but cannot be resumed.** LangGraph's native interrupt stops the graph and shows on `get_state`, and there is no supported resume path — resuming means passing a `Command` as *input*, which is closed by design. Use the session layer's approval gate for human-in-the-loop.
 - **Still unwrapped from LangGraph:** `retry_policy`, `cache_policy`, `durability`, subgraphs. `.inner` reaches them, but execution entry points there fail closed, so `.inner` is an inspection escape hatch and not a way to run the graph.
-- **Cost is recorded when a backend reports one, estimated when it does not.** Both gateways publish the provider's `cost_usd` through the same `llm_output` envelope, the runtime's usage callback writes it onto the node's `end` event, and an agent's `model` events carry the per-call breakdown. A backend that reports no price still falls back to a `RateCard` estimate, and the two figures stay apart — `recorded_cost_usd` is never a guess. Explicit tenants are supported by `attribute_tenant` and `grapharc cost`; legacy unlabelled runs are excluded from named tenant bills. Planner turns and failed node terminal events still omit the provider price, so their tokens need a rate card or remain unpriced.
+- **Cost is recorded when a backend reports one, estimated when it does not.** Both gateways publish the provider's `cost_usd` through the same `llm_output` envelope, the runtime's usage callback writes it onto the node's `end` event, and an agent's `model` events carry the per-call breakdown. A backend that reports no price still falls back to a `RateCard` estimate, and the two figures stay apart — `recorded_cost_usd` is never a guess. Explicit tenants are supported by `attribute_tenant` and `grapharc cost`; legacy unlabelled runs are excluded from named tenant bills. Planner turns and all failed node terminal paths retain the provider price and model identity when reported; parallel node failures retain only their own token spend.
 - **A node's tokens are its own, not the run's movement while it ran.** Worth stating because it was the other way round: an `end` event carried the difference between two readings of the run's *shared* meter, so under fan-out the workers' windows overlapped and each was credited with its siblings' concurrent spend. Three workers costing 8 tokens each traced as 24/16/8, and `metrics` and `cost` agreed on 48 for 24 tokens of real work — doubling the estimated bill purely because the work ran in parallel. Attribution now comes from a per-node scope on the meter, so the same work costs the same serially and in parallel; a hand charge the usage callback never saw still lands on the node that made it.
 - **A planning round is an envelope, not a measurement.** A `round` event used to carry the planner's `tokens` and the round's `duration_ms`, both of which `metrics`, `cost` and `replay` add on top of node totals — and the planner's spend was already reported by its own `plan` event, so it was counted twice, and a round's duration encloses the plan plus every node it ran. Neither is on the event now; both are on its `state_delta` as `round_tokens` / `round_iterations` / `round_duration_ms`, where no reader sums them. `RoundRecord.iterations` also holds a figure now rather than always `0`.
 - **The Claude CLI backend is completion-only, and an agent node on it is *delegated* rather than governed.** The CLI has no tool-calling wire format, so GraphARC cannot run its own gated loop over it. Rather than refuse, `AgentNode` hands the whole loop to Claude Code's headless agent, in one of two named tiers. `allowlist`, the default, pre-approves exactly the Claude Code twins of the node's own registered tools (`read_file`→`Read`, … `run_command`→`Bash`), so one operator declaration governs both the governed loop and the delegated one — but the enforcement is Claude Code's own gating, not this graph's per-call policy, there are no per-tool trace events, and anything unlisted falls to headless default gating, which fails closed. `bypass`, explicit opt-in only, runs `bypassPermissions`: every tool Claude Code has, no checks at all. In either tier the calls are not confined by the sandbox executor and the token figure is the sub-agent's own rather than one GraphARC metered call by call; a `--max-tokens` the delegated path cannot enforce is refused rather than silently unapplied. The workspace boundary and the wall-clock ceiling still hold — the CLI runs in its own session, and the deadline kills the whole process group, not just the direct child. It warns on `DelegatedToolUseWarning` at construction, naming the tier, and marks every trace event `executor=delegated` with its `delegated_mode`; filter that warning to an error to get the old refusal back. Structured output still needs an OpenAI-wire backend: `openrouter`, `openai`, or a local `ollama`.
 - **A session turn is synchronous**, and a runner claim is a claim rather than a lease — a runner that dies holding a session does not release it by itself. `SessionManager.reclaim(session_id)` releases one deliberately, refusing unless the recorded runner pid is gone from this host; it is not automatic, because pid liveness is host-local and pids are recycled.
 - **`.env` and `grapharc.toml` follow the same discovery rule: the working directory, and nowhere else.** Neither searches parent directories — a run must not be governed by a file you did not know about, and must not be *billed* to one either. **This is a behaviour change:** the credential loader used to walk up to `/`, so a `.env` in an ancestor directory (a `$HOME` one on a shared box, a client project one above a demo checkout) was picked up silently. If you relied on that, move the file into the directory you run from, `export` the variable, or pass `env_file=` to name it explicitly. A real environment variable still beats any file.
 - **`grapharc run` has no budget unless you give it one.** Set any of `--max-tokens`, `--max-iterations`, `--max-seconds`, or `--max-concurrency`; without them each dimension is unlimited and the gate admits a topology of any worst-case cost.
 
-**Verified this pass:** `pytest` → green, 2,354 selected and 13 deselected (the live ones); `ruff check .` clean; all eight `grapharc demo` stages green, plus the `trace` / `metrics` / `viz` / `replay` tour against a freshly recorded demo trace; the wheel builds and imports all submodules in a clean virtualenv with `[all]`, and `0.1.8` on PyPI is that wheel. The counts are a snapshot, not a property of the project — `pytest` re-derives them in one command, which is the only reason they are quoted, and `tests/test_deep_dive.py` fails this line rather than letting it drift.
+**Verified this pass:** `pytest` → green, 2,408 selected and 13 deselected (the live ones); `ruff check .` clean; all eight `grapharc demo` stages green, plus the `trace` / `metrics` / `viz` / `replay` tour against a freshly recorded demo trace; the wheel builds and imports all submodules in a clean virtualenv with `[all]`, and `0.1.8` on PyPI is that wheel. The counts are a snapshot, not a property of the project — `pytest` re-derives them in one command, which is the only reason they are quoted, and `tests/test_deep_dive.py` fails this line rather than letting it drift.
 
 [ROADMAP.md](../ROADMAP.md) tracks what is built and what is not, item by item.
 

grapharc/observe/cost.py

@@ -348,7 +348,9 @@ def _price_run(run: ReplayedRun, rates: RateCard | None) -> RunCost:
             entry.errors += 1
             entry.tokens += execution.tokens
             entry.duration_ms += execution.duration_ms or 0.0
-            if execution.tokens:
+            # Dollars and tokens are independent optional measurements. A
+            # provider may bill a failed call without reporting token usage.
+            if execution.tokens or execution.cost_usd is not None:
                 estimate, unpriced_here, models = _estimate(execution, card)
                 entry.models = list(dict.fromkeys([*entry.models, *models]))
                 if execution.cost_usd is not None:

grapharc/planner/proposal.py

@@ -690,10 +690,19 @@ def propose(
             error = ""
             proposal = self._stamp(proposal)
 
+        # Read this call's envelope, rather than a model's mutable last_usage:
+        # the latter can describe another concurrent call, or an earlier call
+        # when this one failed before producing a message.
+        metadata = raw_message.response_metadata if raw_message is not None else {}
+        cost = (metadata.get("token_usage") or {}).get("cost")
+        if cost is None:
+            cost = metadata.get("cost_usd")
         self._emit(
             ctx,
             duration_ms=(time.perf_counter() - started) * 1000,
             tokens=tokens,
+            cost_usd=cost,
+            model=metadata.get("model_name"),
             state_delta={
                 "structured": structured,
                 "nodes": proposal.node_count() if proposal else 0,

grapharc/runtime/graph.py

@@ -855,6 +855,7 @@ def _leave(
         usage: Any = None,
     ) -> Any:
         """Everything both node wrappers do after the node body returns."""
+        measured = _usage_fields(ctx, tokens_before, usage)
         delta: dict[str, Any] | None = None
         if result is not None:
             try:
@@ -864,7 +865,7 @@ def _leave(
                     "error",
                     duration_ms=duration_ms,
                     error=str(err),
-                    tokens=ctx.meter.tokens - tokens_before,
+                    **measured,
                 )
                 raise
 
@@ -881,31 +882,15 @@ def _leave(
                 "error",
                 duration_ms=duration_ms,
                 error=f"budget: {exc.reason}",
-                tokens=ctx.meter.tokens - tokens_before,
+                **measured,
             )
             raise
 
-        # The provider's own price for every model call made inside this node,
-        # collected by the usage callback. Left None when no backend reported
-        # one, so `observe.cost` can keep a measured figure apart from a
-        # rate-card estimate instead of recording a guess as a fact.
-        models = getattr(usage, "models", ())
-        # This node's own spend, from the meter's per-node scope, not the movement
-        # of the run's shared total while the node ran. Under fan-out those differ:
-        # the workers' windows overlap, so every worker used to be credited with
-        # each sibling's concurrent spend — three workers costing 8 tokens each
-        # traced as 24/16/8, and `metrics` and `cost` both reported 48 for 24
-        # tokens of real work, doubling the estimated bill purely because the work
-        # ran in parallel. Falls back to the difference when no scope was open, so
-        # a caller driving `_leave` outside `charging()` still gets a figure.
-        spent = getattr(usage, "tokens", None)
         emit(
             "end",
             state_delta=delta,
             duration_ms=duration_ms,
-            tokens=ctx.meter.tokens - tokens_before if spent is None else spent,
-            cost_usd=getattr(usage, "cost_usd", None),
-            model=models[0] if len(models) == 1 else None,
+            **measured,
         )
         return result
 
@@ -935,7 +920,7 @@ async def awrapped(state: Any, config: RunnableConfig) -> Any:
                         "error",
                         duration_ms=(time.perf_counter() - t0) * 1000,
                         error=repr(exc),
-                        tokens=ctx.meter.tokens - tokens_before,
+                        **_usage_fields(ctx, tokens_before, usage),
                     )
                     raise
                 return self._leave(
@@ -974,7 +959,7 @@ def wrapped(state: Any, config: RunnableConfig) -> Any:
                     "error",
                     duration_ms=(time.perf_counter() - t0) * 1000,
                     error=repr(exc),
-                    tokens=ctx.meter.tokens - tokens_before,
+                    **_usage_fields(ctx, tokens_before, usage),
                 )
                 raise
             return self._leave(
@@ -991,6 +976,22 @@ def wrapped(state: Any, config: RunnableConfig) -> Any:
         return wrapped
 
 
+def _usage_fields(ctx: RunContext, tokens_before: int, usage: Any) -> dict[str, Any]:
+    """The same measured spend on every terminal event, successful or failed.
+
+    A node's own scope excludes concurrent siblings' spend. The shared meter
+    difference is only a fallback for callers outside `charging()`. A missing
+    provider price stays missing; rate-card estimates belong to trace readers.
+    """
+    spent = getattr(usage, "tokens", None)
+    models = getattr(usage, "models", ())
+    return {
+        "tokens": ctx.meter.tokens - tokens_before if spent is None else spent,
+        "cost_usd": getattr(usage, "cost_usd", None),
+        "model": models[0] if len(models) == 1 else None,
+    }
+
+
 def topology_delta(arc: GraphARC) -> dict[str, Any]:
     """The declared shape of a graph, as one trace-event `state_delta`.