Skip to content

feat(server-utils): Instrument the OpenAI decisions API as gen_ai.decide spans - #25285

Merged
andreiborza merged 27 commits into
developfrom
ab/openai-decisions
Oct 14, 2026
Merged

andreiborza merged 27 commits into
developfrom
ab/openai-decisions

Conversation

@andreiborza

@andreiborza andreiborza commented Oct 12, 2026 •

Copy link
Copy Markdown
Member

What

OpenAI decisions.create calls now create decide <model> spans with op gen_ai.decide, recorded in the same shape as Jev calls so Sentry renders them the same way:

  • input is recorded as state, images included as they are.
  • Questions and answers are keyed by question name, as the conventions define.
  • Choice and score question options go in criteria, like Jev: choices as { [value]: description }, score levels as their labels.
  • Choice and score probabilities are keyed by option or score value, and score labels go in a legend.
  • predicate answers keep OpenAI's { type, probability }, which Sentry renders as yes/no.
OpenAI sends / returns Recorded on the span
Evaluated input input: "I want a refund" state: "I want a refund"
Questions [{ name: "wants_refund", type: "predicate", ... }] { wants_refund: { type: "predicate", ... } }
Choice question choices: [{ value: "billing", description: "Charges and refunds" }, { value: "other" }] criteria: { billing: "Charges and refunds", other: "" }
Score question levels: [{ label: "low" }, { label: "high", description: "Blocks the customer" }] criteria: ["low", "high"], descriptions: { "1": "Blocks the customer" }
Answers [{ name: "wants_refund", type: "predicate", ... }] { wants_refund: { type: "predicate", ... } }
Predicate answer { type: "predicate", probability: 0.97 } unchanged
Choice answer choice: true, probabilities: [{ value: true, probability: 0.91 }, ...] choice: "true", probabilities: { "true": 0.91, ... }
Score answer probabilities: [{ value: 1, label: "high", probability: 0.95 }, ...] probabilities: { "1": 0.95, ... }, legend: { "1": "high", ... }

OpenAI decisions have a few things Jev does not:

  • predicate questions and answers instead of noul, with the yes probability in probability.
  • Choice values can be booleans as well as strings. The answer's choice is recorded as a string so Sentry renders it.
  • Score levels can have descriptions, recorded in descriptions keyed by score value. Sentry does not render them yet.
    "urgency": {
      "type": "score",
      "instructions": "How urgent is this ticket?",
      "criteria": ["low", "medium", "high"],
      "descriptions": { "1": "Needs a reply today", "2": "Blocks the customer" }  <-- Jev does not have these
    }
  • A question can come back as a refusal, { type: "refusal" }, with no probability or reason. Sentry does not render it yet.
    "answers": {
      "wants_refund": { "type": "predicate", "probability": 0.97 },
      "urgency": { "type": "refusal" }  <-- Jev does not have these 
    }
  • The input can hold images as base64 data URLs.
    "state": [
      {
        "role": "user",
        "content": [
          { "type": "input_text", "text": "Is this image mostly red?" },
          { "type": "input_image", "image_url": "data:image/png;base64,iVBORw0KGgo..." }  <-- Jev does not have these
        ]
      }
    ]

Why

OpenAI added a Decisions API that works like Jev, and these calls had no span.

Related: #25253

…ide spans

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread dev-packages/node-integration-tests/suites/tracing/openai/v7/test.ts Outdated
@github-actions

github-actions Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

size-limit report 📦

⚠️ Warning: Base artifact is not the latest one, because the latest workflow run is not done yet. This may lead to incorrect results. Try to re-run all tests to get up to date results.

Path Size % Change Change
@sentry/browser 29.83 kB - -
@sentry/browser - with treeshaking flags 27.96 kB - -
@sentry/browser - with treeshaking flags tracing without tracing 27.87 kB - -
@sentry/browser (incl. Tracing) 52.02 kB - -
@sentry/browser (incl. Tracing + Span Streaming) 52.02 kB - -
@sentry/browser (incl. Tracing, Profiling) 54.96 kB - -
@sentry/browser (incl. Tracing, Replay) 91.75 kB - -
@sentry/browser (incl. Tracing, Replay) - with treeshaking flags 80.62 kB - -
@sentry/browser (incl. Tracing, Replay with Canvas) 96.47 kB - -
@sentry/browser (incl. Tracing, Replay, Feedback) 109.43 kB - -
@sentry/browser (incl. Feedback) 47.35 kB - -
@sentry/browser (incl. sendFeedback) 34.89 kB - -
@sentry/browser (incl. FeedbackAsync) 40 kB - -
@sentry/browser (incl. Metrics) 30.85 kB - -
@sentry/browser (incl. Logs) 31.13 kB - -
@sentry/browser (incl. Metrics & Logs) 31.78 kB - -
@sentry/react 31.64 kB - -
@sentry/react (incl. Tracing) 54.34 kB - -
@sentry/vue 37.98 kB - -
@sentry/vue (incl. Tracing) 54.95 kB - -
@sentry/svelte 29.86 kB - -
@sentry/remix (Remix 3 client bundle) 57.02 kB - -
CDN Bundle 31.56 kB - -
CDN Bundle (incl. Tracing) 52.52 kB - -
CDN Bundle (incl. Logs, Metrics) 33.77 kB - -
CDN Bundle (incl. Tracing, Logs, Metrics) 54.46 kB - -
CDN Bundle (incl. Replay, Logs, Metrics) 74.7 kB - -
CDN Bundle (incl. Tracing, Replay) 90.19 kB - -
CDN Bundle (incl. Tracing, Replay, Logs, Metrics) 92.11 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback) 96.34 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback, Logs, Metrics) 98.28 kB - -
CDN Bundle - uncompressed 92.98 kB - -
CDN Bundle (incl. Tracing) - uncompressed 155.83 kB - -
CDN Bundle (incl. Logs, Metrics) - uncompressed 99.52 kB - -
CDN Bundle (incl. Tracing, Logs, Metrics) - uncompressed 161.79 kB - -
CDN Bundle (incl. Replay, Logs, Metrics) - uncompressed 229.56 kB - -
CDN Bundle (incl. Tracing, Replay) - uncompressed 276 kB - -
CDN Bundle (incl. Tracing, Replay, Logs, Metrics) - uncompressed 281.94 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback) - uncompressed 289.7 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback, Logs, Metrics) - uncompressed 295.63 kB - -
@sentry/nextjs (client) 56.75 kB - -
@sentry/sveltekit (client) 52.41 kB - -
@sentry/core/server 41.12 kB - -
@sentry/core/browser 13.81 kB - -
@sentry/node 152.43 kB +0.31% +460 B 🔺
@sentry/node/import (ESM hook with diagnostics-channel injection) 83.81 kB +0.04% +30 B 🔺
@sentry/node - without tracing 94.57 kB +0.04% +34 B 🔺
@sentry/node - without channel injection 130.66 kB +0.38% +484 B 🔺
@sentry/aws-serverless 102.73 kB +0.04% +39 B 🔺
@sentry/cloudflare (withSentry) - minified 210.38 kB - -
@sentry/cloudflare (withSentry) 521.61 kB - -
@sentry/nextjs/cloudflare (withSentry) - minified 228.06 kB - -

View base workflow run

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 0cf6661. Configure here.

Comment thread packages/server-utils/src/ai/openai/utils.ts Outdated
Comment thread packages/server-utils/src/ai/openai/index.ts
@andreiborza
andreiborza added this pull request to stack #25332 October 13, 2026 13:41
@andreiborza
andreiborza marked this pull request as ready for review October 13, 2026 13:57
@andreiborza
andreiborza requested a review from a team as a code owner October 13, 2026 13:57
@andreiborza
andreiborza requested review from mydea and removed request for a team October 13, 2026 13:57
Comment thread dev-packages/node-integration-tests/suites/tracing/openai/v7/test.ts Outdated
Comment thread dev-packages/node-integration-tests/suites/tracing/openai/v7/test.ts Outdated
Comment thread packages/server-utils/src/ai/openai/index.ts
Comment thread packages/server-utils/src/ai/openai/index.ts Outdated
Comment thread packages/server-utils/src/ai/openai/mapping.ts Outdated
Comment thread packages/server-utils/src/ai/openai/mapping.ts
Comment thread packages/server-utils/src/ai/openai/mapping.ts Outdated
Comment thread packages/server-utils/src/ai/openai/mapping.ts Outdated
Comment thread packages/server-utils/src/ai/openai/utils.ts
Comment thread packages/server-utils/src/ai/openai/mapping.ts

@nicohrubec nicohrubec left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for updating, lgtm

@JPeer264 JPeer264 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

{ type: 'predicate', name: 'lgtm', probability: 0.99 }

@andreiborza
andreiborza merged commit 8ec5fe4 into develop Oct 14, 2026
732 of 734 checks passed
@andreiborza
andreiborza deleted the ab/openai-decisions branch October 14, 2026 14:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants