Skip to content

feat(aio): declare session id and per-turn traceparent to the llm gateway - #3927

Merged
trunk-io[bot] merged 5 commits into
mainfrom
posthog-code/aio-sessions-turn-traces-design
Jul 31, 2026
Merged

feat(aio): declare session id and per-turn traceparent to the llm gateway#3927
trunk-io[bot] merged 5 commits into
mainfrom
posthog-code/aio-sessions-turn-traces-design

Conversation

@carlos-marchal-ph

Copy link
Copy Markdown

Problem

Every $ai_generation a PostHog Code task produces lands mis-shaped in AI Observability: the Claude path collapses a whole task (observed: 1,500+ generations over 45h) into one $ai_trace_id, the codex path scatters each LLM call into its own singleton trace, and $ai_session_id is null on 100% of events — so the AIO sessions UI shows nothing for Code.

Changes

  • Claude harness (packages/agent/.../options.ts): buildEnvironment now emits x-posthog-property-$ai_session_id: <ACP session id> in ANTHROPIC_CUSTOM_HEADERS (resume keeps the id, fork mints a new one). For gateway-routed sessions only, it sets the four env vars that make CLI 2.1.197 stamp a per-turn W3C traceparent header (verified locally: trace-id rotates per user turn, survives a dead OTLP endpoint), and strips inherited TRACEPARENT/TRACESTATE so ambient CI/sandbox trace context can't collapse turns. BYOK sessions byte-identical.
  • Codex harness: both hosts send $ai_session_id: <task id> — cloud via configureEnvironment's openaiCustomHeaders, desktop via codexOptions.httpHeaders (previously unset). Record form only; the Claude header string never gains the key (test-locked against double-send). Codex has no per-turn signal, so its traces stay per-call; the session is its task-level grouping.

Inert until the gateway counterpart (PostHog/posthog feat(aio)/llm-gateway-turn-traces) deploys — today's gateway ignores these headers. Once both are live, posthog_code trace cardinality rises ~5-20× on the Claude path (dashboards treating trace=task should be audited) and Code tasks appear in the AIO sessions UI.

How did you test this?

Automated only: 6 new/updated Vitest cases — session-header ordering in ANTHROPIC_CUSTOM_HEADERS, tracing vars gated on gateway routing, user OTEL_* overrides honored, ambient TRACEPARENT stripped, BYOK untouched, codex header present/absent by task id, TOML $-key serialization. Full @posthog/agent suite 1665/1665, typecheck clean, Biome warnings unchanged from base. The traceparent flag recipe itself was verified against the real pinned CLI run in stream-json mode against a header-logging fake gateway (two turns → two trace-ids, shared within turn). End-to-end SQL/UI verification (two-prompt repro) deferred to rollout — needs the gateway change deployed.

Automatic notifications

  • Publish to changelog?
  • Alert Sales and Marketing teams?

Created with PostHog Code

Generated-By: PostHog Code
Task-Id: 1119b267-6fcd-4dba-bc80-f15665f7a9b1
Generated-By: PostHog Code
Task-Id: 1119b267-6fcd-4dba-bc80-f15665f7a9b1
Generated-By: PostHog Code
Task-Id: 1119b267-6fcd-4dba-bc80-f15665f7a9b1
@carlos-marchal-ph
carlos-marchal-ph requested a review from a team July 29, 2026 10:38
@carlos-marchal-ph carlos-marchal-ph self-assigned this Jul 29, 2026
@trunk-io

trunk-io Bot commented Jul 29, 2026

Copy link
Copy Markdown

😎 Merged successfully - details.

@github-actions

github-actions Bot commented Jul 29, 2026

Copy link
Copy Markdown

React Doctor found no issues in the changed files. 🎉

Reviewed by React Doctor for commit 8bbeb78.

Comment on lines +202 to +204
OTEL_TRACES_EXPORTER: process.env.OTEL_TRACES_EXPORTER ?? "otlp",
OTEL_EXPORTER_OTLP_PROTOCOL:
process.env.OTEL_EXPORTER_OTLP_PROTOCOL ?? "http/json",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Inherited settings disable turn tracing

When a gateway-routed Claude session inherits OTEL_TRACES_EXPORTER=none or an incompatible OTEL_EXPORTER_OTLP_PROTOCOL, this block preserves that configuration even though tracer initialization is required to stamp traceparents, causing generations to remain untraced or incorrectly grouped.

Knowledge Base Used: Agent SDK (packages/agent)

Prompt To Fix With AI
This is a comment left during a code review.
Path: packages/agent/src/adapters/claude/session/options.ts
Line: 202-204

Comment:
**Inherited settings disable turn tracing**

When a gateway-routed Claude session inherits `OTEL_TRACES_EXPORTER=none` or an incompatible `OTEL_EXPORTER_OTLP_PROTOCOL`, this block preserves that configuration even though tracer initialization is required to stamp traceparents, causing generations to remain untraced or incorrectly grouped.

**Knowledge Base Used:** [Agent SDK (`packages/agent`)](https://app.greptile.com/posthog-org-19734/-/custom-context/knowledge-base/posthog/code/-/docs/agent-sdk.md)

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed!

@greptile-apps

greptile-apps Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor
Prompt To Fix All With AI
### Issue 1
packages/agent/src/adapters/claude/session/options.ts:202-204
**Inherited settings disable turn tracing**

When a gateway-routed Claude session inherits `OTEL_TRACES_EXPORTER=none` or an incompatible `OTEL_EXPORTER_OTLP_PROTOCOL`, this block preserves that configuration even though tracer initialization is required to stamp traceparents, causing generations to remain untraced or incorrectly grouped.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Reviews (1): Last reviewed commit: "feat: forward the task id as the codex a..." | Re-trigger Greptile

// leaking to a local collector on the default port. Export never matters.
const gatewayTracing: Record<string, string> = gateway?.anthropicBaseUrl
? {
CLAUDE_CODE_ENABLE_TELEMETRY: "1",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Medium: Repository settings can exfiltrate tool content through telemetry

Project .claude/settings.json files are loaded by default and can define environment variables. A malicious repository can set OTEL_EXPORTER_OTLP_TRACES_ENDPOINT (which takes precedence over the dead generic endpoint) and enable tool-content attributes; this new telemetry switch then exports tool inputs and outputs, including source code read by the agent, to the attacker's collector. Pin or strip the signal-specific OTel endpoints, exporters, headers, and content-enabling variables after project settings are resolved, or prevent repository settings from configuring telemetry for these embedded sessions.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this a pre-existing issue? You just looked at this code now, but nothing was stopping anyone before from just setting ALL of these env vars on the JSON file and exfiltrating data right? I repro'd it locally on main.

Also the attack vector for this is a compromised repo that gets trusted by Claude Code, in which case there's a million different ways to be evil, like scripts, skills, etc. Am I missing something here that I caused?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the detail. I can't automatically confirm this is safe to dismiss, so I'm leaving the thread open for a maintainer to make the call.

@veria-ai

veria-ai Bot commented Jul 29, 2026

Copy link
Copy Markdown

PR overview

This PR adds session ID and per-turn traceparent propagation to the LLM gateway and updates Claude session telemetry options to support the new tracing behavior.

One security issue remains open. A malicious repository can use project-level Claude settings to redirect telemetry and include tool inputs and outputs, potentially exposing source code and other agent-accessed content to an attacker-controlled collector.

Open issues (1)

Fixed/addressed: 0 · PR risk: 7/10

@tatoalo tatoalo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree on veria comment but stamping!

Generated-By: PostHog Code
Task-Id: 409fb75d-ab0b-4dca-b960-df1f721982c1
@posthog

posthog Bot commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

👋 Visual changes detected for this PR.

Review and approve in PostHog Visual Review

If these changes are unexpected, they may be caused by a flaky test or a broken snapshot on master. Don't approve — rerun the job or wait for a fix.

@carlos-marchal-ph
carlos-marchal-ph enabled auto-merge (squash) July 31, 2026 15:55
@trunk-io
trunk-io Bot merged commit 4cf4612 into main Jul 31, 2026
34 checks passed
@trunk-io
trunk-io Bot deleted the posthog-code/aio-sessions-turn-traces-design branch July 31, 2026 16:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants