fix(gemini): normalize tool call/response parts in captured input - #823
Open
marco-g-pm wants to merge 1 commit into
Open
fix(gemini): normalize tool call/response parts in captured input#823marco-g-pm wants to merge 1 commit into
marco-g-pm wants to merge 1 commit into
Conversation
Contributor
posthog-python Compliance ReportDate: 2026-07-31 16:23:25 UTC ✅ All Tests Passed!111/111 tests passed Capture_V1 Tests✅ 94/94 tests passed View Details
Feature_Flags Tests✅ 17/17 tests passed View Details
|
marco-g-pm
added a commit
to PostHog/posthog
that referenced
this pull request
Jul 31, 2026
posthog-python 7.30.1-7.35.x wrote Gemini's native part shapes to
`$ai_input` verbatim:
{"type": "function_call", "function_call": {name, args}}
{"type": "function_response", "function_response": {name, response}}
Neither matches any shape `isToolResult`/`isToolStepItem` know, so
traces captured in that window misclassify their tool loop three ways:
- `isToolMessage` computes `hasText` from content items that are neither
a tool step nor a tool result, so a message carrying only a tool
response counts as having real text and gets surfaced in the traces
list as the human-readable answer.
- `isInternalToolResultUserMessage` requires every content item to be a
tool result, so the framework-appended tool-response user message
isn't classified internal and leaks as a visible user bubble.
- `isUnrenderableContentItem` treats the block as non-text content the
transcript can't render.
The SDK now normalizes both kinds (PostHog/posthog-python#823), but
already-ingested traces keep the old shape, so match it here too.
`function_call` needs its nested object checked: OpenAI's Responses API
uses the same `type` with `name`/`call_id` at the top level, and
`isOpenAIResponsesFunctionCall` already owns that case.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2 tasks
Gemini's native `function_call`/`function_response` parts were passed
through to `$ai_input` verbatim, in a shape no PostHog consumer
recognizes. The trace renderer matches tool results against Anthropic
`tool_result`, Vercel `tool-result`, OpenAI Responses
`function_call_output`, and `{type: "function", tool_name, content}` —
none of which match — so tool responses never rendered, and the eval
formatter fell through to a raw `repr()` dump instead of a labeled tool
result.
The same SDK already normalized tool calls on the response path to
`{type: "function", function: {...}}`, which is why a call rendered in
the output but vanished once it appeared in the next request's history.
Normalize both part kinds in `_format_part`. Tool results use
`tool_name`/`content` rather than an id-carrying shape because Gemini
matches responses to calls by name and has no call ids to thread.
`$ai_output_choices` is unaffected: `format_gemini_response` hits its
explicit `function_call` branch before the `_format_part` fallback and
already emitted the normalized shape.
Fixes #725
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
marco-g-pm
force-pushed
the
claude/posthog-python-issue-725-573c16
branch
from
July 31, 2026 16:16
517694e to
5b65166
Compare
marco-g-pm
added a commit
to PostHog/posthog
that referenced
this pull request
Jul 31, 2026
posthog-python 7.30.1-7.35.x wrote Gemini's native part shapes to
`$ai_input` verbatim:
{"type": "function_call", "function_call": {name, args}}
{"type": "function_response", "function_response": {name, response}}
Neither matches any shape `isToolResult`/`isToolStepItem` know, so
traces captured in that window misclassify their tool loop three ways:
- `isToolMessage` computes `hasText` from content items that are neither
a tool step nor a tool result, so a message carrying only a tool
response counts as having real text and gets surfaced in the traces
list as the human-readable answer.
- `isInternalToolResultUserMessage` requires every content item to be a
tool result, so the framework-appended tool-response user message
isn't classified internal and leaks as a visible user bubble.
- `isUnrenderableContentItem` treats the block as non-text content the
transcript can't render.
The SDK now normalizes both kinds (PostHog/posthog-python#823), but
already-ingested traces keep the old shape, so match it here too.
`function_call` needs its nested object checked: OpenAI's Responses API
uses the same `type` with `name`/`call_id` at the top level, and
`isOpenAIResponsesFunctionCall` already owns that case.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
marco-g-pm
added a commit
to PostHog/posthog
that referenced
this pull request
Jul 31, 2026
posthog-python 7.30.1-7.35.x copied Gemini function calls and their
responses into `$ai_input` in the raw form the google-genai SDK emits:
{"type": "function_call", "function_call": {name, args}}
{"type": "function_response", "function_response": {name, response}}
Neither matches any shape `isToolResult`/`isToolStepItem` know, so
traces captured in that window misclassify their tool loop three ways:
- `isToolMessage` computes `hasText` from content items that are neither
a tool step nor a tool result, so a message carrying only a tool
response counts as having real text and gets surfaced in the traces
list as the human-readable answer.
- `isInternalToolResultUserMessage` requires every content item to be a
tool result, so the framework-appended tool-response user message
isn't classified internal and leaks as a visible user bubble.
- `isUnrenderableContentItem` treats the block as non-text content the
transcript can't render.
The SDK now converts both kinds (PostHog/posthog-python#823), but
already-ingested traces keep the old form, so match it here too.
`function_call` needs its nested object checked: OpenAI's Responses API
uses the same `type` with `name`/`call_id` at the top level, and
`isOpenAIResponsesFunctionCall` already owns that case.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
marco-g-pm
marked this pull request as ready for review
July 31, 2026 21:19
Contributor
|
Reviews (1): Last reviewed commit: "fix(gemini): normalize tool call/respons..." | Re-trigger Greptile |
marandaneto
approved these changes
Aug 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
💡 Motivation and Context
Fixes #725.
The Gemini wrapper copied function calls and their responses into
$ai_input, in the raw form the google-genai SDK emits:{"type": "function_call", "function_call": {"name": "...", "args": {}}} {"type": "function_response", "function_response": {"name": "...", "response": {}}}No PostHog consumer knows those two shapes. The trace view matches four other tool-result shapes, so it never showed the tool response. The eval formatter found no
textkey and nocontentkey, so it fell back to a rawrepr()dump. Evals then flagged correct answers as hallucinations.The response path already emitted the shared shape. That is why a tool call showed in the output, then vanished from the next request's history.
Changes
_format_partnow emits the shared shapes:function_callbecomes{"type": "function", "function": {name, arguments}}function_responsebecomes{"type": "function", "tool_name": ..., "content": ...}Tool results carry
tool_name, not a call id. Gemini pairs a response to a call by name and has no id to pass through.$ai_output_choicesdoes not change.Out of scope
Automatic function calling. google-genai runs that tool loop inside
generate_content. The turns live only onresponse.automatic_function_calling_history, which this wrapper never reads. I wrote and tested that fix, then dropped it to keep this change small. It needs its own issue.Backward compatibility
The old shape shipped in 7.30.1, four days before this fix. Before 7.30.1 the input path dropped these parts. Nothing reads the old shape. The whole
ai_observabilityproduct has no match forfunction_response.Two points for a reviewer:
format_gemini_inputsits in__all__, so its return shape is public. The Public API snapshot check still passes, because the signature did not change.functionkey. That is what stopsisToolStepItemfrom reading it as a call.💚 How did you test it?
test_function_call_and_response_turns_preservedfor both new shapes.posthog/test/ai/: 438 pass, 61 skip. I left outotel/, which needs two packages my local env lacks.format_gemini_inputover a realgoogle-genaifunction-calling turn.repr()dump. The new shape gives[FUNCTION]and the tool output, the same as Anthropictool_result.I did not test against a live Gemini key or a real project.
📝 Checklist
If releasing new changes
sampo addto generate a changeset fileFollow-up
PostHog/posthog#76028 teaches the trace view the old shape, so traces already stored render too. That fix covers the view only. Those traces still feed evals a
repr()dump, because the eval path is a separate Python formatter.🤖 Agent context
Autonomy: Human-driven (agent-assisted)
Written with Claude Code (Opus 5), directed by @marco-g-pm.
I first assumed a frontend patch would also fix the evals. It does not. Evals run through a separate Python formatter, which I checked by running it against both shapes. That result is what made the SDK the right place to fix this.