Skip to content

Conversation

@alexander-alderman-webb
Copy link
Contributor

@alexander-alderman-webbalexander-alderman-webb commented Dec 10, 2025

Description

Add prompt, response, and total token counts to LangGraph invocation spans.

Issues

Contributes to #5170

Reminders

@codecov
Copy link

codecovbot commented Dec 10, 2025

Codecov Report

❌ Patch coverage is 92.30769% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.22%. Comparing base (1b7085d) to head (4f3fab3).
⚠️ Report is 6 commits behind head on master.
✅ All tests successful. No failed tests found.

Files with missing linesPatch %Lines
sentry_sdk/integrations/langgraph.py92.30%1 Missing and 1 partial ⚠️
Additional details and impacted files
@@ Coverage Diff @@## master #5211 +/- ## ========================================== + Coverage 84.17% 84.22% +0.04%  ========================================== Files 181 181 Lines 18443 18486 +43 Branches 3283 3295 +12 ========================================== + Hits 15524 15569 +45 + Misses 1904 1899 -5 - Partials 1015 1018 +3 
Files with missing linesCoverage Δ
sentry_sdk/integrations/langgraph.py76.27% <92.30%> (-0.81%)⬇️

... and 5 files with indirect coverage changes

@alexander-alderman-webbalexander-alderman-webb marked this pull request as ready for review December 11, 2025 09:22
Copy link

@cursorcursorbot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Usage data miscounted when PII collection disabled

When should_send_default_pii() is false or include_prompts is false, input_messages remains None. This causes _get_new_messages in _set_response_attributes to return all output messages instead of just the new ones. Since LangGraph state accumulates messages, usage data will include tokens from all messages in the response rather than only the new messages added during this invocation. The input messages need to be parsed unconditionally (at least for _get_new_messages) to correctly calculate usage data regardless of PII settings.

sentry_sdk/integrations/langgraph.py#L184-L204

# Store input messages to later compare with output
input_messages=None
if (
len(args) >0
andshould_send_default_pii()
andintegration.include_prompts
):
input_messages=_parse_langgraph_messages(args[0])
ifinput_messages:
normalized_input_messages=normalize_message_roles(input_messages)
scope=sentry_sdk.get_current_scope()
messages_data=truncate_and_annotate_messages(
normalized_input_messages, span, scope
)
ifmessages_dataisnotNone:
set_data_normalized(
span,
SPANDATA.GEN_AI_REQUEST_MESSAGES,
messages_data,
unpack=False,
)

sentry_sdk/integrations/langgraph.py#L240-L260

input_messages=None
if (
len(args) >0
andshould_send_default_pii()
andintegration.include_prompts
):
input_messages=_parse_langgraph_messages(args[0])
ifinput_messages:
normalized_input_messages=normalize_message_roles(input_messages)
scope=sentry_sdk.get_current_scope()
messages_data=truncate_and_annotate_messages(
normalized_input_messages, span, scope
)
ifmessages_dataisnotNone:
set_data_normalized(
span,
SPANDATA.GEN_AI_REQUEST_MESSAGES,
messages_data,
unpack=False,
)

Fix in CursorFix in Web


Bug: Usage data miscounted when PII collection disabled

When should_send_default_pii() is false or include_prompts is false, input_messages remains None. This causes _get_new_messages in _set_response_attributes to return all output messages instead of just the new ones. Since LangGraph state accumulates messages, usage data will include tokens from all messages in the response rather than only the new messages added during this invocation. The input messages need to be parsed unconditionally (at least for _get_new_messages) to correctly calculate usage data regardless of PII settings.

sentry_sdk/integrations/langgraph.py#L184-L204

# Store input messages to later compare with output
input_messages=None
if (
len(args) >0
andshould_send_default_pii()
andintegration.include_prompts
):
input_messages=_parse_langgraph_messages(args[0])
ifinput_messages:
normalized_input_messages=normalize_message_roles(input_messages)
scope=sentry_sdk.get_current_scope()
messages_data=truncate_and_annotate_messages(
normalized_input_messages, span, scope
)
ifmessages_dataisnotNone:
set_data_normalized(
span,
SPANDATA.GEN_AI_REQUEST_MESSAGES,
messages_data,
unpack=False,
)

sentry_sdk/integrations/langgraph.py#L240-L260

input_messages=None
if (
len(args) >0
andshould_send_default_pii()
andintegration.include_prompts
):
input_messages=_parse_langgraph_messages(args[0])
ifinput_messages:
normalized_input_messages=normalize_message_roles(input_messages)
scope=sentry_sdk.get_current_scope()
messages_data=truncate_and_annotate_messages(
normalized_input_messages, span, scope
)
ifmessages_dataisnotNone:
set_data_normalized(
span,
SPANDATA.GEN_AI_REQUEST_MESSAGES,
messages_data,
unpack=False,
)

Fix in CursorFix in Web


@alexander-alderman-webbalexander-alderman-webb marked this pull request as draft December 11, 2025 09:29
@alexander-alderman-webbalexander-alderman-webb marked this pull request as ready for review December 11, 2025 09:52
@alexander-alderman-webb
Copy link
ContributorAuthor

This is okay, input messages do not have token info attached to them.

usage data will include tokens from all messages in the response rather than only the new messages added during this invocation.

@alexander-alderman-webbalexander-alderman-webb merged commit 40e5083 into masterDec 11, 2025
154 checks passed
@alexander-alderman-webbalexander-alderman-webb deleted the webb/langgraph-major branch December 11, 2025 13:57
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@alexander-alderman-webb@sentrivana