usage.prompt_tokens_details=None

[tommy-fcy]

Confirm this is an issue with the Python library and not an underlying OpenAI API

• This is an issue with the Python library

Describe the bug

I encountered the problem of usage.prompt_tokens_details=None.
This is the output of response.usage:
CompletionUsage(completion_tokens=57, prompt_tokens=2181, total_tokens=2518, completion_tokens_details=None, prompt_tokens_details=None, reasoning_tokens=280, traffic_type='ON_DEMAND', promptTokensDetails=[{'modality': 'TEXT', 'tokenCount': 2181}], candidatesTokensDetails=[{'modality': 'TEXT', 'tokenCount': 57}])

Docs for Prompting caching: https://platform.openai.com/docs/guides/prompt-caching

Requirements
Caching is available for prompts containing 1024 tokens or more, with cache hits occurring in increments of 128 tokens. Therefore, the number of cached tokens in a request will always fall within the following sequence: 1024, 1152, 1280, 1408, and so on, depending on the prompt's length.
All requests, including those with fewer than 1024 tokens, will display a cached_tokens field of the usage.prompt_tokens_details Response object or Chat object indicating how many of the prompt tokens were a cache hit. For requests under 1024 tokens, cached_tokens will be zero.

"usage":{
 "prompt_tokens": 2006,
 "completion_tokens": 300,
 "total_tokens": 2306,
 "prompt_tokens_details":{
 "cached_tokens": 1920
 },
 "completion_tokens_details":{
 "reasoning_tokens": 0,
 "accepted_prediction_tokens": 0,
 "rejected_prediction_tokens": 0
 }
}

To Reproduce

My openai version is 1.99.6:
completion = client.chat.completions.create( model="gemini-2.5-pro", messages=messages, tools=available_tools, tool_choice="auto", max_tokens=20000, extra_headers={"X-TT-LOGID": ""} )

Code snippets

OS

Linux

Python version

Python 3.11.2

Library version

openai v1.99.6

[NikkiAung]

Hello @RobertCraigie ,

My team, @m1r9n1 and I are currently working on this issue through CodeDay Labs. CodeDay is open-source organization, where they love to assign open source issue to students to get experience in tech industry and we’re excited to contribute!

After debugging, I came to realize this is chat completion object JSON issue, referring to this The chat completion object document

As @tommy-fcy mentions, Caching should be available for prompts containing 1024 tokens or more. After making some investigation below

• OpenAI Lib with OpenAI Api Key (Any OpenAI Model) works fine
• OpenAI Lib with Gemini Api Key using Gemini Model has some changes depending on both prompt_tokens and gemini model

Let's say for example
For gemini-2.5-flash with prompt_tokens : 1470 , which is >= 1024 tokens, produced cached_tokens

FIRST REQUEST (building cache) ...

{
 "model": "gemini-2.5-flash",
 "object": "chat.completion",
 "service_tier": None,
 "system_fingerprint": None,

 "usage":{
 "completion_tokens": 744,
 "prompt_tokens": 1469,
 "total_tokens": 2496,

 "completion_tokens_details":{
 "accepted_prediction_tokens": None,
 "audio_tokens": None,
 "reasoning_tokens": None,
 "rejected_prediction_tokens": None
 },

 "prompt_tokens_details": None
 }
}

SECOND REQUEST (should use cache) ...

{
 "model": "gemini-2.5-flash",
 "object": "chat.completion",
 "service_tier": None,
 "system_fingerprint": None,

 "usage":{
 "completion_tokens": 825,
 "prompt_tokens": 1469,
 "total_tokens": 3299,

 "completion_tokens_details":{
 "accepted_prediction_tokens": None,
 "audio_tokens": None,
 "reasoning_tokens": None,
 "rejected_prediction_tokens": None
 },

 "prompt_tokens_details":{
 "audio_tokens": None,
 "cached_tokens": 1006
 }
 }
}

But for gemini-2.5-pro, the cached_tokens is only available with prompt_tokens, 5819, which is >= 4096 tokens, meaning that gemini-2.5-pro can only provide cached_tokens with minimum 4096 tokens.

FIRST REQUEST (building cache) ...

{
 "model": "gemini-2.5-pro",
 "object": "chat.completion",
 "service_tier": None,
 "system_fingerprint": None,

 "usage":{
 "completion_tokens": 0,
 "prompt_tokens": 5819,
 "total_tokens": 5938,

 "completion_tokens_details":{
 "accepted_prediction_tokens": None,
 "audio_tokens": None,
 "reasoning_tokens": None,
 "rejected_prediction_tokens": None
 },

 "prompt_tokens_details": None
 }
}

SECOND REQUEST (should use cache) ...

{
 "model": "gemini-2.5-pro",
 "object": "chat.completion",
 "service_tier": None,
 "system_fingerprint": None,

 "usage":{
 "completion_tokens": 414,
 "prompt_tokens": 5819,
 "total_tokens": 7817,

 "completion_tokens_details":{
 "accepted_prediction_tokens": None,
 "audio_tokens": None,
 "reasoning_tokens": None,
 "rejected_prediction_tokens": None
 },

 "prompt_tokens_details":{
 "audio_tokens": None,
 "cached_tokens": 4077
 }
 }
}

How do I test this?

Referring to contributing guideline, I set my local env set up using both @tommy-fcy ver and current ver, the issue is persisted like @tommy-fcy mentioned.

tommy-fcy ver
# Openai version is 1.99.6:
# Python version is 3.11.2
# Library version is v1.99.6
# OS Linux

Current ver tested by me
# Openai version is 2.9.0:
# Python version is 3.12.9
# Library version is latest ver
# OS Mac

After setting up, I try to use local SDK and try debugging using print from src/openai/_response.py (line 265 - 266) to generate RAW Data from gemini end point before parsing with OpenAI lib Completion type from src/openai/types/completion.py like below

data = response.json()
import json
print("=" * 80)
print("RAW GEMINI RESPONSE (before OpenAI parsing)")
print("=" * 80)
print(json.dumps(data, indent=2))
print("=" * 80)

# RAW GEMINI RESPONSE (before OpenAI parsing)

FIRST REQUEST (building cache) ...
{
 "choices": [
{
 "finish_reason": "length",
 "index": 0,
 "message":{
 "content": "Of course. Here is a detailed ...",
 "role": "assistant"
 }
 }
 ],
 "created": 1765341671,
 "id": "5_k4ae_jNpKJ-sAPxOWc6QY",
 "model": "gemini-2.5-pro",
 "object": "chat.completion",
 "usage":{
 "completion_tokens": 380,
 "prompt_tokens": 5819,
 "total_tokens": 7817
 }
}

SECOND REQUEST (should use cache) ...
{
 "choices": [
{
 "finish_reason": "length",
 "index": 0,
 "message":{
 "content": "Of course. Here is a detailed ...",
 "role": "assistant"
 }
 }
 ],
 "created": 1765341693,
 "id": "_fk4aeHZEoP3jMcPndSmiQs",
 "model": "gemini-2.5-pro",
 "object": "chat.completion",
 "usage":{
 "completion_tokens": 414,
 "prompt_tokens": 5819,
 "prompt_tokens_details":{
 "cached_tokens": 4077
 },
 "total_tokens": 7817
 }
}

My plan would be making 'prompt_tokens_details':{'audio_tokens': None, 'cached_tokens': None} instead of 'prompt_tokens_details': None so that another user like @tommy-fcy won't get confused if cached_tokens isn't included inside prompt_tokens_details from Gemini Model Raw Data end point

[NikkiAung]

@RobertCraigie, I'd be really appreciate for your code review and the next step. :)