Return inference cost in cost_in_usd_ticks
I just noticed that xAI returns the actual billed inference cost in the response JSON:
"usage": {
"prompt_tokens": 151,
"completion_tokens": 4,
"total_tokens": 749,
"prompt_tokens_details": {
"text_tokens": 151,
"audio_tokens": 0,
"image_tokens": 0,
"cached_tokens": 128
},
"completion_tokens_details": {
"reasoning_tokens": 594,
"audio_tokens": 0,
"accepted_prediction_tokens": 0,
"rejected_prediction_tokens": 0
},
"num_sources_used": 0,
"cost_in_usd_ticks": 15493500
},
“cost_in_usd_ticks” is the actual inference cost in “ticks” (10^10 ticks per dollar).
I think this would be a useful value for all model providers to return. Right now I have to compute it myself from the returned token counts, but it would be better to get an authoritative answer directly from the provider especially since there can be additional costs like internal tool calls.
Discussion in the ATmosphere