AI & LLM cost
Input vs. output tokens
Most LLM APIs price input tokens, the prompt you send, and output tokens, the response generated, separately. Output usually costs several times more.
Last updated
Definition
Input tokens are everything you send to the model: the prompt, any conversation history, and retrieved context. Output tokens are what the model writes in response. Generation costs more computing than reading. So output tokens are typically priced at a multiple of the input rate, often 3 to 5 times. A workload that writes long responses can cost far more than its prompt size suggests.
This split matters for cost control. Trimming an oversized prompt helps. But a feature that writes verbose, unbounded output can dominate spend even with a lean prompt.
Where it shows up
Input and output token counts appear as two separate fields in the usage object of an API response. Providers publish two separate rates for them on their pricing page. For the same model, the output rate is typically several times the input rate.
What makes it expensive
A team trims an oversized prompt but leaves a feature free to write long, unbounded responses. Those responses can dominate spend even with a lean prompt. A combined token total hides which half is growing. Split it. Then cap the output side, because that side is priced at a multiple of the other.
Related
Token spend moves every day.
Model prices and rate limits change between invoices. CostMon tracks your AI spend daily, so a change shows up before the bill does.