AI & LLM cost
Input vs. output tokens
Most LLM APIs price input tokens (the prompt you send) and output tokens (the response generated) separately, with output usually priced several times higher.
Last updated
Definition
Input tokens are everything sent to the model: the prompt, any conversation history, retrieved context. Output tokens are what the model generates in response. Because generation is more computationally expensive than reading, output tokens are typically priced at a multiple of the input rate (often 3–5x), which means a workload that generates long responses can cost far more than its prompt size alone would suggest.
This split matters for cost control: trimming an oversized prompt helps, but a feature that generates verbose, unbounded output can dominate spend even with a lean prompt.
Where it shows up
Input and output token counts appear as two separate fields in an API response's usage object, and providers publish two separate rates for them on their own pricing page, with the output rate typically set at several times the input rate for the same model.
What makes it expensive
This gets expensive fast: trimming an oversized prompt while leaving a feature free to generate long, unbounded responses, which can dominate spend even with a lean prompt. A combined token total hides which half is growing. Split it, then cap the output side, because that is the side priced at a multiple of the other.
Related
Token spend moves every day.
Model prices and rate limits change between invoices. CostMon tracks your AI spend daily, so a change shows up before the bill does.