AI & LLM cost
Fine-tuning vs. inference cost
Fine-tuning cost is a one-time (or periodic) training expense to specialize a model; inference cost is the ongoing, usage-based cost of running it.
Last updated
Definition
Fine-tuning takes a base model and further trains it on your own examples, and its cost is driven by training data volume and compute time. It's paid once per training run, not per use. Inference is what happens every time the resulting model answers a request, priced the same way as any other model call: per token, per request.
The tradeoff to model against isn't training cost in isolation. It's whether a fine-tuned model's inference cost (often lower, since a smaller specialized model can sometimes replace a larger general one) pays back the upfront training investment across the request volume you expect.
Where it shows up
Fine-tuning cost appears as a one-time training job charge on the provider's invoice, priced by training data volume and compute time for that single run, while inference cost for the resulting model appears afterward as ordinary per-token usage on every call it goes on to serve.
What makes it expensive
Training cost is the easy number to model, and on its own it answers nothing. What matters is whether the fine-tuned model's inference cost pays that training back across the request volume the workload will really see. Put its per-request cost next to the general model it replaced, then divide the training bill by the difference.
Related
Token spend moves every day.
Model prices and rate limits change between invoices. CostMon tracks your AI spend daily, so a change shows up before the bill does.