Start with the API. Move to fine-tuning only when you have documented evidence that prompting has plateaued below your quality bar. Train from scratch essentially never, unless the model itself is your product.
That's the short version, and it's right about 90% of the time. Here's the reasoning, and the 10% where it isn't.
The comparison
| Buy (API) | Fine-tune | Build from scratch | |
|---|---|---|---|
| Time to first version | Days | Weeks | Months to years |
| Upfront cost | Near zero | Moderate | Very high |
| Cost per call at scale | Highest | Lower | Lowest |
| Quality on general tasks | Excellent | Excellent | Depends entirely on you |
| Quality on narrow, unusual tasks | Variable | Best | Best |
| Switching cost later | Low | Moderate | Enormous |
| Team required | Product engineers | ML-literate engineers | ML research team |
| Improves automatically | Yes, vendor upgrades | No, you re-tune | No |
That last row decides more cases than any other. When you buy, someone else's research team improves your product while you sleep. When you fine-tune, you inherit the maintenance.
The four questions
1. Has prompting actually plateaued?
Most teams that "need" fine-tuning haven't seriously tried structured prompting, retrieval, or decomposing the task into smaller steps. Fine-tuning to fix a prompt problem is expensive and it doesn't work.
Requirement before you tune: a written evaluation set, a measured baseline, and documented attempts that fell short.
2. Is your data genuinely differentiated?
Fine-tuning pays off when you have proprietary data encoding knowledge the base model doesn't have — your domain's vocabulary, your customers' specific patterns, judgments your team makes that aren't written anywhere public.
If your data is "customer support tickets, like everyone else's," the base model has seen that shape of problem. Retrieval will serve you better than tuning.
3. What are your latency and cost curves at 100x?
An API call that costs a fraction of a cent is irrelevant at 10,000 calls a month and material at 10 million. Run the arithmetic at the volume you're actually planning for, not today's.
This is the legitimate case for fine-tuning a smaller model: not better quality, but comparable quality at meaningfully lower cost per call.
4. Is the model your product, or inside your product?
If your company's value proposition is the model — you're selling AI capability itself — training may be your moat. If the model is a component inside a product whose value is elsewhere, training from scratch is an expensive distraction from the thing customers actually pay for.
Almost every company is in the second category, and a surprising number act like they're in the first.
The mistake I see most
Teams pick fine-tuning early because it signals technical seriousness. Then they spend two months on data preparation, ship something marginally better than a well-structured prompt, and can't upgrade when a better base model launches six weeks later.
The most expensive AI decision is the one that locks you out of the next model.
A practical sequence
- Ship with an API and structured prompting. Measure honestly.
- Add retrieval when the failure mode is missing knowledge, not missing skill.
- Fine-tune a smaller model when cost or latency at scale becomes the binding constraint — and you have the evaluation set to prove quality holds.
- Revisit quarterly. The correct answer changes as models improve.
Keep your abstraction layer clean enough that steps 1 through 3 are swappable. That flexibility is worth more than any single model choice.
FAQ
Should we build or buy our AI model? Buy first. Use an API with structured prompting until you have measured evidence it can't meet your quality bar, then consider fine-tuning.
When does fine-tuning make sense? When you have genuinely proprietary data, a documented plateau with prompting, and either a cost or latency constraint at scale that a smaller tuned model solves.
Should any company train a model from scratch? Only if the model itself is the product being sold. If it's a component inside a broader product, training is a costly distraction.
What's the biggest risk in committing to a custom model? Lock-in. Base models improve rapidly, and a heavily customized stack can leave you unable to adopt a significantly better model.
I'm Ravi Jadav, Chief Product Officer and Co-Founder at Sunbots Innovations and Co-Founder at Xwits Developers. If you're weighing a build-versus-buy decision, get in touch.