Should You Build or Buy AI? Why It’s Really Two Decisions
What running your own AI model really costs, and which part of the AI stack is worth owning instead.

Alec Merle, Senior Forward Deployed Engineer, Foxtrot
In brief
- Build or buy AI is two decisions: where the model runs, and who owns the layer that connects it to your business.
- By our estimate, running one of the strongest open-weight models on your own servers costs roughly $1 million a year in production, most of it fixed, while commercial tools cost a small fraction of that for a typical engineering team.
- For most work, buy access to frontier models and build the control layer yourself.
- Keep models inside your own boundary for the workloads where data control comes first.
Sooner or later, someone on the board asks whether the company controls its AI. The question tends to arrive after a run of AI contracts that aged badly, and around the time someone in engineering says the team could build the next tool instead of buying it.
Both instincts show up in the data. In a Dataiku and Harris Poll survey of 600 enterprise CIOs, 74% said they regret at least one major AI vendor or platform selection made in the past 18 months, and 40% said vendor lock-in or LLM pricing changes are having a major or devastating impact on their AI budget. Meanwhile, McKinsey's State of AI 2026 found that 32% of organizations have decided against buying one or more software products or features because they could build them internally, and nearly half of AI high performers had done the same.
Build or buy sounds like one question, but it folds two decisions together, and they have different answers. For most work, buy access to frontier models, and own the layer that connects them to your business.
The two decisions inside build or buy
The first decision is where the model comes from. You can rent access to a commercial model, or you can run one yourself on hardware you control, which is what people usually mean by self-hosting. There's also a middle path: inference providers that serve open-weight models by the token, which removes the hardware and the infrastructure team but still leaves you on a model that trails the frontier.
The second decision is who owns everything between that model and your business. We call that the control layer. The control layer is the part of an AI system that holds the context agents reason from, the permissions that govern what they can do, the record of what they did, and the routing that decides which model handles which work.
The two are easy to blur, but they're independent: a team can build its own agents and still run them on a model it rents. We explored the second decision in an earlier piece on what you'd have to rebuild if you changed AI providers tomorrow. To test the first, we priced its most complete answer: owning the model outright.
What does it cost to self-host an AI model?
By our estimate, running one of the strongest open-weight models yourself, in production, costs roughly $1 million a year, depending on the provider, and most of that cost is fixed. Commercial AI tools work the other way: you pay per person and per use, so the bill grows and shrinks with the team. The open-weight models themselves are free to download, so the cost is in the setup, mainly two GPU servers, one held as a backup, and the two engineers it takes to keep them running.
To see how far apart the two are, we priced the work our own engineers know best: a team of 10 to 20 engineers using AI coding agents. Coding agents are heavy to run. Anthropic tells enterprise admins to budget more for a coding seat than a chat seat, since one debugging session can consume more than a day of chat. A heavy workload produces a large commercial bill, so among the tools people use interactively, coding agents give ownership one of its best chances. If renting comes out ahead here, it's likely to come out ahead for lighter interactive work too.
Ownership means paying directly for two things a subscription bundles in, starting with hardware. The strongest open-weight models for agentic coding, meaning models anyone can download and run, need a server with eight high-end NVIDIA GPUs, which rents for a few hundred thousand dollars a year depending on provider and commitment. Production needs a second server, because when the only one goes down, every engineer goes down with it. Buying the servers outright can lower the hardware cost over their useful life, though power, space and replacement costs add some of it back, and the people cost below doesn't change.
The second cost is people. Someone has to own on-call coverage, model upgrades, inference tuning, patching and capacity planning, which in practice means at least two specialized infrastructure engineers. The week a stronger open model is released, someone also has to decide whether it's worth pulling those two engineers off everything else to test and redeploy it. Hardware and people together come to roughly $1 million a year.
The coding agents powering all that building are themselves rented, and the comparison shows why. Anthropic reports that Claude Code averages $150 to $250 per developer per month across enterprise deployments, and OpenAI puts Codex at about $100 to $200. For a team of 10 to 20 engineers, that comes to tens of thousands of dollars a year, including enterprise seat fees, a small fraction of the cost of running your own. Even if every engineer were a heavy user, spending at a level Anthropic says 90% of users stay below, self-hosting would still cost several times more.
The people cost matters more than it looks. Even if the servers cost nothing, two infrastructure engineers would cost several times what commercial coding agents cost a team this size. Smaller open models change the cost side, because they run on a fraction of the hardware and need far less looking after. What they give up is capability: they trail the commercial leaders by even more than the strongest open models we priced, so for this work the trade-off is about results more than cost.
Does team size change the answer?
Team size changes how wide the gap is, and the small end surprises people most. Production needs the same two servers and the same two people whether five engineers rely on them or twenty, so a small team carries the whole fixed cost while its commercial bill stays modest, which is why the gap is widest for the smallest teams.
Large organizations can spread a specialized infrastructure team across many more people, which brings the cost per person down. Hardware doesn't spread the same way, because GPU capacity has to match the number of agents running at the same moment, so a bigger engineering organization needs more servers. At that scale the cost comparison gets closer and is worth running on your own usage, but the capability gap described below doesn't shrink with headcount.
Utilization matters at every size. A reserved server bills for every hour of the month, and a team that works weekdays keeps it busy for less than a third of those hours, while commercial usage rises and falls with the work people actually do. An independent cost model from IntuitionLabs points the same way. It finds that self-hosted cost per token falls as hardware utilization rises, and that no single break-even point applies across workloads.
On cost alone, self-hosting makes sense when a workload keeps hardware busy around the clock, like a high-volume processing pipeline. Engineers who work in bursts during the business day rarely fit that description.
The opportunity cost of complete data control
Complete data control is a real requirement for some workloads, and it sounds appealing for all of them. Applied to every workload by default, it carries a cost that never shows up on an invoice, which is the capability you give up.
New capability reaches the leading closed models first. Epoch AI found that since January 2026, the most capable open-weight models have trailed frontier closed models by an average of four months.
The gap is clearest in coding agents working through long, multistep engineering tasks. When we checked the Artificial Analysis Coding Agent Index in October, open models running inside coding agents still trailed the leading commercial agents by a clear margin. On older benchmarks that frontier models have largely saturated, open models look close to equal, which is where most claims that open models have caught up come from. Engineers would likely keep reaching for commercial tools on their hardest problems, so a self-hosted stack built for them would add to the commercial bill instead of replacing it.
The strictest data terms can also put the newest models out of reach. Some of the newest frontier models retain prompts and responses for up to 30 days, so an organization that insists on zero retention for everything can't use them without a separate arrangement with the provider.
That leaves one question to settle for each workload: what comes first? Where data control does, as in sovereign or air-gapped work, a model running inside your own boundary is the right call, and the capability trade is worth making. Where the best possible result comes first, the frontier labs are usually the stronger partner, and negotiated data terms handle much of the remaining risk. As we argued in our piece on changing AI providers, control should scale with the stakes of the work.
The layer worth building
Most of what leaders mean when they say they want to own their AI doesn't require owning a GPU.
They usually mean some combination of four things: confidence about where their data goes and whether it's retained or used for training, the freedom to change models without rebuilding, a record of what AI did and why, and firm limits on what an agent can see and do. Owning the hardware addresses the first of these, and the other three have to be built no matter where the model runs.
Contracts now cover much of the first. The major providers' business plans don't use customer data for training by default, and retention terms can be negotiated workload by workload.
The other three live in the control layer, and how much of it you need to build depends on what the agents touch. For coding agents, much of the layer is generic, and commercial coding tools now ship a good part of it, from permission settings to usage reporting. That's one reason buying the whole coding stack is a sensible default. Agents that act on your operations are different. The context they reason from is a working model of your business, and the permissions and approvals that govern them encode who in your organization is allowed to decide what. Nobody can sell you that ready-made, and it decides whether a model you rent produces work you can trust.
THREE WAYS TO OWN IT
| Approach | What you own | What a provider still runs | How the cost behaves |
|---|---|---|---|
| Buy everything, under strong data terms | Data terms and access policy | The agent tooling, the model and the GPUs | Scales with use; tens of thousands of dollars a year for 10 to 20 engineers |
| Buy the model, build the layer around it | The control layer: context, permissions, logs and model routing | The model and the GPUs | The usage above, plus building and maintaining the layer |
| Own the whole stack on rented GPUs | Everything above, plus the model and inference | Bare GPUs only | Mostly fixed; roughly $1 million a year in production for two servers and two engineers, plus building and maintaining the layer |
When the control layer belongs to you, changing models becomes a planned project, and what you encode about how your business works and makes decisions grows more valuable the longer it runs, through every model you swap in beneath it. That points to a practical rule: own the layer that compounds, and decide workload by workload where the model should run.
Where Foxtrot builds that layer
Foxtrot's Foundry and AIP engineers, more than 70 of them with over 200 combined years of Palantir experience, build the control layer with clients, and what we build is yours to own and operate. It comes in two parts. The Semantic Layer turns scattered, sometimes contradictory data into one clear model of how the business works, so agents reason from the right context. The Judgment Harness governs what those agents are allowed to do, with permissions, approvals, runtime limits and human escalation wired in.
In practice | Construction technology
Cost is part of the work from the start. For one construction technology company, our team cut compute costs by 90% before the broader architecture was fully deployed.
On Palantir Foundry and AIP, where much of our work happens, the ontology, Foundry's working model of an organization's data and decisions, is the foundation we build that layer on, and the platform supports model choice. Enterprises can connect their own models alongside the ones Palantir provides, and Palantir has launched an engine for deploying NVIDIA Nemotron open models in sovereign environments. That gives clients a ready path for the sovereign and air-gapped workloads described above, and it keeps the decision about where a model runs reversible as the economics change.
When self-hosting AI makes sense
Full ownership becomes a stronger option when any of these happen:
- An open-weight model matches the commercial frontier on an independent agentic benchmark for the work you actually do.
- The strongest open models run on half the hardware they need today, which roughly halves the server bill.
- GPU rental prices fall and stay down. This fall they've been moving the other way, with several GPU clouds raising H200 prices in September and October.
- A contract or regulator requires models to run inside a specific boundary, in which case the requirement settles it.
With the control layer in place, acting on any of these means ordering hardware and moving a workload, a much smaller decision than starting over.
So, should you build or buy AI?
For most work, buy access to frontier models and build the control layer yourself. Where complete data control comes first, run the model inside your own boundary and accept the capability trade. The control layer is the part worth owning in every case, because that's where your advantage accumulates and what lets you make the model decision one workload at a time.
When the board asks whether the company controls its AI, the strongest answer describes what your agents are allowed to do and how quickly you could move to a different model if you had to. Both answers live in the control layer. That's also the practical response to the 40% of CIOs in the Dataiku survey who say lock-in or LLM pricing changes are hurting their AI budget: when you own the control layer, a price change or a better model becomes a migration you can plan.
Next step
See where your organization stands
Foxtrot's free AI readiness assessment, the Agentic Enterprise Score, gives you a baseline before you decide what to build and what to buy.
Frequently asked questions
Does self-hosting AI mean building it?
No. Self-hosting means running a model on infrastructure you control. Building means your team makes the agents, workflows and integrations, which today usually run on a commercial model you rent. The two are separate decisions, and most enterprises should build far more than they self-host.
Is it cheaper to self-host an LLM than to use commercial AI tools?
Not for most teams. Self-hosting is mostly a fixed cost: Foxtrot estimates roughly $1 million a year for a production setup of two eight-GPU servers and two infrastructure engineers, while a team of 10 to 20 engineers on commercial coding agents spends tens of thousands of dollars a year. The gap is widest for small teams, and self-hosting pays mainly when hardware stays busy around the clock.
When does self-hosting AI make sense?
When a workload keeps hardware busy around the clock, such as a high-volume processing pipeline, or when data can't leave a boundary, as in sovereign or air-gapped environments. A contract or regulation that requires models to run inside a specific boundary settles the question on its own.
Is data privacy worth giving up frontier AI capability?
For some workloads, yes. Sovereign and air-gapped work needs models that run inside the organization's own boundary, and the capability trade is worth it there. For most other work, the capability cost is real: since January 2026, the most capable open-weight models have trailed frontier closed models by an average of four months, according to Epoch AI.
What should an enterprise build and own?
The control layer between its data and its models: the context agents reason from, the permissions and approvals that govern them, the record of what they did, and the routing that decides which model handles which work. That layer is specific to how the business works, so it can't be bought ready-made, and it keeps its value every time the model underneath changes.
Should we train our own AI model?
Rarely from scratch. Epoch AI estimates that training Grok 4 cost about half a billion dollars. Fine-tuning is far cheaper: a lightweight fine-tune of one of the largest open models, three passes over a 100-million-token dataset, costs about $3,000 in compute at Fireworks' published fine-tuning prices. Most of the cost sits in curating the data and proving the tuned model is better, and serving it brings back the hardware costs above.