If You Had to Change Your AI Provider Tomorrow, What Would You Have to Rebuild?

Real control means institutional sovereignty: keeping the context, workflows and governance that make your AI valuable, even when the technology underneath it changes.

By Alec Merle
Senior Forward Deployed Engineer, Foxtrot
12 min read

Most companies don’t think about AI sovereignty until something changes. Often, the first question is about data.

• Where does our information go?
• Who can access it?
• How long is it retained?
• What happens when proprietary data becomes part of an AI workflow?

Those questions matter. They’re also only one piece of the sovereignty puzzle. Once AI starts creating value inside a business, control diffuses across the system. It reaches the data the AI uses, the logic behind decisions, the models doing the work, the compute underneath them and the rules that determine what the system is allowed to do. That’s why we think the idea of institutional sovereignty is useful.

The market often talks about model sovereignty or AI sovereignty. We think the more practical question is who keeps the decision rights. Who decides how company data is used? How easily can you change the model or provider when requirements change? Where does the model run? Who owns the business logic? What happens when the company needs a different architecture? Each of those is a decision about the AI system itself. The enterprise should be the one making them.

The model is the visible part. The bigger stakes often sit underneath it.

WHERE CONTROL SITS IN A PRODUCTION AI SYSTEM

Model THE VISIBLE PART	Produces responses, predictions and tool calls. The most visible layer, and the layer the architecture should make easiest to swap.
Business logic	The rules, decision criteria and workflow definitions that encode how the company operates.
Governance	Permissions, evaluations, audit trails and human-review rules that set what the system may do.
Compute	Where the model runs: a provider’s cloud, a private cloud or on-premise hardware.
Data	The company information the system can reach, and the boundaries that limit its access.

Each layer carries its own decision rights, and many of the harder dependencies sit below the model.

Our perspective comes from our Forward Deployed Engineers, embedded in real customer workflows, where they see firsthand how enterprise knowledge, business logic, permissions and evaluations become connected to the model layer. Foxtrot’s sovereignty work is built around preserving those enterprise-owned assets even when the underlying model changes. That gives companies more freedom to evaluate, switch or combine models as performance, security, latency, regulatory requirements and cost change. We call that model agility. With the best available model changing from one release to the next, it’s one of the most practical tests of whether a company actually has sovereignty.

Sovereignty is bigger than the model

Swapping your LLM may sound simple. In practice, the model is only one piece of a production AI system. The workflow around it may contain business rules, enterprise context, evaluations, permissions, tool access and connections to other systems. Those elements are what turn a capable general model into something that can actually work inside a particular company.

Over time, they can become deeply embedded.

Portability depends on keeping the enterprise-owned layers around the model as durable and separable as possible. That can include a shared data layer with single sources of truth, enterprise-owned knowledge and workflow definitions, clear interfaces to models and tools, permissions enforced independently of the model, reusable evaluation datasets and execution records the company can retain or export.

That doesn’t make changing models plug-and-play. A new model still needs to be revalidated for accuracy, tool use, latency, behavior and the specific requirements of the workflow, and some changes may require engineering work. The advantage is that the enterprise doesn’t have to rebuild its business context, workflow definitions, permissions and evaluation criteria from scratch.

That’s where sovereignty becomes more than portability.

For an enterprise, sovereignty means retaining enough control over the system that a technology decision made today doesn’t quietly determine how the business has to operate years from now. The company should be able to preserve the knowledge and operating logic it has built while continuing to make new choices about the technology underneath it. That’s the difference between having access to another model and having meaningful freedom to use one.

Build for model agility

The model leaderboard doesn’t sit still. One week the strongest model comes from one provider, and the next week another provider’s release overtakes it. The differences aren’t binary, either. One model might score 90% on a coding benchmark and another 88%, while the second is clearly stronger at writing.

A company that can only use one model has to accept every tradeoff that model makes.

Benchmarks alone won’t tell you which model fits a workflow. Sometimes the only way to know is to get hands-on with a model the day it’s released, before any consensus has formed about what it’s good at. That takes people who understand which models are good at what, and an architecture that lets the company act on their judgment quickly.

That architecture is the harness around the model: how context is fed in, how it’s stored and how it relates to the model. Models are probabilistic and won’t return the same answer every time, so the systems around them have to create determinism. Hooks, tool calls and structured outputs should return data in the same format no matter which model sits behind them or which provider built it. That consistency is what makes a model hot-swappable.

Switching will always carry some cost. The closer a company gets to minimizing it, the faster it can adopt a better model when one appears, and the less any single provider shapes how the business operates.

Privacy is where the conversation usually starts

Enterprises are putting increasingly sensitive information into AI workflows. That may include financial information, customer records, proprietary research, operational data, intellectual property or knowledge that has historically lived only inside the company. It makes sense that privacy gets attention first.

The trust question underneath it isn’t new. Enterprises have been answering it with cloud providers for years, placing some trust in the provider in exchange for less infrastructure to configure and run themselves. What’s different is that AI is a newer field, so that trust is harder to gauge, and many organizations are moving fast enough to skip the question entirely.

The right protections will depend on the product, provider, enterprise agreement and deployment. Some workloads may be well served by managed models with strong enterprise privacy, security and retention controls. Other workflows carry different stakes. A highly sensitive transaction, a regulated process or a mission-critical operation may require tighter boundaries around where data moves, where inference happens and who ultimately controls the infrastructure.

Deciding that a provider can be trusted with some data but not the rest is only half the problem. Enforcing it is harder. A local model can screen requests before they leave, or a routing layer can decide which data goes to which model, but model-based screening and routing are not fully deterministic. Deterministic enforcement can mean inspecting outbound requests for restricted content, which adds more overhead than most organizations expect. And once employees have an open conversational interface, what goes into the model is up to their discretion, because it’s free text. That’s why the starting point has to be least privilege: decide what each model and interface is trusted with before anyone starts typing.

The sovereignty question is therefore bigger than whether a provider can be trusted. It’s about deciding which rights the enterprise needs to keep for the work in front of it.

Protect how your company works

A general-purpose model doesn’t arrive knowing how your business makes decisions. It doesn’t know which exception an experienced operator recognizes immediately. It doesn’t know why one signal matters more than another, which unwritten rule changes a decision or how people resolve the cases that never made it into the process manual.

That knowledge already exists inside the enterprise. Some of it lives in structured data. Some of it is buried in documents, systems and conversations. A lot of it still lives in people’s brains. The people who carry that knowledge are often the least likely to notice it. After years of working under the same constraints and assumptions, tribal knowledge stops feeling like knowledge and starts feeling like the way things are done. It often takes someone new to the organization to see it.

A model starts where a new hire does, without any of that tribal context, and it can’t absorb it informally. It won’t pick up the unspoken cues around a project or what was decided in meetings nobody transcribed. If it isn’t written down, an agent can’t use it.

This is where forward deployed engineering becomes particularly important. An FDE is the unbiased person looking over your shoulder. Embedded in the workflow, our FDEs work with operators to uncover the edge cases, decision rules and operating patterns that make the business work in practice, then translate them into structures the AI system can actually use.

In an ontology-led architecture, that can mean turning institutional knowledge into business objects and relationships, workflow definitions, permissions and evaluation criteria. The goal is to make the knowledge that drives the business explicit enough to use, govern and evaluate in production. Writing it down is only the first step. The architecture also has to make that knowledge discoverable. And because an agent works within a limited context window, the system has to surface the most valuable information for the problem at hand, so the answer holds up against what a senior operator with years of context would say.

There’s another layer to this that we think will matter more over time. A company can begin capturing the decision loop itself. What information did someone consider? What decision did they make? What action followed? What happened as a result?

That operating history has the potential to become an AI asset in its own right. Over time, capturing those loops can make institutional knowledge more durable and reusable.

Once a company has done the hard work of encoding that context and logic, it shouldn’t have to rediscover how its own business works every time the technology changes.

Sovereignty isn’t a switch. It’s a gradient of decision rights.

This is where the conversation often becomes too binary. A company is either sovereign or it isn’t. A model is either self-hosted or it isn’t. The data either leaves the environment or it doesn’t. Real enterprise environments are more complicated.

Different workloads need different levels of control.

CONTROL SCALES WITH THE STAKES

WORKLOAD	APPROPRIATE LEVEL OF CONTROL
Low-risk internal productivity workflow	May be well served by a managed model under strong enterprise controls
Workflow touching more sensitive information	May call for tighter boundaries around data, routing or infrastructure
Highly regulated or mission-critical process	May justify customer-controlled infrastructure, self-hosted models or other forms of structural separation

The right level of control rises with the sensitivity of the data and the criticality of the process.

We think of this as a gradient, with the appropriate level of control depending on the sensitivity of the data and the criticality of the process. That’s why we find decision rights more useful than a simple yes-or-no definition of sovereignty. As the stakes rise, an enterprise may choose to retain more authority over the data, logic, model, compute, governance and outcomes surrounding the workflow.

In practice, one way the gradient is enforced is through access. A model should only be able to reach the subset of data its workflow is authorized to use, with markings, guardrails and governance holding that boundary in place. It works the way security clearances do on a classified network: someone cleared for Secret sees Secret information and nothing above it. The gradient is real, and each level on it can function as a gated knowledge silo.

The goal isn’t maximum control everywhere. It’s the right control for the work.

More control comes with more responsibility

Self-hosting sits toward one end of that spectrum. Running an open-weight or custom model inside infrastructure the enterprise controls can provide tighter boundaries around data, inference and model operations. Foxtrot’s work includes these patterns for situations where private cloud, customer-controlled infrastructure or on-premise deployment may be appropriate.

That level of control is powerful, but it isn’t free. The enterprise takes on more responsibility for model serving, infrastructure, performance, security and compute. Self-hosting models that approach frontier performance requires significant hardware and specialized technical expertise.

In practice, specialized infrastructure expertise can be a meaningful constraint for companies pursuing self-hosted approaches outside established platforms.

There’s also a ceiling on what self-hosting can reach today. An enterprise could have unlimited budget, GPUs, people and harnesses, and some of the most capable proprietary frontier models would still only be available as hosted services. Open-weight models keep getting closer, so that may change. For now, even if the best weights were released, very few organizations could afford the compute to run them, and the most controlled end of the spectrum can mean accepting a capability gap.

So the question shouldn’t be whether self-hosting is inherently better. The question is whether the additional control is worth the added responsibility (and any capability tradeoff) for that particular workload. For some, it will be. For others, a managed model with the right enterprise controls will be the smarter architecture.

Sovereignty is valuable because the company retains the ability to make that choice.

Governance becomes part of sovereignty when AI can act

The control question gets more serious as AI moves from producing information to taking action. An agent may query sensitive systems, create records, initiate workflows, recommend decisions or act on someone’s behalf.

Now the enterprise has to control much more than the model response. Who owns the agent? What can it access? Which actions is it authorized to take? How is its performance evaluated? When does someone need to step in? Can the organization reconstruct what happened later?

Foxtrot’s governance work covers identity and ownership, permissions, authorized actions, data access, policy enforcement, provenance, auditability, evaluation, escalation and security controls.

These controls matter in every enterprise. They become particularly important in regulated and high-stakes environments, where the organization may need to explain what happened, who had authority and which policies were applied.

There’s a sovereignty implication here too.

Governance shouldn’t have to be reinvented every time the model changes.

Take a customer asking an AI agent for a refund. How much any model is allowed to refund should be capped by deterministic business logic that sits outside the model and is accountable for the outcome. Then it doesn’t matter which model handles the conversation, because the rule holds either way.

If changing one component of the AI stack also means reconstructing the permissions, evaluations and human-review rules around the workflow, the enterprise has created another form of dependency. Durable governance gives the company more freedom to change the technology without giving up control of how the work gets done.

Sovereignty is easier to promise than it is to engineer

The concept sounds clean. Real enterprise environments rarely are. Many companies don’t yet have their data or operating knowledge organized well enough to make that context durable. Legacy systems create another challenge. Some were built long before modern APIs or newer agent connectivity standards existed, which makes connecting AI safely to existing data and tools far more complicated.

Self-hosted environments require specialized infrastructure expertise. Governance has to span systems. Evaluations need to remain meaningful as models change. Data boundaries have to be understood rather than assumed.

In practice, data readiness, legacy integration and specialized infrastructure expertise are among the biggest barriers to sovereign AI architectures.

The practical goal is to make every part of a workflow that can be deterministic actually deterministic, and leave the model to handle the rest. Getting close is hard when systems and data change daily, but it’s achievable. It takes sustained time and effort, and forward deployed engineers close enough to the business to keep up as it changes.

That’s why we think sovereignty is better understood as something companies build toward than something they simply turn on.

The architecture has to earn the promise.

Sovereignty gets tested when something changes

An AI environment can feel flexible when everything keeps working exactly as planned. The revealing moment comes when the business needs something different. A new model performs better. The economics change. A workload becomes subject to stricter regulation. Sensitive information needs to remain inside a controlled environment. An agent is given more autonomy.

That’s when earlier architecture decisions become visible.

Those questions tell you a lot about where the real decision rights sit.

The durable asset is the company’s ability to choose

A more durable investment is protecting the parts of the system that belong to the enterprise. The context should remain usable. Governance should remain enforceable. Business logic shouldn’t disappear because a model changes. The architecture should leave room for different choices when performance, security, regulation or economics require them, which is what model agility looks like in practice.

That’s what institutional sovereignty means to us.

It’s the ability to keep making decisions about your own business as the technology around it changes.

Frequently asked questions

What is enterprise AI sovereignty?

Enterprise AI sovereignty is the ability to retain control over the data, business logic, models, compute and governance that determine how AI operates inside an organization, while preserving enough flexibility to change technology as requirements evolve.

What is institutional sovereignty?

Institutional sovereignty looks beyond the model itself. It focuses on whether the enterprise keeps the decision rights over the knowledge, logic, infrastructure and outcomes that shape how AI works inside the business.

Is AI sovereignty the same as data sovereignty?

No. Data sovereignty focuses primarily on where data resides, who can access it and which legal or jurisdictional requirements apply. AI sovereignty also considers model choice, compute, workflow logic, governance and the enterprise’s ability to change those elements over time.

Does AI sovereignty require self-hosting?

No. Self-hosting can provide a high level of technical control and may be appropriate for sensitive or critical workloads. Other workflows may be well served by managed models with strong enterprise controls. The appropriate level of sovereignty depends on the work.

What are decision rights in AI?

Decision rights define who has authority over choices such as where data can move, which models can be used, where compute runs, how business logic is implemented and what actions AI systems are allowed to take.

What is model agility in enterprise AI?

Model agility is the ability to evaluate, switch or combine models as capabilities and requirements change without rebuilding the enterprise context, workflows and controls around them from scratch. It is one practical measure of AI sovereignty because it preserves the enterprise’s ability to choose as the model layer evolves.

Why does AI sovereignty matter in regulated environments?

Regulated and high-stakes workflows often carry stricter requirements around data access, permissions, auditability, human review and deployment. A sovereign architecture gives enterprises more control over how those requirements are met as AI technologies change.