Sovereign AI: your options for EU-hosted language models.

Five reasons the hosting question is not academic

Sovereignty in AI means being able to decide, and prove, where your data is processed, who can access it, under which law, and what happens if a provider changes terms or disappears. For European companies that concern is driven by concrete obligations rather than abstract principle.

• GDPR. Personal data in prompts, documents and outputs needs a legal basis, a processor agreement and, for transfers outside the EU, a valid transfer mechanism. Data-protection officers ask about this before anything else.
• Confidentiality. Contracts, product designs, M&A material and source code are valuable regardless of whether they contain personal data. Provider terms on training, retention and access decide whether they may be sent at all.
• Sector regulation. Financial services, healthcare, public bodies and critical infrastructure face outsourcing and ICT-risk rules that require documented control over providers and locations.
• Procurement and client contracts. Many client agreements and public tenders specify EU processing. A non-compliant AI stack disqualifies you from work you already have.
• Dependency risk. Pricing, model deprecation and export controls are outside your control. Being able to switch providers is worth something even if you never do.

None of this means everything must run in Europe. It means the decision should be made per data class and use case, with the trade-offs understood. Read our AI governance service page for how the decision is documented.

Four ways to run a language model for European work

Options are presented from least to most control. Product names are examples of what exists at the time of writing, not recommendations or claims about specific certifications; check each provider's current terms.

1. US model providers via API, with EU safeguards

OpenAI, Anthropic and Google offer their models through APIs with enterprise terms that typically include no training on customer data, defined retention periods, data-processing agreements and, increasingly, options for EU-based processing. Capability is the strongest, integration is the easiest, and the compliance story rests on contracts plus transfer mechanisms.

2. Hyperscaler EU regions

The same or comparable models are available inside the large clouds: Azure OpenAI Service in EU regions, Amazon Bedrock in EU regions, Google Vertex AI in EU regions. Data stays in the chosen region under the cloud provider's existing contracts, security certifications and identity systems your IT already manages. The providers remain US-headquartered, which matters to some regulators and clients and not to others.

3. European providers and clouds

European model developers such as Mistral and Aleph Alpha, and European infrastructure providers such as IONOS, STACKIT, Open Telekom Cloud and OVHcloud, offer models and hosting under EU jurisdiction. Capability of the top European models is competitive for many business tasks, though the frontier tends to be set elsewhere; the ecosystem of tools and integrations is smaller but growing.

4. Self-hosted open-weight models

Model families with openly available weights, such as Llama, Mistral, Qwen and Gemma, can run on your own servers or on rented EU GPU capacity. Nothing leaves your environment, you control versions and updates, and there is no per-token dependency. The price is operational: hardware or GPU rental, MLOps skills, security patching and the fact that you are now the one responsible for the model's behaviour.

Route by data sensitivity, not by slogan

The mistake we see most often is a company-wide decision in either direction: "everything on the US frontier model" or "nothing leaves our data centre". Both waste money or capability. The workable approach is a classification of data and use cases, and a routing rule for each class.

Most companies end up with two or three of the four options in use at the same time, which is fine as long as the routing is enforced in the architecture rather than left to individual users.

A practical architecture

• An abstraction layer. Applications and agents call models through a gateway that applies the routing rule, logs usage and can swap the provider behind a use case without code changes.
• Classification at the source. Documents and data sources carry a sensitivity label; the gateway refuses to send a highly sensitive document to a route not cleared for it.
• Evaluation per route. Every model behind the gateway is tested against the same evaluation set for each use case, so you know the capability cost of a more sovereign route before choosing it.
• Exit options. Prompts, evaluation sets and fine-tuning data are kept in provider-neutral formats. Switching should be a configuration change and a regression test, not a project.

What to do now

1. Inventory the data classes your first AI use cases will touch and label them.
2. Agree the routing rule per class with your data protection officer and IT security, and write it down.
3. Set up a model gateway, even a simple one, so the rule is enforced and usage is logged.
4. Run the same evaluation set on at least two routes for each use case before committing.
5. Review the decision every six months. The market moves fast, and a route that was not viable last year may be the right one now.

Our agent development and data foundations services build this architecture; the governance service documents it for auditors and clients.