GPAI Obligations Begin: What Changes for Model Providers and Enterprises

Last month was security for connectors.

This month is what happens when security stops being “best practice”… and becomes a regulated contract boundary.

Because in late 2025, “pick an LLM provider” is no longer just a technical decision.

It’s a compliance decision.

And that changes the architecture.

Not in a philosophical way.

In a boring, operational way:

• you need inventories
• you need controls you can prove
• you need logs you can retrieve
• you need incident runbooks
• you need vendor packets you can version
• you need a governance surface that matches the reality of agentic systems

This post is about the moment GPAI obligations begin—and what it forces model providers and enterprises to change.

First: What “GPAI obligations” really means in practice

“GPAI” (General-Purpose AI) is the foundation model layer: models that can be used across many tasks and then adapted downstream.

That matters because this layer behaves like a platform dependency, not a feature:

• it updates frequently
• it’s probabilistic
• it’s used across many products
• its failure modes are non-local (a single model change can break many workflows)

So the AI Act-era move is predictable:

If a component is a platform dependency, it gets a governance interface.

The governance interface isn’t just “security.”
It’s documentation, transparency, and post-market monitoring as a routine part of operating the model.

The big reframing

Before:

• “We integrated an LLM API.”

After:

• “We integrated a regulated upstream component with obligations, evidence, and monitoring expectations.”

That framing is what stops this from turning into chaos.

Providers vs enterprises: two different kinds of work

This post is split into two tracks:

• Model providers (GPAI providers): what you must produce (artifacts + controls) and how you operate them.
• Enterprises (deployers/builders): what you must demand, store, and enforce when you build products on top of GPAI.

Because most teams lose a year by mixing these up.

The new compliance interface

When obligations kick in, the model stops being “just an endpoint.”

It becomes a dependency with versioned evidence.

Think of it like this:

• In 2010, databases forced teams to learn migrations.
• In 2020, microservices forced teams to learn contracts.
• In 2025, GPAI forces teams to learn evidence + governance as part of integration.

Here’s the simplest mental model I’ve found useful:

The rest of this article is: how to build these interfaces into your platform.

What changes for GPAI model providers

If you are a provider, you are no longer shipping “a model.”

You are shipping:

• a model
• plus a packet of evidence
• plus an operating model for risk and incidents
• plus change control that downstream teams can survive

1) You need a “System Card”, not just a model card

A normal model card is a description.

A System Card is a contract:

• intended use vs prohibited use
• known limitations
• safety assumptions (what your mitigations do not cover)
• evaluation scope (what you tested, and what you didn’t)
• integration requirements (logging, retention, input constraints, rate limits, etc.)

If you don’t ship this as a stable artifact:

• enterprises invent their own
• vendors get blamed for downstream misuse
• audits become a fight instead of a review

2) Transparency becomes an API surface

Providers should assume downstream teams will ask for:

• documentation packages tied to model versions
• policies around training data + copyright handling
• summaries of training data characteristics (at the level allowed/required)
• instructions for safe integration (including security notes)

Which means you need artifact versioning as a first-class system:

• model version → artifact bundle version
• artifact bundle version → policy + evaluation run IDs
• evaluation run IDs → stored results + prompts + environments

3) Systemic-risk models turn “safety work” into SRE work

For the class of models treated as higher risk, safety becomes operational:

• continuous evaluation and regression detection
• incident reporting pathways
• red-teaming as a scheduled discipline
• security hardening of serving infrastructure
• monitoring for abuse patterns at scale

That means safety needs the same things SRE needs:

• on-call
• runbooks
• alert hygiene
• postmortems
• “known failure modes” lists
• capacity planning, because denial-of-wallet is also a safety incident

What changes for enterprises (even if you’re “just using an API”)

Enterprises tend to make the same mistake:

They treat regulation as a procurement problem.

It is also a runtime problem.

Because the AI Act doesn’t care that your architecture diagram is “clean.”
It cares whether you can:

• control where the model is used
• constrain what it can do
• show what happened
• respond to incidents
• update safely

So the main shift is simple:

You need a Model Governance Control Plane.

If you don’t build it, you’ll rebuild it repeatedly, badly, in each product team.

The Model Governance Control Plane (MGCP)

A control plane is how platforms avoid chaos.

For GPAI, your MGCP is the place where you answer:

• Which models are approved?
• For which use-cases?
• Under which constraints?
• With which evidence packet?
• With which monitoring?
• With which incident workflow?

The MGCP has five concrete components

Notice what’s missing:

There’s no “compliance team” box here.

Because the platform must make compliance default for engineers.

The connector is now the choke point

In August we talked about connector security:

• least privilege
• injection resistance
• safe toolchains

In September, the conclusion is sharper:

The connector is where you enforce governance.

Why?

• It’s the place all traffic passes.
• It’s where you can apply policy consistently.
• It’s where you can capture logs without begging teams.

So in a regulated world, your connector layer should do four things reliably:

This is why “LLM gateway” products exist.
But you can build the core yourself if you treat it like an API gateway with extra semantics.

A practical enterprise playbook: become “GPAI ready” in 6 steps

This is the smallest plan I’ve seen work in real orgs.

It’s boring. That’s why it works.

“Eventually correct” governance: why outbox and sagas matter here

In June we talked about agents as distributed systems.

This month is where that becomes compliance-critical.

A regulated platform needs durable workflows for:

• approvals
• evidence ingestion
• policy changes
• incident handling
• model upgrade rollouts
• audit exports

If these workflows rely on “someone remembering,” you will fail audits and reliability simultaneously.

So the same patterns apply:

• transactional outbox for “we approved model X for use-case Y” events
• sagas for “upgrade model version across 12 services with canaries and rollbacks”
• idempotency keys for every step in the governance pipeline

A concrete example: an agent that sends emails on a customer’s behalf

Let’s make it real.

You ship a “support agent” that:

• reads a ticket
• drafts an email
• optionally sends it
• updates the CRM
• opens a refund workflow if needed

In 2024, this is mostly “tool use + guardrails.”

In 2025, it also needs:

• proof of which model version generated what
• proof that the agent had the right scope to send email
• proof that the agent didn’t leak other customers’ data into the prompt
• a rollback plan if the model starts producing unsafe drafts

That translates to platform features:

• connector gateway with scoped tokens for email/CRM tools
• trace graph that links ticket ID → prompt hash → tool calls → sent message
• an “approval required” state for refunds above a threshold
• automated eval scenarios that test prompt injection attempts inside ticket text
• an incident runbook (“disable send tool”, “pin to safe model version”, “force human handoff”)

This is what I mean by:

Compliance is a runtime surface.

Common traps (and how teams get hurt)

The enterprise procurement questions that finally matter

If you take one list into vendor calls, take this one.

Notice how none of these are “what’s your benchmark score.”

That’s not because benchmarks don’t matter.

It’s because in 2025, operability beats capability.

Resources

What’s next

September is the month where agents become explicitly regulated platforms.

The next month is where the governance story meets a brutal UX constraint:

voice.

Because voice agents are where latency, caching, reliability, and handoff stop being “nice-to-haves.”

Next article: Voice Agents You Can Operate: reliability, caching, latency, and human handoff