Tool Use with Open Models: function calling, sandboxes, and “capability boundaries”

July was about truth.

If you want to ship RAG without lying to users, you need a retrieval pipeline you can evaluate:

• queries you can inspect
• rerankers you can A/B
• citations you can audit
• and boundaries that define “what must never be wrong”

August is about action.

Because the moment you let the model do anything beyond “answer in text” — browse, call APIs, run code, create tickets, trigger workflows — you’ve crossed a line:

The model is now an integration surface.

And that changes everything.

With closed models, the platform often gives you a lot “for free”:

• stable function calling
• strong instruction-following
• safety layers
• high-quality tool selection

With open weights, you get something else:

control + portability + cost leverage — but also:

you own the reliability.

This month is about how to design tool use so it’s:

• operable (you can debug it)
• safe (it can’t do dangerous things)
• predictable (it does what the contract says)
• and scalable (works across model tiers)

Tool Use Is Not “Agents” — It’s Typed Integration

The internet loves the word agent.

Engineers should love a different phrase:

typed integration.

A tool call is not “the model deciding to do something.”

It’s the model producing a structured request that you interpret and execute.

That subtle framing matters because it forces the right architecture:

• the model proposes
• the system decides
• tools execute inside constraints
• results come back as evidence

If you let the model “execute” directly, you’ve already lost.

What “Function Calling” Really Means

Function calling is a family of techniques for making tool use reliable.

At minimum, you need:

• a tool registry (name + description + schema)
• a tool-call format (JSON-like)
• a dispatcher (select tool → validate args → run)
• a result envelope (success/error, structured output)

With open models, you usually also need:

• strict schema validation
• repair loops (re-ask the model to fix malformed args)
• constrained decoding (JSON grammar / schema-guided output) where available
• and fallbacks when the model can’t reliably produce structure

The Three Boundaries That Matter

There are three boundaries in any tool-using system:

1. Language boundary (prompt ↔ model output)
2. Capability boundary (what the model is allowed to attempt)
3. Execution boundary (what the system is allowed to do)

Most failures happen because these boundaries are blurred.

Here’s a useful rule:

The model is untrusted input at every boundary.

It can be wrong. It can be manipulated. It can be confidently unsafe.

So we design as if tool calls are user input — because functionally, they are.

Capability Boundaries: Make Power Explicit

A capability boundary is a policy layer that answers:

• Which tools can be used in this context?
• How many times?
• With what parameter constraints?
• Under what approval rules?
• With what audit requirements?

This is where “model selection becomes architecture” becomes real.

Not every model gets the same capabilities.

Not every request gets the same capabilities.

A simple capability matrix

Sandboxes: The Execution Boundary Is Where Adults Live

A sandbox is the environment where tool execution happens under restrictions.

It’s not optional.

It’s how you turn “the model can run code” into “the model can run code safely.”

A good sandbox gives you:

• isolation (filesystem, processes, network)
• resource limits (CPU, memory, timeouts)
• capability limits (no arbitrary network, allowlisted domains only)
• deterministic IO (input/output envelopes, audit logs)
• clean teardown (no state leaks between runs)

Two kinds of sandboxes you’ll actually use

In practice, the “connector sandbox” is your most important one.

Because the most dangerous tool is not “run Python.”

It’s “call production APIs with credentials.”

The Tool Gateway Pattern (My Default)

If you do one thing, do this:

Tools don’t call APIs directly. Tools call a gateway.

The gateway is where you enforce:

• authentication and scoping
• allowlisted endpoints
• rate limits and quotas
• payload size limits
• PII redaction
• request/response logging (evidence)
• retry policies and idempotency

This makes tools boring — and boring is good.

The Contract Surface: Schemas, Versions, and Error Codes

Tool use becomes reliable when you stop thinking “prompt” and start thinking “protocol.”

Tool schema rules I use

• every tool has a name and a version
• inputs are strictly typed (JSON schema-like)
• outputs are structured (never “just text” if you can avoid it)
• errors are typed:
  • VALIDATION_ERROR
  • AUTH_ERROR
  • RATE_LIMITED
  • UPSTREAM_TIMEOUT
  • NOT_FOUND
  • CONFLICT
  • INTERNAL_ERROR

And every tool call is wrapped in an envelope:

{
  "tool": "crm.create_ticket",
  "version": "v1",
  "args": { "title": "...", "priority": "P2", "customerId": "..." },
  "requestId": "..." 
}

The model is allowed to propose this.

Your system is allowed to reject it.

That separation is the whole game.

Failure Modes: How Tool Use Breaks in the Real World

Tool-using systems don’t fail like normal software.

They fail like distributed systems with a stochastic planner.

Here are the patterns you should assume will happen:

Tool Use with Open Models: What Changes (Practically)

Open models are totally usable for tool use — but you need to compensate for variability:

• function calling may be less consistent across prompts
• outputs may be less strictly structured
• tool selection may be noisier
• refusal behavior may be weaker or absent
• long-horizon planning may degrade faster

So you shift more responsibility into system constraints.

Here’s the trade:

If you want open weights to feel “production-grade,” the trick is simple:

Upgrade the model from “decision maker” to “proposal engine.”

A Minimal Architecture That Actually Works

This is the smallest tool-use architecture I’m comfortable shipping:

Components:

1. Context assembly
   • system policy + task instructions
   • user intent
   • relevant retrieved facts (from July)
   • tool registry summary (names + when to use)
2. Model
   • produces either an answer or tool calls
3. Capability gate
   • decides allowed tools for this request and model tier
   • enforces budgets
4. Validator
   • schema checks + grounding checks
5. Tool gateway + sandbox
   • executes safely
   • returns structured results
6. Evidence log
   • stores tool calls + outputs + traces
7. Response composer
   • renders an answer with citations, results, and uncertainty

This is not “overengineering.”

It’s the minimum to avoid being surprised in production.

Budgets: Tokens, Tool Calls, and Blast Radius

A tool-using system needs three budget types:

The blast radius budget is the most overlooked.

Examples:

• max “create ticket” operations per request = 1
• max emails sent per day per tenant = N
• max external API calls per request = 5
• max file writes per sandbox session = 20MB

Budgets turn “AI behavior” into something you can operate.

Implementation Guide: A Practical Checklist

The Hidden Payoff: Evidence-Ready Systems

Once you build tool use with:

• capability boundaries
• sandboxes
• gateway controls
• and evidence logs

…you get something incredibly valuable for free:

You can prove what happened.

And that’s not just good engineering.

That’s where this series is heading next.

Because the uncomfortable truth is:

When regulation shows up, teams scramble. When your system already produces evidence, regulation becomes an architecture problem — not a panic.

Resources

What’s Next

August was about turning “tool use” into a real subsystem:

• the model proposes
• policy gates
• sandboxes execute
• evidence logs make it auditable

Next month we make that explicit:

Regulation becomes architecture.

Not as legal theory.

As controls, traceability, and proof.

Regulation as Architecture: turning the EU AI Act into controls and evidence