Decouple the Agent: Why Prompts, Tools, and Models Don't Belong in Your Client

Here is the agent client most teams ship:

// agent.ts — everything your agent is, compiled into the binary
const SYSTEM_PROMPT = `You are a support agent for Acme Corp.
Always answer in English. Never discuss pricing...`; // 400 more lines

const MODEL = 'gpt-5.5'; // chosen in a meeting, six months ago

const tools = [
  { type: 'function', function: { name: 'lookup_order', /* ... */ } },
  { type: 'function', function: { name: 'issue_refund', /* ... */ } },
]; // executed locally, with API keys from .env

const res = await openai.chat.completions.create({
  model: MODEL,
  messages: [{ role: 'system', content: SYSTEM_PROMPT }, ...history],
  tools,
});

It works. It demos well. And every capitalized constant in that file is a release cycle waiting to happen:

• The prompt needs a tweak? Release.
• A tool has a bug? Release.
• The model gets deprecated, repriced, or outperformed? Release — after someone greps every repo that hardcoded it.

Your agent's interface changes maybe twice a year. Its intelligence needs to change weekly. Coupling them means the slow one sets the pace for both.

The fix is one rule:

The client holds a session, not a brain. System prompts, tools, and models are server-side concerns. The client renders the conversation.

Here is the same agent as a thin client against Vivgrid:

// agent.ts — the whole thing
const openai = new OpenAI({
  baseURL: 'https://api.vivgrid.com/v1',
  apiKey: process.env.VIVGRID_TOKEN,
});

const res = await openai.chat.completions.create({
  messages: history, // that's it
});

Notice what's missing: no model, no tools, no system prompt. They didn't disappear — they moved to where they can change without shipping software. Let's walk through each one.

Best practice #1: Treat the system prompt as runtime config

A system prompt is not source code. It is operational behavior — closer to a feature flag than to a function. It will be edited by people who don't write TypeScript (product, legal, support leads), and it will need to change at the worst possible time.

The 2 a.m. scenario: your agent starts confidently quoting a discount policy that was retired last quarter. With the prompt compiled into the client, the fix is a hotfix release and an app-store review — your agent keeps misquoting policy for days.

With the prompt managed in the Vivgrid Console, the fix is: edit, save. Every conversation that starts after that second uses the corrected prompt. The incident lasts minutes, and the postmortem includes who changed what, when.

The deeper win is iteration. When prompt changes are free, your team actually makes them — tightening tone, patching edge cases, encoding what support learned this week. When every change costs a release, the prompt fossilizes.

Best practice #2: Make the model a platform decision — and enforce it

Hardcoding a model ID feels harmless — it's one string. But that string is a pricing commitment, a latency profile, a compliance surface, and a deprecation timeline, all chosen at compile time and frozen until the next release.

It also shouldn't be every developer's decision. When any engineer can switch the production agent to whatever model they benchmarked last night, your cost and behavior drift one commit at a time. Model selection is a governance decision: run the evals, compare cost and latency on real traffic, then switch — for every agent at once, in the console, with no client change.

A policy is only as good as its enforcement, and enforcement here has two halves:

• Clients should not send model. A thin client has no business hardcoding one (look back at the thin client above — it doesn't).
• The server should not trust it if they do. The gateway ignores any client-supplied model and replaces it with the value configured for the project.

// even if a stray client sends one…
await openai.chat.completions.create({
  model: 'gpt-4o-mini', // ignored — the console's choice serves the request
  messages: history,
})

Why enforce instead of merely document? Because any honored client value is a bypass. An old install pinning a deprecated model can block your migration. A leaked token that's allowed to pick models can run up your bill on the most expensive one.

Done right, this cuts both ways:

• Upgrades: a frontier model ships, your evals confirm it wins, and production is on it the same day — not next quarter when the release train leaves.
• Stability: nobody "just tries" a model in production, because the client physically cannot specify one.

Developers lose nothing except a foot-gun — you still experiment freely against dev projects. On Vivgrid, the model that serves a request is the one set in the console, and the response's model field reports what actually ran: clients keep full transparency without holding any authority. Treat the request field as a hint at best, and configuration as truth.

Best practice #3: Run tools as Managed Skills

Tools are where agents touch the real world — and where client-side architecture hurts most. A tool that runs on the client needs its credentials on the client, ships its bugs to every install, and logs its failures to a console nobody is watching.

On Vivgrid, a tool is a strongly typed serverless function. Here's a complete one:

// app.go — a Managed Skill
func Description() string {
	return `Get current weather for a given city.`
}

func InputSchema() any {
	return &Args{}
}

type Args struct {
	City string `json:"city" jsonschema:"description=The city name"`
}

func Handler(ctx serverless.Context) {
	var p Args
	ctx.ReadLLMArguments(&p)
	ctx.WriteLLMResult(getWeather(p.City))
}

Deploy it to every region at once:

yc deploy app.go --env WEATHER_API_KEY=xxx

Three things just happened that your client-side version couldn't do:

1. Every agent got the tool instantly. No version skew, no waiting for users to update. Fix a bug at 10:00, and the 10:01 conversations run the fix.
2. The API key went server-side. --env lives with the function in your cloud, not in a laptop's .env file. Clients hold a token, not your secrets.
3. Every invocation became observable. Arguments in, results out, latency, cost — centrally logged. When a tool misbehaves, you read the trace in the console instead of asking a customer to send screenshots.

That last one changes debugging more than any framework ever will. The worst bugs in agent systems live in the seams — the model called the tool with arguments you didn't expect, or interpreted a result you didn't anticipate. When every seam is recorded, those bugs go from unreproducible to obvious.

What you just gained

For developers, the day-to-day is simply smaller. The client codebase shrinks to UI and session handling. Debugging starts at one console with full traces instead of grepping logs across machines. Prompt tweaks, tool fixes, and model swaps stop being your release problem.

For the business, the product gains something it has never had before: control that survives launch.

Ship the interface once. Iterate the intelligence forever.

Vivgrid is the Managed Skills platform: serverless LLM function calling that lets enterprise AI agents run their tools in the cloud — with observability, evaluation, and globally distributed inference built in. Start with the Quick Start, or talk to us at hi@vivgrid.com.