The production tool-calling loop for AI agents

An LLM does not execute your tool.

It proposes a structured call. Your application validates the request, decides whether it is allowed, runs the function, and returns the result to the model. That distinction is the foundation of a reliable agent.

The loop has four steps

1. Send messages and tool definitions to the model.
2. Receive one or more tool-call requests.
3. Validate and execute each approved call.
4. Return results with their call identifiers and request the final response.

A minimal TypeScript shape looks like this:

const first = await client.chat.completions.create({
  model,
  messages,
  tools,
});

const calls = first.choices[0].message.tool_calls ?? [];

for (const call of calls) {
  const args = JSON.parse(call.function.arguments);
  const result = await executeSafely(call.function.name, args);
  messages.push({
    role: "tool",
    tool_call_id: call.id,
    content: JSON.stringify(result),
  });
}

The code is short. The controls around executeSafely determine whether it is production-ready.

Treat schemas as an interface contract

Use precise tool names, descriptions, enums, required fields, and constraints. Validate arguments after parsing; JSON syntax does not guarantee that an ID exists, a date is acceptable, or a user has permission to change a record.

Keep tools narrow. manage_account invites ambiguity. Separate functions such as get_account, update_billing_address, and cancel_subscription make the agent’s decision observable and permissions easier to enforce.

Put authorization in application code

The prompt is not a security boundary. A model instruction saying “ask before deleting” cannot replace access control.

Before execution, check:

• The authenticated user can perform the action.
• The requested resource belongs to the permitted scope.
• Required confirmation is present and recent.
• Arguments satisfy business invariants.
• The tool is enabled in this environment.

High-impact tools should return a preview or require an explicit approval token before committing a side effect.

Design for retries and duplicate calls

Network timeouts create uncertainty. The tool may have completed even when the agent did not receive the response. Use idempotency keys for payments, bookings, messages, and updates. Return a stable operation identifier so a retry can recover the existing result.

Classify errors into retryable, correctable, and terminal. Give the model a safe, structured result rather than a raw stack trace:

{
  "ok": false,
  "code": "SLOT_UNAVAILABLE",
  "message": "Choose another appointment time",
  "retryable": false
}

Never let the agent announce success before the tool result confirms it.

Handle parallel calls deliberately

Models may request several tools in one turn. Execute independent reads in parallel, but preserve ordering for dependent actions. A call that charges a customer must not race the call that verifies the order.

Streaming adds another constraint: arguments can arrive across multiple deltas. Collect the complete call before parsing or executing it.

Evaluate trajectories, not just replies

A correct final sentence can hide a wrong tool, unnecessary call, invalid argument, or failed side effect. Evaluate:

• Whether the required tool was called
• Whether prohibited tools were avoided
• Argument correctness
• Ordering and dependencies
• Use of the returned result
• Recovery behavior
• End-to-end outcome

Include failures from production in the regression set. The most valuable tool eval often begins as a real incident.

Trace tool calling with Currai

Currai records the user message, model response, tool request, tool result, and final outcome as one trace. Teams can identify silent failures, compare models, and alert when an agent reports an outcome that the tool never produced.

Use the Currai integration skill, or read the guide to evaluating tool calling and task completion.

Sources and further reading

• OpenRouter: Tool calling across models
• OpenRouter tool-calling documentation
• Currai: trace parallel function calling