GPT-5.6 Sol vs Terra: benchmarks, pricing, and agent evals

TL;DR: GPT-5.6 Sol is the first model to test for long-running coding agents, multi-file changes, and review jobs where missing a bug is expensive. GPT-5.6 Terra is the lower-cost lane for bounded implementation, triage, and workflows that can escalate difficult cases. GPT-5.6 Luna belongs at the high-volume, low-reasoning edge of the system.

The benchmark headline is only the start. The model with the lower token price does not necessarily deliver the lowest cost per completed task. Teams should compare task success, output tokens, latency, retries, tool calls, and review quality on their own production-shaped workloads. That is where tracing and evals turn a model launch into a routing decision.

What changed with GPT-5.6 Sol, Terra, and Luna

OpenAI's GPT-5.6 release separates the family into three durable capability tiers instead of presenting one default model for every job:

• GPT-5.6 Sol is the flagship tier for complex, long-horizon work.
• GPT-5.6 Terra balances capability and cost for everyday workloads.
• GPT-5.6 Luna is the fastest and least expensive tier.

That structure makes model routing a first-class engineering concern. A coding agent can use Sol for a difficult repository change, Terra for a contained fix, and Luna for a pull request summary without changing the rest of the agent architecture.

The API pricing published by OpenAI is:

GPT-5.6 also supports explicit cache breakpoints and a minimum cache lifetime of 30 minutes. Cache writes cost 1.25 times the uncached input rate, while cache reads receive a 90% discount. That matters for agents that repeatedly load the same repository context, policies, tools, or system prompt during a long run.

Price per token is still an incomplete comparison. If one model uses twice as many output tokens, retries more often, or leaves more tasks for a human to finish, the cheaper rate may not produce the cheaper workflow.

The official GPT-5.6 coding benchmark picture

OpenAI reports strong coding results across the GPT-5.6 family. These are useful directional signals, but they are vendor-reported scores. They should guide the first experiment, not replace an evaluation on your own repositories.

Sol leads the family on all four published coding measures. Terra stays close on several of them while costing half as much per input and output token. Luna also remains competitive despite its lower price.

The gap between a public benchmark and a production agent is everything around the final answer: reading the correct files, choosing tools, preserving context, recovering from a failed test, and knowing when the task is actually finished. Those behaviors are why a team should log the whole run rather than only the last generation.

What CodeRabbit found in coding agents and code review

CodeRabbit published a separate GPT-5.6 Sol and Terra benchmark covering long-horizon coding tasks and its code-review harness. The following figures are CodeRabbit's results, not Currai benchmarks, and should be read in the context of its test setup.

In a coding run spanning more than 100 tasks across several programming languages, CodeRabbit reported:

The important result is the interaction between quality and usage. Terra's output tokens cost half as much, but its average task used more than twice as many output tokens in this run and solved fewer tasks. That does not make Terra a poor model. It shows why list price alone cannot settle a routing decision.

Code review produced a different trade-off:

Sol added recall in CodeRabbit's ensemble, but it also produced substantially more comments and nitpicks. Terra was quieter and slightly more precise in the reported run, while finding fewer expected issues. For a review product, neither number wins on its own. Missing a serious bug and overwhelming developers with low-value comments are different failure modes, and each needs its own eval.

Sol vs Terra is a cost-per-outcome decision

The simplest model-cost dashboard multiplies tokens by a rate. The useful dashboard connects that cost to an outcome.

For a coding agent, track at least:

• Whether the task passed its behavioral and regression tests.
• Input, cached input, reasoning, and output tokens.
• Wall-clock latency and time spent waiting on tools.
• Tool-call count, errors, retries, and repeated file reads.
• Human corrections or follow-up turns after the agent stopped.
• Total model cost per successful task, not only per attempt.

A basic cost-per-success calculation is:

cost per successful task = total cost of all attempts / successful tasks

That denominator prevents a cheap failed attempt from looking efficient. It also makes routing policies testable. A Terra-first policy may win if most tasks are bounded and only a small fraction escalate. A Sol-first policy may win when failed attempts create expensive retries or human cleanup.

The same principle applies to code review. Cost per pull request is useful, but it should sit beside expected issues found, precision, critical-issue recall, developer acceptance, and comment dismissal. The objective is not to generate the cheapest review. It is to produce the most useful review at an acceptable cost.

How to evaluate GPT-5.6 with production traces

Public benchmarks help choose candidates. Production traces tell you how those candidates behave inside your application.

Create one trace for the complete agent task. Record model generations and reasoning usage inside it, then represent repository search, shell commands, tests, code review, and other tool calls as spans. Attach the model name, reasoning effort, prompt version, routing policy, repository or task class, and final outcome as structured metadata.

This gives the eval enough context to answer questions that an output-only log cannot:

• Did Terra fail because of reasoning, a tool error, or an early stop?
• Did Sol spend more because the task was harder, or because it repeated work?
• Did prompt caching reduce the cost of later turns in the same repository?
• Which model produced the patch that passed, rather than the patch that merely looked complete?
• Which review comments were correct, actionable, accepted, or dismissed?

In Currai, the same trace can carry quality, latency, token, and cost signals. Failed production traces can become regression cases, and the next model or prompt version can run against the same cases. Read how to test agent cost-efficiency and run LLM evals on production traces for the full workflow.

A practical routing guide for GPT-5.6

Start with a small number of explicit lanes. A routing system is easier to evaluate when every decision has a reason that can be recorded on the trace.

This is a starting hypothesis, not a universal ranking. A team working on small, well-specified pull requests may find Terra has the best cost per resolution. A team running autonomous migrations across large repositories may find that Sol's higher unit price avoids enough retries to cost less overall.

Run a controlled GPT-5.6 model eval

Use a dataset that represents the work you expect the models to do. Include successful production traces, known failures, long-context cases, ambiguous tasks, tool errors, and tasks that require multiple test-and-fix loops.

Then compare Sol and Terra with the same:

• Agent scaffold, tools, and permissions.
• Prompt version and completion criteria.
• Reasoning-effort policy.
• Task dataset and test environment.
• Retry, timeout, and escalation rules.
• Outcome rubric and human-review process.

Run enough cases to separate a real pattern from a few lucky tasks. Segment the results by task type and complexity instead of averaging every workload into one score. A model can be the best choice for implementation and the wrong choice for high-volume review triage.

Finally, promote the result into a routing rule and keep evaluating it. Model behavior, prompts, tools, repositories, and prices change. A routing decision is an experiment that needs production monitoring, not a permanent truth.

Should you switch to GPT-5.6 Sol or Terra?

Test Sol first when completion matters more than the lowest per-token rate: long-running coding agents, difficult repository changes, test repair, and review passes where recall is the priority.

Test Terra when the work is bounded, repeatable, and easy to escalate. Its lower price creates a useful middle lane, but measure total tokens and solved-task cost before sending all coding traffic to it. Use Luna for lightweight work that has standalone value without deep reasoning.

The best GPT-5.6 model is not the one with the strongest launch table. It is the one that produces the best measured outcome for a specific workload. Trace the agent, evaluate the result, compare cost per success, and route from evidence.

Related: LLM observability, LLM evals, track token cost, and observability for AI agents.