AI Models

Choosing a model

The right model depends on your task. A rough heuristic: use Sonnet for most day-to-day agent work, o3 for high-stakes or complex planning tasks, and GPT-4o when you need fast turnaround on simpler jobs.

You can configure a fallback model in your agent config. If the primary model is unavailable or returns an error, Queue will automatically retry with the fallback.

Metric

MMLU Score

GPQA Score

Context Size

Parameters

Input Price

Inference Speed

Throughput

Latency

Gemini 3.1 Pro

92.0%

59.1%

1,000,000

~1T

$3.50

150

320

0.62

Sonnet 4.6

88.3%

65.0%

200,000

~400B

$3.00

142

280

0.71

Context window and token limits

Every model has a maximum context window. Queue automatically manages context to stay within limits: it summarizes older turns, trims tool outputs, and prioritizes the most relevant parts of your codebase. You can tune this behavior with the context_strategy field in your agent config.

Model configuration

Set the model in your queue.yaml agent config under the model key. You can also pass --model at runtime to override it for a single run without editing your config file.

Usage and billing

Token usage is tracked per run and visible in the Queue dashboard. You are billed based on the model and the number of tokens consumed across all steps in a run, including tool call inputs and outputs.