PassedPassed on this receipt
Bn fit modify
Completed the recorded checks.
Exact task metadata
bn-fit-modify · Task ID bn-fit-modify
No recorded stop reason
Reward1
Assertions—
Steps—
Tool calls—
Model calls—
Tokens in / out—
CodingTerminal
Terminal-Bench 2.1
Version 2.1 · 89 tasks. Exact task sets and protocols remain separate.
Can it finish a task in a real terminal?
69.7%Recorded score · 62 / 89 tasks
GLM-5.3-Flash · NVFP4/MXFP8 CSF-QAD · 2× GB10 (DGX Spark head + ASUS GX10 worker) · 256GB total
Inspect the measured run ↗How scoring works ↓THE QUESTION IT ANSWERS
Can it finish a task in a real terminal?
Measures whether an agent can navigate an environment, execute commands, and complete a technical task.
- 01Task + environment→
- 02Use tools over many steps→
- 03Task verification
Interpret the score The harness, tools, runtime, and context budget all contribute to the outcome.
BEYOND THE HEADLINE
What happened across 89 tasks?
Each block represents one recorded task, in receipt order from left to right, then top to bottom. Fill shows the outcome; work-category colors are not used here.
Fully passed62 69.7%
No credit24 27.0%
Other / unscored3 3.4%
Tested by LocalsOnly · NVFP4/MXFP8 CSF-QAD · 2× GB10 (DGX Spark head + ASUS GX10 worker) · 256GB total. Partial progress still requires further work before a task is complete.
Inspect individual task outcomes →What this test is
What this benchmark measures
Multi-step tasks in a terminal environment.
The paper separates execution, coherence and verification failures. A task can fail because the environment or tools break, because the agent loses track of the goal, or because it fails to check its work.
Terminal-Bench research ↗How it grades
How a task earns its score
Each scored item is a pass or fail in a terminal environment.
The LocalsOnly measured run is Harbor-style: per-task pass and, when present, an issue (for example AgentTimeoutError). A setup issue is an execution failure on the receipt, not a separate qualitative score.
Published rows are vendor or bench cites. They do not share the measured task grid.
SuiteTerminal-Bench 2.1
Tasks89 on the measured receipt
PassRecorded pass on the task
FailRecorded fail on the task
Setup issueAgentTimeoutError
Follow one task · this measured recipe
A passed task and a failed one
Same LocalsOnly receipt on 2× GB10 (DGX Spark head + ASUS GX10 worker) · 256GB total. GLM-5.3-Flash · NVFP4/MXFP8 CSF-QAD · 2026-10-28. Assertions, steps, and tool calls are on the task record. There is no agent transcript on this page.
passed vs AgentTimeoutError
Not passedFailed on this receipt
Adaptive rejection sampler
Stopped at the recorded time limit.
Exact task metadata
adaptive-rejection-sampler · Task ID adaptive-rejection-sampler
AgentTimeoutError · Agent execution timed out after 3600.0 seconds
Reward0
Assertions—
Steps—
Tool calls—
Model calls—
Tokens in / out—
Subject & category breakdown · this recipe
Results by subject or work category
This receipt does not record a subject or work-category breakdown. A suite-wide label cannot support a finer subject comparison. Outcome groups such as “passed” and “timeout” are not subjects.
Ranking
Results & published references
Place is assigned only among rows that share a comparable score on this board. Evaluation settings can differ across sources.
Tested by LocalsOnly
Results from the exact local setup shown below.
| # | Model / recipe | Reported score ↓ | Runs on | Evidence | |
|---|---|---|---|---|---|
| 01 | ◎GLM-5.3-Flash · FlareNVFP4/MXFP8 CSF-QAD · Flare | 69.7% | 2× GB10 (DGX Spark head + ASUS GX10 worker) · 256GB total | ● Tested by LocalsOnlyLocalsOnly | ↗ |
| 02 | ✳Qwen 3.8 Flash-Next · llama.cppUD-IQ3_XXS · llama.cpp | 48.3% | GB10 · 128GB | ● Tested by LocalsOnlyLocalsOnly | ↗ |
| 03 | ◎GLM-5.3-Flash · TensorFoldEXL3 TR3 4bpw · TensorFold | 43.8% | 2× GB10 (DGX Spark head + ASUS GX10 worker) · 256GB total | ● Tested by LocalsOnlyLocalsOnly | ↗ |
| 03 | ◎GLM-5.3-Flash · vLLMEXL3 TR3 4bpw · vLLM | 43.8% | 2× GB10 (DGX Spark head + ASUS GX10 worker) · 256GB total | ● Tested by LocalsOnlyLocalsOnly | ↗ |
| 05 | ✳Qwen 3.8 27B · llama.cppUD-Q4_K_XL · llama.cpp | 30.3% | GB10 · 128GB | ● Tested by LocalsOnlyLocalsOnly | ↗ |
GLM-5.3-Flash · SGLang NVFP4: DNQ (did not qualify): the SGLang NVFP4 + DFlash2 Terminal-Bench 2.1 run was stopped by choice at 32 of 89 trials, because a full run would take several more days (full runs took about 25.4 h on vLLM and 17.8 h on TensorFold). Partial results are not a score and are not ranked. Hardware: DGX Spark head + ASUS GX10 worker.
GLM-5.3-Flash · Flare NVFP4/MXFP8 CSF-QAD: Same dataset and Harbor terminus-2 settings as the vLLM and TensorFold GLM-5.3-Flash rows (max_turns 40, timeouts x2/x4, one trial per task), but not an engine-only comparison: Flare forces thinking on (the others ran thinking off), and the quant (NVFP4/MXFP8 vs EXL3 4bpw), context (1M vs 100k) and sampling differ. 3 harness errors are scored 0; excluding them gives 62/86 = 72.1%, a secondary figure, not the score. Hardware: DGX Spark head + ASUS GX10 worker.
Published reference
Results reported by model developers and benchmark authors.
| # | Model / recipe | Reported score ↓ | Runs on | Evidence | |
|---|---|---|---|---|---|
| 01 | ◎GPT-5.6 SolSol Ultra · four agents · Four-agent evaluation | 91.9% | Published reference | ↗ Published resultOpenAI | ↗ |
| 02 | ◎GPT-5.6 SolPublished reference · Reported evaluation | 88.8% | Published reference | ↗ Published resultPublished by OpenAI | ↗ |
| 03 | ◎GPT-5.6 TerraPublished reference · Reported evaluation | 87.4% | Published reference | ↗ Published resultPublished by OpenAI | ↗ |
| 04 | ◎GPT-5.6 LunaPublished reference · Reported evaluation | 84.7% | Published reference | ↗ Published resultPublished by OpenAI | ↗ |
| 05 | ◎GLM-5.3-FlashPublished reference · Reported evaluation | 84.3% | Published reference | ↗ Published resultZ.ai | ↗ |
| 05 | ✺Claude Fable 5Anthropic system card · Anthropic evaluation | 84.3% | Published reference | ↗ Published resultAnthropic | ↗ |
| 07 | ◎GLM-5.3-Flash NVFP4Published reference · Reported evaluation | 83.2% | Published reference | ↗ Published resultNVIDIA | ↗ |
| 08 | ✺Claude Fable 5Published reference · Reported evaluation | 83.1% | Published reference | ↗ Published resultPublished by OpenAI | ↗ |
| 09 | ◎DeepSeek V4 FlashPublished reference · Reported evaluation | 82.7% | Published reference | ↗ Published resultDeepSeek | ↗ |
| 10 | ✺Claude Opus 4.8Published reference · Reported evaluation | 78.9% | Published reference | ↗ Published resultPublished by OpenAI | ↗ |
| 11 | ✺Claude Opus 4.8Anthropic system card · Adaptive thinking · max | 74.6% | Published reference | ↗ Published resultAnthropic | ↗ |
| 12 | ✳Qwen 3.8 27BBF16 reference · Reported evaluation | 73.0% | Published reference | ↗ Published resultQwen | ↗ |
| 13 | ✦Gemini 3.1 Pro PreviewPublished reference · Reported evaluation | 70.7% | Published reference | ↗ Published resultPublished by OpenAI | ↗ |