The Office Taxes: Which Model Should Do Your Office Work?

We gave five LLMs a Monday morning: reconcile a Q3 budget, write one vendor follow-up email, build a one-page status report.

188 graded runs. No agent harness. A blind judge from a different model family graded every run.

Here is which model should actually do your office work — and how much of its work you still have to check.

The models: Claude Fable 5.1 and Claude Opus 5.5 (the reference tier — the strongest we could run; they mark the ceiling), Claude Haiku 5.5, DeepSeek V4.1 1017, and Gemini 4 Argon.

How we ran it
Three tasks, each a 20+ minute job for a competent human. One prompt per attempt — no system prompt, no tools, no history, effort set to high, output capped at 64K tokens. The blind judge (GPT-6.1 Sol, cross-family, so Claude never grades Claude) scored each run out of 100: completeness 50, accuracy 30, format 10, actionability 10. One critical miss caps the score at 69; an invented fact stated as fact caps it at 39; passing is 70. A user's pass is stricter than the judge's: the output has to work as a usable draft, with anything unverified flagged outside the document — not buried in it, and not covered by the model's own "Done". The reference pair ran 9 and 12 times (they set a ceiling, not a ranking); Haiku 63, DeepSeek V4.1 1017 65, Argon 39. Total cost: about $28 in model calls, $20 in grading.

The headline: there is no average model

The headline isn't "model X wins." It's that performance splits by task type — for every model.

On the reconciliation, everyone is a finished product — the cheapest model posts the highest mean score in the row, and the reference tier adds nothing.

On the email and the report, pass rates collapse. For everyone. Below 55% for every model outside the reference tier — and not to 100% even inside it.

Checkable work is effectively solved. Judgment work is not. Sort your workload before you sort models: work a colleague can verify against a source in two minutes is checkable — buy it cheap and stop deliberating. Work whose quality depends on what the model decided or left out is judgment work — that's where your budget and your review time belong.

You run it once

You don't run a task 21 times and keep the best one. You run it once, under deadline, and use what comes back.

So the number that matters is not the average. It's the probability that your one run is usable:

• Email: Fable 3 in 3 · Opus 2 in 4 · DeepSeek V4.1 1017 11 in 23 · Argon 5 in 12 · Haiku 2 in 21
• Status report: Opus 3 in 4 · DeepSeek V4.1 1017 11 in 21 · Haiku 6 in 21 · Fable 1 in 3 · Argon 1 in 14

Same price tier, Haiku and DeepSeek V4.1 1017. Very different odds. Every cost figure below should be read under this rule: cost per pass is an average over many attempts, useful for breaking ties between models whose one run is likely enough — never as the main criterion.

The email, model by model

One brief for everyone: a 40-message vendor thread, four days of silence, a signature deadline. 63 graded runs. Each model failed in its own way — and the pattern is what you actually have to manage.

Fable 5.1 — 84, 84, 82. A clean email first, then "Reviewer Notes (not part of the email)": source map, conflict table, assumptions — every proposed date labelled a new ask, not a prior commitment. The only model whose email quality you can plan around. (Its word-count self-reports still can't be taken at face value.)

Opus 5.5 — 90, 80, 69, 69. The highest single score in the study, and half its runs capped. Its worst failure was reasoning, not recall: "17 active + 3 = 20, which implies 3 stay and 1 drops." The judge: "The equation is correct, but the implication is not." Seats are capacity, not occupancy. It invented a conflict by arithmetic.

Gemini 4 Argon — 5 passes in 12. The good runs are genuinely good. But 11 of its 12 emails carried "not in source" placeholders inside the deliverable, where a client would see them.

DeepSeek V4.1 1017 — 11 passes in 23. About half are usable drafts. One passing run (72) shows the gap between the judge's standard and yours: 8 of its 10 due dates read "Not in source", and its self-check marked the table Done anyway. A pass — and unsendable.

Haiku 5.5 — 2 passes in 21. Eight runs scored exactly 69, one point below passing. It finds most of the facts and misses about one critical item, every time. Predictable — and near-certain.

For the user: what separated the passes from the 69s wasn't knowledge. It was discipline: unknowns flagged outside the document, proposals labelled as proposals, a self-check that matches the document it describes.

Per token they tie. Per finished task, they don't

On the email:

• Haiku: $0.193 per pass — 19 of 21 attempts fail
• DeepSeek V4.1 1017: $0.045 per pass
• Fable: $0.607 per pass — roughly 100× the token price, about 3× the price per finished email

Haiku's number is the trap: $0.193 per pass is the harvest of ~10 attempts and ~10 reviews. Your one run fails 9 times out of 10.

On the reconciliation, the premium buys nothing at all (Haiku: $0.009 per pass, best mean in the row). On the report, Opus costs $0.391 per pass against DeepSeek V4.1 1017's $0.050 — justified for board-facing documents, not internal ones.

Budget per successful task. Never per token.

How much should you trust a model's "Done"?

About half of it. Audited against the results, the models overclaimed 36–52% of the items they marked Done — and 81% for DeepSeek V4.1 1017 on the email.

Admissions of a shortfall are rare, and they come from only two models: Haiku admitted one in 19 of 63 runs, Opus in 3 of 12. Fable, DeepSeek V4.1 1017, and Argon never admitted one — in 113 runs between them.

This is also why a 90+ score still costs you an hour of checking. Checking time is set by defects per document, not by headline scores: 0.2 per document for the reference tier, 0.5 for DeepSeek V4.1 1017, 0.9 for Haiku. Counted as employee time, a cheap-model task still costs roughly 10–14 minutes against 20 doing it yourself. A real saving — and a much smaller one than the speed gap suggests.

The five models, scored 1–5 on six axes (the Office Hexagon):

For the user: keep requiring the self-check — but read it as a map of where to dig, not as a certificate. If a model has never once said "Partial", treat every "Done" it issues as unverified by definition.

The one that worried us

Argon fills blanks with confident specifics.

On the reconciliation it is perfect: 13/13, mean 98.4. On the status report it passes 1 run in 14. The source says "this week." Its report says "Tue, Oct 13, 2026" — stated as fact, in a decisions table. In 12 of 14 runs it converted vague source timing into specific dates that no source contains, and each of those runs was capped at 39. The reports underneath were often good — weighted scores of 72.5–81.5 before the cap.

Opus, facing the identical blank, wrote: "Not in source; suggest Fri Oct 16." A few words turn a fabrication into a stated assumption — and a capped 39 into a pass. That is the behaviour you want.

Most failures in this study are of this kind: format failures, not intelligence failures. "(due: not in source)" inside a finished email, an unlabelled invented date, a self-check certifying blanks — one server-side rule removes each of them, for every model, permanently. What remains after those fixes is genuinely the model's, and it's the smaller half of the defect bill. Our next test arm measures exactly that: same models, one thin managed skill layer, hexagon re-scored.

Route by task, beat every flagship

Assign each task to the model that does it best, and the portfolio beats every single model:

• Reconciliation → Haiku · Email → Fable · Status report → Opus
• 92% of a mixed workload passes, at $0.330 per pass — 14 points above the best single model, at a lower cost per pass than either flagship

The budget version (reconciliation → Haiku, email + report → DeepSeek V4.1 1017) passes 67% at $0.028 per pass — cheaper per pass than any single model. If you must standardise on one: Opus (75%, no catastrophic task) as the all-rounder, DeepSeek V4.1 1017 (63%) as the budget standard with mandatory review.

One caveat: the email leg of that assignment rests on Fable's 3 runs (95% interval 29–100%). Read the 92% as "high", not as a precise number.

The table we'd hand a team

Three rules we're keeping

1. Sort the work before you sort the models. Checkable work is solved — buy it cheap. Judgment work is where budget and review time belong.
2. Budget per passed task, and select on single-run odds. You get one run. Averages describe a selection process you don't have time to perform.
3. Read the self-check as a map, not a certificate. It tells you where to dig. It does not tell you the work is done.

One more thing, on cost accounting: 76–92% of billed output tokens in this study were reasoning you never see, and the share is unstable — under a 32K output cap, 8 of Haiku's 20 email runs produced no deliverable at all, the budget spent entirely on reasoning. Track cost per finished task, not per token.

The bottom line: current models can do the work. Their reports about the work aren't yet reliable enough to skip the reading.