When the cheapest model stops being the cheapest: the cost of a wrong answer (run of 2026-11-04)
The pick on price alone is GPT-4o mini: $0.0000087 per answer against $0.0000203 for GPT-6 Luna. But GPT-4o mini was wrong on 6 of 75 answers (8%): 2 of the 25 tasks (the leap-year day count and the sentence without the letter e), each missed in all three repeats. GPT-6 Luna was wrong on none. Whether the cheap model is really cheaper depends on one number the benchmark cannot supply: what a wrong answer costs you.
In one line, from the run of 4 November 2026: GPT-4o mini cost $0.0000087 per answer and was wrong on 6 of 75 answers (8%, a miss rate of 0.08); GPT-6 Luna cost $0.0000203 per answer and was wrong on none, on the same 75 answers.
The formula, using only the figures of this run: total cost per task = price per answer + miss rate x cost of one wrong answer.
- GPT-4o mini: $0.0000087 + 0.08 x cost of a miss.
- GPT-6 Luna: $0.0000203 + 0.00 x cost of a miss.
They are equal when the cost of a miss is ($0.0000203 - $0.0000087) / 0.08 = $0.000145, about 0.0145 cents.
| Cost of one wrong answer to you | GPT-4o mini: cost per 1,000 tasks incl. misses | GPT-6 Luna: cost per 1,000 tasks incl. misses | Cheaper overall |
|---|---|---|---|
| $0.000100 | $0.0167 | $0.0203 | GPT-4o mini |
| $0.000145 | $0.0203 | $0.0203 | equal |
| $0.001000 | $0.0887 | $0.0203 | GPT-6 Luna |
| $0.010000 | $0.8087 | $0.0203 | GPT-6 Luna |
| $0.100000 | $8.0087 | $0.0203 | GPT-6 Luna |
So: if a wrong answer costs you more than $0.000145, the model that was never wrong is the cheaper choice overall; below that, the cheaper model wins. A wrong answer that triggers a person's time, a support message or a retry usually costs far more than that. What would change the pick: a different miss rate on your own tasks, a price change, or a task kind where the cheap model's misses do not occur (the misses of GPT-4o mini here were in dates and units and in instructions only).
How the run was made (4 November 2026, file date 2026-11-04)
25 small tasks in five kinds (dates and units 4, extraction 4, instructions 5, knowledge 4, reasoning 8), each asked 3 times, so 75 answers per model. Same prompt to every model, temperature 0, no system prompt. A program scores every answer (a number within its tolerance, an exact word, JSON equal to the expected value, or a format rule such as a line count, word count or forbidden letters); no model judges another. A wrong format counts as wrong. No request failed, so every model has 75 scored answers. Repeats at one temperature are not independent trials, so the three repeats do not triple the evidence. Seconds are wall time from one PC and include the network. Prices are the providers' list prices as the run read them on 2026-11-04 (they may have changed since); the cost of a model is its input and output tokens times those prices.
What this run cannot tell
It cannot say how a model does on tasks that are long, open-ended or in another language, only on these 25 short ones. 25 tasks is a small sample: a model that scored 100% here can still miss on a task nobody wrote. It cannot say how prices or models change after 2026-11-04. The tasks are published as a list of kinds, and the run is repeated on a date so a change can be seen.