DeepSeek V4.1 Flash tied Claude Sonnet 5.5 on my blog intros, at about an eighth of the cost
Four AI models got the same three blog intros to write for this site. DeepSeek V4.1 Flash and Claude Sonnet 5.5 tied on blind scores at 3.93 out of 5, yet Sonnet cost about eight times as much per intro: roughly $0.014 against $0.002. The cheapest model in the test was Claude Haiku 5.5, at about $0.001 per intro and only 0.13 points behind the leaders. It was three prompts with one run per model, so treat it as an early signal about short, routine writing.
How I ran the test
My AI test agent sent the same job to each model through OpenRouter (one API for many models): an intro of 110 to 130 words, in my voice, for three topics from my blog, including my guide to virtual assistant job sites and my learning roadmap for VAs. Every model got the same system prompt, temperature 0.7 and a 6,000-token output limit. The prompt banned invented stats, prices and experience, em dashes, and the "it's not X, it's Y" pattern.
The contenders were DeepSeek V4.1 Flash (released September 10, 2026), Claude Sonnet 5.5 (September 28), Claude Haiku 5.5 (October 7) and Google's Gemini 3.8 Flash.
Each intro got a score from 1 to 5 on accuracy, voice, usefulness, following instructions and AI tells (phrasing that reads as machine-written). Scores came from my AI test agent, itself built on a Claude model, with the model names hidden behind letters. An OpenAI model that wasn't in the test, GPT-5.4 mini, scored every intro again as a tie-break.
| Model | Average score | Cost per intro | Median time |
|---|---|---|---|
| DeepSeek V4.1 Flash | 3.93 | $0.0017 | 9.4 s |
| Claude Sonnet 5.5 | 3.93 | $0.0141 | 10.5 s |
| Claude Haiku 5.5 | 3.80 | $0.0010 | 10.0 s |
| Gemini 3.8 Flash | 3.30 (2 of 3 intros) | $0.0178 | 28.5 s |
Costs are what OpenRouter billed. Times include a little overhead from my own tools.
DeepSeek V4.1 Flash vs Claude Sonnet 5.5: what the scores show
On these three intros, DeepSeek V4.1 Flash scored the same as Claude Sonnet 5.5: 3.93 each. The tie-break judge picked DeepSeek's intro as the best for two of the three topics, but it gave DeepSeek, Sonnet and Haiku the same average (4.40), so I'm calling it a tie.
DeepSeek's weak spot was AI tells, where it scored 3, 3 and 2. One of its intros included "One pass is a signal, not a verdict," a good line and a close cousin of the "it's not X, it's Y" pattern my prompt banned. On an early trial run it also wrote "I have applied for virtual assistant work for a few years now," a personal history nobody gave it, so my test agent added a rule banning experience that isn't in the brief, and every model ran on that revised prompt.
Sonnet's job-sites intro scored only 2 of 5 on voice, while its roadmap intro tied Haiku's for the best single score in the test, 4.2.
On value, Claude Haiku 5.5 came out ahead: 3.80 at about a fourteenth of Sonnet's cost per intro. It stumbled on voice in the job-sites intro too, but its roadmap intro had lines like "My answer is a sequence of AI skills for virtual assistants that builds on what you already know, and none of it requires becoming a developer."
Why the bill doesn't match the price list
Most of what each model billed was reasoning tokens, the hidden thinking a model does before it writes. They made up 85% to 97% of the output tokens each model charged for, on intros of about 120 words. OpenRouter's reasoning tokens documentation says these "are considered output tokens and charged accordingly," so the thinking costs the same rate as the words you keep.
That's how Gemini 3.8 Flash cost more per intro than Sonnet despite a lower list price on OpenRouter: $0.75 per million input tokens and $3.75 per million output, against $2 and $10. Gemini reasoned for about 4,500 tokens per intro and Sonnet for about 1,100, so Gemini came to about $0.018 per intro and Sonnet to $0.014.
At those averages, 1,000 intros would cost about $1.03 with Haiku, $1.75 with DeepSeek, $14.12 with Sonnet and $17.75 with Gemini.
Reasoning share of output tokens: Haiku 90.7%, DeepSeek 91.5%, Sonnet 85.1%, Gemini 97.0%.
The empty-output trap
Set the token limit too low and a reasoning model can return nothing at all, while you still pay for its thinking. The first attempt, before the scored run, used a 1,500-token limit, which looks generous for a 120-word intro. Two of DeepSeek's three intros came back empty because the model spent the whole budget reasoning and never wrote a word. One Haiku intro and all of Gemini's came back cut short.
OpenRouter's docs explain why: on most providers the limit covers reasoning and visible output combined. Their fix is to set max_tokens well above the expected reasoning length, or to cap reasoning with reasoning.max_tokens or a lower reasoning.effort. All four models then ran again at 6,000 tokens, and only that run counts in the scores.
What this test doesn't prove
- It's three prompts, one task type and one run per model: 11 scored intros.
- An AI agent and one judge model did the scoring, with no panel of human readers.
- I tested through OpenRouter only. DeepSeek's own API charges half its peak rate during off-peak hours, and I didn't try that route.
- Prices change: Google's pricing page lists Gemini 3.8 Flash's current rate as valid only through December 31, 2026.
- My research-agent test of Haiku, DeepSeek and GPT-6 Luna measured a different job, and this test doesn't change its findings.
What I'd do with this
For short, routine drafting, I'd start with a cheap model, keep rules like "don't invent experience" in the prompt, check the billed cost per finished piece and pay for a bigger model only where it clearly writes better. That matters most in a pipeline that writes again and again, like the Multi-Agent Content Automation workflow I built in n8n. You can see it with my other agents on my projects page.