🔍 Read the full analysis: My News Pipeline Test Found A Better AI Model Than The One I Run on ThorstenMeyerAI.com
Get business pricing on monitors, keyboards and dev gear
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
A test of 71 model configurations on one publisher’s news-writing workflow ranked GPT-6 Astra highest, while GPT-6 Luna was selected for routine production at a much lower reported cost. The publisher also removed two fallback models after tests found fabricated quotes or unusable output; the results are specific to this workflow and have not been independently verified.
Thorsten Meyer says a test of 71 AI model configurations on his news-writing pipeline found GPT-6 Astra produced the highest-rated articles, while he kept the much cheaper GPT-6 Luna in production. The test also led him to remove two backup models after they generated fabricated quotes or failed to produce usable articles, a reminder that model choice affects both the quality and reliability of automated publishing.
Meyer, writing on ThorstenMeyerAI.com on October 13, said he ran the evaluation from September 8 to October 13, 2026, using the production prompt and real news sources. The configurations included API and subscription models, free-tier options accessed through OpenRouter, and open-weight models running on his own hardware. Each source was turned into both an English and a German article, then evaluated by two AI judges: GPT-6.1 Sol and Claude Opus 5.5.
In the standard test, GPT-6 Astra scored 4.00 out of 5 in both languages and received a 5.00 source-fidelity score. Across a further 10 articles in a robustness run, it averaged 3.93. Meyer reported a price of about $0.12 per article and an average generation time of roughly 50 seconds. GPT-6 Luna scored 3.67 in English and 3.68 in German in the standard test; its robustness-run score was 3.67. Meyer said it cost about $0.0012 per article in his comparison and took about 17 seconds.
Meyer said Luna remains the production choice because it offers acceptable quality at a far lower reported cost, while Astra’s additional score did not justify its price for his publishing volume. The test also found that raising Luna’s reasoning effort did not produce a dependable quality improvement: the maximum setting reportedly cost about 12 times as much as the default and took about 12 times as long, with score differences within the author’s stated margin of noise.
Infographic 01 · Who writes the best news article?
The best writer is not the one in production
71 model configurations, one real job: turn a news source into an article for an English and a German edition. GPT-6 Astra writes the best articles. GPT-6 Luna writes them for about a hundredth of the price.
Robustness run: 12 real news sources, English edition
Dot = mean score (1–5), bar = 95% bootstrap interval. German edition mean below each score. Overlapping bars are not a real difference.
Two judges, two caveats: GPT-6.1 Sol and Claude Opus 5.5 grade every article and were also tested as writers, so they partly grade themselves. Claude models ran with effort medium unless noted.
Source: Thorsten Meyer AI model-scout benchmark, 8 Sep – 13 Oct 2026. Task: write a news article from a real source with the production prompt, English and German editions. Judges: GPT-6.1 Sol (reasoning high) and Claude Opus 5.5, both with the source in context; score 1–5, mean of both judges. Robustness run: 2 test articles plus 10 more real sources, mostly short like production; mean with 95% bootstrap interval.
Powered by Thorsten Meyer AI
Quality Versus Cost in Daily Publishing
The results illustrate why a general model ranking may not predict which system works best for a particular publishing task. A news pipeline needs more than fluent copy: it must preserve what the source says, avoid unsupported details and return output that can be used by the publishing system. Meyer’s comparison suggests that task-specific testing can change the deployment decision, even when a different model leads on a broader or unrelated benchmark.
The fallback findings have a direct operational implication. A backup is not harmless if it is activated when the primary model fails: it can publish unsupported quotations or break the output format. Meyer said GPT-4.1 nano and gpt-oss-120b were removed after the tests, citing poor source-fidelity results and invalid outputs. That is his account of the test, not an independent audit of the models or a finding about all uses of them.
Cost also matters at scale. Meyer’s reported figures put Astra’s per-article price at roughly 100 times Luna’s, while the score difference in the additional run favored Astra. For other publishers, the useful takeaway is not that Luna is universally preferable, but that quality thresholds, source types, language needs, latency and failure handling should be tested against the outlet’s own workflow.
As an affiliate, we earn on qualifying purchases.
How Meyer Tested the Models
The evaluation used two source texts: a 6,340-character product filing and a 619-character political news item. Meyer said the shorter item was the more representative challenge because much of his production work begins with brief sources. Models had to produce articles in English and German, with the original source available to both judges.
The judges scored source fidelity, fabrication, clarity about what was known, news value, structure and polish; German output also received a language-quality score. Meyer said an article counted only if it passed the same repair step used in production and returned valid output. He treated score gaps below about 0.3 points in the two-article standard test as potentially inconclusive, and relied more heavily on the robustness run for ranking. The report says the results include 95% bootstrap intervals, but the supplied account does not provide full scoring data for every configuration.
Among the other reported results, Claude Opus 5.5 scored 3.46 in English and 3.39 in German in the standard test, and 3.52 in the robustness run. Meyer said Claude Sonnet 5.5 was close to Luna’s standard scores but cost about $0.036 per article at an API-equivalent rate. These findings reflect article writing from the tested sources, not coding ability or performance across all tasks.
“The right model is the cheapest one that clears your quality bar reliably, not the one at the top of the chart.”
— Thorsten Meyer, ThorstenMeyerAI.com
As an affiliate, we earn on qualifying purchases.
Limits of the Reported Results
The results come from one publisher’s benchmark, using two initial sources and a 10-article robustness run for the strongest configurations. The article does not report independent replication, human-editor scores, or enough detail to assess how representative the sources are of other newsrooms’ output. The judges were also AI models, one of which—Claude Opus 5.5—was itself among the evaluated systems.
The reported costs and timings may depend on Meyer’s specific access, settings and pipeline and should not be treated as fixed prices or service guarantees. It is also unclear from the supplied account how every model configuration performed across the full source set, or how the benchmark handles changes in model versions over time. The test supports conclusions about this workflow as described; it does not establish a universal ranking of news-writing models.
As an affiliate, we earn on qualifying purchases.
Luna Remains the Production Choice
Meyer said GPT-6 Luna remains in production at the default reasoning setting, while GPT-6 Astra was rejected for routine news generation because of its reported cost. He said GPT-4.1 nano and gpt-oss-120b were removed from the failover route on October 7. The source account says Luna now runs first, but its final description of the backup arrangement is incomplete, so the current fallback sequence is not clear.
The next useful check would be continued testing on a larger, varied set of real sources, including repeat evaluations as models and prices change. Meyer’s report does not give a schedule for another benchmark or say whether the pipeline’s scores will be published in full. Until then, the reported rankings should be read as a snapshot of one production system and its tested configurations.
As an affiliate, we earn on qualifying purchases.
Key Questions
Which model scored highest in Meyer’s test?
GPT-6 Astra led the reported results, scoring 4.00 out of 5 in both English and German in the standard test and 3.93 across 12 articles in the robustness run.
Why did Meyer keep GPT-6 Luna in production?
Meyer said Luna’s quality was sufficient for his workflow at a much lower reported price: about $0.0012 per article, compared with about $0.12 for Astra. Those figures describe his test setup, not guaranteed prices for other users.
What did the test find about higher reasoning effort?
Meyer reported no dependable quality improvement from raising Luna’s reasoning setting. The maximum setting cost and took about 12 times as much as the default, while the score differences fell within the test’s stated noise range.
Why were two fallback models removed?
Meyer said GPT-4.1 nano received very low source-fidelity scores and produced fabricated quotations, while gpt-oss-120b generated no valid articles in four attempts and also invented named quotes. These are findings from his test, not independent assessments of those models in other settings.
Do these results show which model is best for every newsroom?
No. The evaluation tested one publisher’s prompt and pipeline, a small set of source texts, and English and German output. Other newsrooms would need to test their own sources, editorial standards, costs and fallback behavior before drawing a similar conclusion.
Source: ThorstenMeyerAI.com
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
