Technical benchmarks rarely help you understand the current landscape. Sometimes they make it even murkier. We decided to figure it out ourselves, drawing exclusively on high-quality independent research.
We evaluated models across three parameters. Long context means how accurately a model handles large documents: reports, contracts, academic texts. Speed covers how quickly it responds and completes a task. Reasoning addresses how deeply and correctly it thinks through complex analytical questions. For each parameter, we used data from independent sources only: Artificial Analysis, Arena (formerly LMArena), LiveBench, and OpenRouter. (We've listed all sources in full at the end of this article.) We took no figures from the providers themselves, since there's always far more marketing than truth in those numbers, and thus the data in our table sometimes contradicts an LLM's established reputation.
We deliberately chose not to evaluate the models' ability to generate literary prose. On one hand, we believe all of them still do it very poorly (see: "Hemingway's Writing Style, as Faked by Five AI Models"). On the other, we have absolutely no confidence in benchmarks in this area.
Model comparison table
Below each model's name, we've listed its price range ($ – $$$$) and three scores based on independent data: EfC (effective context), Sp (speed), and Th (thought, or reasoning ability). Each score reflects how well the model performs on that parameter, with 0 meaning poor and 3 excellent. Scores are for the best mode Artificial Analysis measured. If a provider has already announced a new price, we list that one.
On the EfC scale, a score of 2 is a strong result; 3 goes only to models that stand out by a clear margin. The absence of negatives doesn't mean a model is perfect, just that independent testing hasn't been extensive enough to reveal flaws yet. EfC is based on the AA-LCR long-context test from Artificial Analysis; if there's no data, there's no score, and the card says so. "Preliminary score" means the measurement was taken in a different mode, on an older build, or from a single source. Th is based on the Artificial Analysis Intelligence Index v5 (October 27, 2026), which is stricter than its predecessor and noticeably lowered the scores of many budget models. "New" means the model is appearing in our ranking only now. "Generates images*" indicates that a separate model from the same provider provides the images in question.
Flagships
| Claude Fable 5.1 (NEW) $$$$ EfC 2 · Sp 0 · Th 3 |
Anthropic's top Claude model, released September 1 as the replacement for Fable 5. At launch it took first place in Agent Arena, a ranking where users run models on real long-running tasks: it was praised roughly twice as often as its nearest rival. In Arena's text ranking on October 27 it sits fifth (1504 points), within the margin of error of Opus 5.5 (1509), behind Gemini 4 Argon and Carbon. On the independent Artificial Analysis Intelligence Index (v5) it scores 50: the upper group, but not the leader, with Opus 5.5 (58) and GPT-6 Astra (57) ahead. Fable thinks deeply, handles facts carefully, and holds its own on demanding tasks, though in highly specialized domains it can be just as confidently wrong as the rest of the pack. Use it for serious analysis and hard decisions. On long documents it performs at the level of the other flagships: 82% on AA-LCR, but that was measured in low mode, so the EfC score is preliminary. Downsides: Fable is very expensive and very slow. It costs $10 per million input tokens and $50 per million output tokens; AA's cost per Intelligence Index task averages $10.85, the highest of any model in the table. Speed is about 63 tokens per second, below average, and it takes more than a minute before it starts answering (in xhigh mode). It uses roughly 1.7 times as many tokens as Fable 5, so despite the cache discount it didn't get cheaper than its predecessor. Safety classifiers route cybersecurity and biology requests to the weaker Opus: in AA's tests it accounted for about 4% of output tokens. Fable goes off on tangents, rarely gets answers right on the first try, and tends to play it safe. |
| Claude Opus 5.5 (NEW) $$$ EfC 2 · Sp 1 · Th 3 |
Released September 22 as the replacement for Opus 5.0. The price dropped to $4 / $20 per million tokens — 2.5 times cheaper than Fable 5.1 — while results are higher: 58 on the Artificial Analysis Intelligence Index (v5) versus Fable 5.1's 50, the second-best result in the ranking. In the blind comparison by people on October 27 it is third in Text Arena (1509 points), behind Gemini 4 Argon (1520) and Gemini 4 Carbon (1528, preliminary); the gap to Fable 5.1 (1504) is within the margin of error, and at launch it held first place. In Agent Arena on October 23 it is first. On long documents it performs at the level of the other flagships: 85% on AA-LCR, the top of the main group. It follows instructions precisely and offers an accelerated mode. A sensible choice for serious work when budget matters. Downsides: Opus is very wordy. Running the AA index in max mode used 220 million tokens (the median for models at a similar price is about 70 million), so a task costs $10.04 despite the low per-token price. In max mode it takes about 12 minutes before it starts answering; xhigh mode is almost as good (57 points), costs 40% less, and starts answering in two minutes. Generation speed is above average, about 96 tokens per second. |
| GPT-6.1 Sol (NEW) $$ EfC 2 · Sp 1 · Th 3 generates images* |
Released September 29 as the replacement for GPT-5.6 Sol. A near-flagship at a fifth of the price: 55 on the Artificial Analysis Intelligence Index (v5) versus 57 for GPT-6 Astra, at $2 / $10 per million tokens against $10 / $50. Use it for analysis and reports. On long documents it performs at the level of the other flagships: 83% on AA-LCR. In creative work it's drier than Claude and more tightly structured, with less tonal range. Downsides: it isn't fast. Generation speed is about 80 tokens per second, around the average, but in max mode it takes about five minutes before it starts answering; in xhigh mode, about two and a half (index 52). On the other hand, it's economical: the AA index run used 64 million tokens against a median of 70 million, and a task costs $1.27 — almost eight times less than Opus 5.5. |
| GPT-6 Astra (NEW) $$$$ EfC 2 · Sp 1 · Th 3 generates images* |
OpenAI's flagship, released September 3. It scores 57 on the independent Artificial Analysis Intelligence Index (v5), fourth in the ranking: higher than Fable 5.1 (50), at a lower task cost ($6.24 versus $10.85). In AA's measurements at launch, its hallucination rate fell from 92% to 51%, roughly half that of GPT-5.6 Sol, which matters when you work with facts. It's strongest where the result can be checked: code, computer use, 3D, math. On long documents it performs at the level of the other flagships: 81% on AA-LCR. Downsides: it's expensive — 2.5 times the price of GPT-5.6 Sol ($10 / $50 versus $4 / $20 per million tokens) — and not fast: about 78 tokens per second, below average, and in max mode it takes several minutes before it starts answering. It's weaker with text than with code: in Louis-François Bouchard's editorial test it placed 11th against GPT-5.6 Sol's 6th, and AA recorded a drop of about 80 Elo points on the GDPval-AA professional-tasks test compared with GPT-5.6 Sol. Opinions on style differ: at Every, it was given the draft of the publication's own review, and the editor-in-chief didn't notice it wasn't written by a human. |
| Gemini 3.1 Pro $$ EfC 2 · Sp 2 · Th 2 generates images* |
Google's previous top model: released in February and still in preview (Google hasn't announced a shutdown date), while Gemini 4 Argon, released October 13, has taken the top of the lineup. It stays in the table because of its price: $2 / $12 per million tokens ($4 / $18 for prompts longer than 200,000 tokens), and after December 1 it will be cheaper than Argon ($4 / $20). It accepts text, images, audio, and video, handles long context very well, and understands science and images. Use it for writing and working with large documents, just don't expect deep reasoning. There is no independent measurement for the new AA index (v5) or for AA-LCR, so the EfC and Th scores are carried over from the previous version of this article and are preliminary; the speed is fresh: about 145 tokens per second, faster than average. Downsides: hallucinations and sycophancy are documented across the entire Gemini 3 family. The language is drier and more formal than in previous versions. |
| Gemini 4 Argon (NEW) $$$ EfC 2 · Sp 1 · Th 3 generates images* |
The first model of the Gemini 4 generation and Google's current flagship. Announced on September 30 for a narrow circle of cybersecurity partners; since October 13 it has been available to everyone through the paid API, and in the Gemini app only with an AI Pro or Ultra subscription. In the blind comparison by people it is second in Text Arena (1520 points on October 27), although it scores 50 on the independent Artificial Analysis Intelligence Index (v5) and trails Opus 5.5 (58) and GPT-6 Astra (57). Its main trait is caution: in AA's measurements at launch its hallucination rate is just 15% versus 51% for Astra, but its answer accuracy is lower too (50% versus 63%), meaning the model would rather stay silent than make something up. On long documents it performs at the level of the other flagships: 80% on AA-LCR; it accepts text and images, and its output can run up to a million tokens. Downsides: the price doubles on December 1 — now $2 / $10 per million tokens, then $4 / $20 (the card uses the future price). It's wordy: the AA index run used 110 million tokens (median 70 million), so after the increase a task costs $7.68, more than Astra ($6.24). Speed is about 83 tokens per second, slightly above average, and in high mode it takes less than a minute before it starts answering. |
| Grok 4.7 (NEW) $$ EfC 2 · Sp 1 · Th 1 generates images* |
A new version that replaced Grok 4.6 on September 21: built on a larger base model at the same price — $2 / $6 per million tokens (twice as much for prompts longer than 200,000 tokens). It was released by SpaceXAI, the name the former xAI has used since July, after the merger with SpaceX. It's up to date on the latest news and social media, and generates images exceptionally well. But on the independent Artificial Analysis Intelligence Index (v5) it scores just 35: below average and far behind the flagships (Opus 5.5 has 58, Astra 57). It beat its predecessor only slightly: 46 points versus 44 on the previous version of the index. On long documents it's mid-pack: 77% on AA-LCR, but its context is only 500,000 tokens, half that of its competitors. Downsides: it's wordy — the AA index run used 120 million tokens (median 70 million), so despite the low per-token price a task costs $6.18, almost as much as GPT-6 Astra ($6.24), with far less intelligence. Speed is about 78 tokens per second, slightly below average, and it takes a long time to get going: 30 seconds to a minute before it starts answering. For Grok 4.5, AA recorded 54% hallucinations; we found no comparable measurement for 4.7, so verify facts. SpaceXAI (the former xAI) has significantly dialed back content restrictions, and a string of controversies has followed, including deepfake-related incidents. Don't use Grok where audit and compliance requirements are stringent. Whenever the cost of a mistake is high, verify the model's answers independently. |
| Kimi K3.1 Pro (NEW) $$ EfC 2 · Sp 0 · Th 2 |
An update of K3: released October 23 as a further-trained version of the same architecture (2.8 trillion parameters, 104 billion active, 1M context), and a third cheaper — $2 / $10 per million tokens instead of $3 / $15. Moonshot calls it open and promised to publish the weights by November 4; for now it's available only through the API, while the earlier K3 remains open. It scores 46 on the independent Artificial Analysis Intelligence Index (v5): a good result, but noticeably below the leaders (Opus 5.5 has 58, Astra 57). Still, it costs half as much as Opus 5.5 and a fifth as much as Fable 5.1. It has improved noticeably over K3 at document work: 31% versus 22% on AA's professional-documents test (GDP.pdf). On long context it's at the top of the main group: 86.3% on AA-LCR (K3 had 88.7%), but we found this measurement in only one source, so the EfC score is preliminary. Use it for large documents — just keep in mind that speed is low. Downsides: the slowest in the table — about 31 tokens per second (K3 had 66) — and it takes more than a minute before it starts answering. It's wordy: the AA index run used 110 million tokens (median 70 million), and a task costs $3.68. For text work it matters that in AA's new test of language expressiveness, K3.1 Pro trails K3 noticeably: it wins less often on vocabulary variety, imagery, and natural sound. The weights haven't been published yet, and the license hasn't been announced. |
| Meta Muse Spark 1.3 (NEW) $ EfC 2 · Sp 3 · Th 1 |
Meta's model focused on coding and agentic tasks, released September 2 — the fourth version of the family in five months. It's cheap: $1.25 / $4.25 per million tokens. There's also a Contributor tier that is 12–21 times cheaper ($0.10 / $0.20), but on it Meta trains its models on your prompts, so it isn't suitable for work or confidential texts. One of the fastest in the table, at about 168 tokens per second. On long documents it's mid-pack: 83% on AA-LCR. Its independent Artificial Analysis Intelligence Index (v5) score is 36, below average. On the previous version of the index it scored 61–62, like GPT-5.6 Sol, but the new version, where most tasks are closed, rates it noticeably lower. In Arena's text ranking it is 11th (1495 points on October 27). Downsides: it's wordy — the AA index run used 120 million tokens (median 70 million), and a task costs $2.39. It takes about 30 seconds before it starts answering. According to AA's measurements, version 1.3 more often declines to answer when it's unsure: this lowers its hallucination rate, but its answer accuracy drops too. AA stopped regularly testing this model after the release of Muse Wonder, and Meta hasn't promised to open the weights of version 1.3. |
| Meta Muse Wonder (NEW) $$$ EfC 2 · Sp 2 · Th 2 |
Meta's first "frontier-class" model: released October 12 as the successor to the Muse Spark line, and immediately noticeably more expensive — $4 / $20 per million tokens (Spark 1.3 costs $1.25 / $4.25). In the blind comparison by people it is fourth in Text Arena (1507 points on October 27), in the same bracket as Opus 5.5 and Fable 5.1: the gap to them is within the margin of error. It scores 43 on the independent Artificial Analysis Intelligence Index (v5): above average, but noticeably below the leaders. It accepts text, images, audio, and video; on long documents it's mid-pack (81% on AA-LCR), with 1M context. Available only on the Standard tier: Meta doesn't use your prompts for training. Downsides: the API is still in preview, and the stronger Muse Wonder Full is available only through a Muse Code subscription (from $50 a month) and without an API. It's wordy — the AA index run used 110 million tokens (median 70 million), and a task costs $6.95. It takes about 30 seconds before it starts answering. At launch, AA noted the model's low scores on knowledge tests, partly offset by a low hallucination rate. |
For everyday use
| Claude Sonnet 5.5 (NEW) $$ Sp 2 · Th 3 no long-context data yet |
The core Claude model, released September 28 as the replacement for Sonnet 5 at the same price — $2 / $10 per million tokens. It scores 54 in max mode on the independent Artificial Analysis Intelligence Index (v5), eighth in the ranking and just 4 points below Opus 5.5 at half the per-token price. Fast and capable: about 138 tokens per second, well above average. It holds long context well: Sonnet 5 had 77% on AA-LCR, but there's no independent measurement for 5.5 yet, so we haven't given it an EfC score. It tackles complex, multi-step tasks with confidence and writes in a lively, natural voice. Use it for documents, emails, and texts of medium complexity. In the blind comparison by people it has 1477 points in Text Arena on October 27, noticeably below Opus 5.5 (1509). Downsides: results depend heavily on the mode. In max, the model thinks for about eight minutes before it starts answering, while in high and medium the index is noticeably lower — 41 and 30; xhigh gives 50, starts answering in half a minute, and costs $4.09 per task. It is the wordiest model in AA's ranking (57th out of 57): the index run used 340 million tokens (median 70 million), so in max mode a task costs $10.02 — the same as for Opus 5.5 ($10.04), which is twice as expensive per token. On the AA-Omniscience knowledge test it trailed Opus 5.5 at launch (answer accuracy 54% versus 66%), but it was less likely to invent an answer when it didn't know one (47% versus 59%). |
| GPT-6.1 Terra (NEW) $ Sp 1 · Th 3 no long-context data yet generates images* |
A balanced GPT model, released October 26 as the replacement for GPT-5.6 Terra and half the price of Sol: $1 / $5 per million tokens ($2 / $7.50 for prompts longer than 272,000 tokens). It scores 51 in max mode on the independent Artificial Analysis Intelligence Index (v5) — just 4 below Sol (55). A task costs $0.96, about ten times less than Sonnet 5.5 in max mode ($10.02), at an index of 51 versus 54. It runs fast: about 97 tokens per second, above average — and it generates images. It holds long context well: that was true of the previous version, but there's no independent measurement for 6.1 yet, so we haven't given it an EfC score. Like all OpenAI models, it's more structurally disciplined than those from other companies, making it a solid choice for emails and articles where form matters. Downsides: in ChatGPT it's available only in Work and Codex, not in the regular chat for now (according to OpenAI's release notes). The AA index run used 99 million tokens (median 70 million), and in max mode it takes about 4.5 minutes before it starts answering. Results are lower in less powerful modes: xhigh gives 45, high 42, medium 38. |
| DeepSeek V4 Pro $ EfC 2 · Sp 1 · Th 0 |
DeepSeek's flagship (the August 13 version) with open weights under the MIT license: 1.6 trillion parameters, 1M context, text only. It holds long context at nearly flagship level: 75% on AA-LCR, the lower edge of the main group. It runs faster than previously assumed: about 94 tokens per second, above average among open models. It costs next to nothing. Use it for analysis and research on a tight budget. But it scores just 23 on the independent Artificial Analysis Intelligence Index (v5), while the smaller V4.1 Flash (32) outscores it and costs 3–4 times less. Today it makes more sense to start with V4.1 Flash and test Pro on your own tasks. Downsides: it got more expensive — since August 16 the price depends on the time of day: $1.32 / $3.96 per million tokens during weekday peak hours, and half that ($0.66 / $1.98) the rest of the time. It almost never admits that it doesn't know: on AA-Omniscience its false-answer rate is 94% (V4.1 Flash: 64%). Censorship is baked into the model and cannot be removed even with ordinary self-hosted deployment — previous-generation models (R1) showed ~85% refusal rates on topics the Chinese government dislikes. CTGT researchers showed that a model trained on DeepSeek V4 Flash's answers may not inherit the censorship, but only one such case has been checked. Consumer service data is stored on servers in China, and regulators in several countries have banned downloading the app or using it for government employees. |
Fast and light
| Gemini 3.8 Flash (NEW) $$ Sp 2 · Th 1 no long-context data yet generates images* |
Google's new fast model, released September 2 as the replacement for 3.6 and 3.7 Flash (it's built on 3.7 Flash). It accepts text, images, audio, and video, with 1M context. Through December 31 it costs $0.75 / $3.75 per million tokens; from January 1, 2027 the price doubles ($1.50 / $7.50; the card uses the future price). There's a free tier in the API and AI Studio with strict limits; in the Gemini app it's available only with an AI Pro or Ultra subscription. Fast: about 120 tokens per second, well above average. In the blind comparison by people it is 10th in Text Arena (1496 points on October 27, preliminary). It scores 31 on the independent Artificial Analysis Intelligence Index (v5), below average. Use it for drafts and quick questions. There is no independent long-context data yet. Downsides: it's very wordy — the AA index run used 130 million tokens (median 70 million), so despite the low per-token price a task costs $7.77, more than GPT-6 Astra ($6.24). It takes about 25 seconds before it starts answering. At launch, AA recorded that when the model doesn't know an answer, it makes one up in 55% of cases (Gemini 3 Flash: 91%). |
| Claude Haiku 5.5 (NEW) $ EfC 2 · Sp 3 · Th 1 |
The fastest and most affordable Claude model, released October 7 as the replacement for Haiku 4.5 and ten times cheaper: $0.10 / $0.50 per million tokens for prompts up to 100,000 tokens ($0.50 / $2.50 for longer ones; Haiku 4.5 cost $1 / $5). It has 1M context and accepts text and images. It scores 36 in max mode on the independent Artificial Analysis Intelligence Index (v5): first among models in its class, but noticeably below the flagships (Opus 5.5 has 58). Fast: about 236 tokens per second, twice the average. Use it for short responses, summaries, and simple high-volume tasks. On long documents it's mid-pack: 78% on AA-LCR, but that was measured in xhigh mode rather than max, so the EfC score is preliminary. Downsides: it's very wordy. The AA index run used 300 million tokens against a median of 70 million, and in max mode it takes about six minutes before it starts answering; so the real cost of a task ($0.41) is higher than the per-token price suggests. |
| GPT-6 Luna (NEW) $ EfC 2 · Sp 2 · Th 1 generates images* |
The most affordable model in the new GPT lineup, released September 22 as the replacement for GPT-5.6 Luna: $0.10 / $0.50 per million tokens ($0.20 / $0.75 for prompts longer than 272,000 tokens), half of the promotional prices of GPT-5.6. It has 1M context and accepts text and images. It scores 28 in max mode on the independent Artificial Analysis Intelligence Index (v5): above average for its class, but far from the flagships (Opus 5.5 has 58). Speed is about 118 tokens per second, slightly above average. Use it for simple tasks: condensing text, sorting, and finding what you need. On long documents it's mid-pack: 79% on AA-LCR, but that was measured in high mode rather than max, so the EfC score is preliminary. Downsides: it's wordy and thinks for a long time. The AA index run used 110 million tokens (the median for models at a similar price is 81 million), and in max mode it takes about two minutes (113 seconds) before it starts answering. Even so, a task comes out cheap — $0.15. In ChatGPT it's free for Free and Go users: since September 22 in the desktop app, and since October 8 in the regular chat as well, but the rollout is phased. |
Smart savings
| DeepSeek V4.1 Flash (NEW) $ EfC 2 · Sp 3 · Th 1 |
DeepSeek's smaller model with open weights under the MIT license: released September 10 and updated under the same name on October 17. It has 1M context and accepts text and images. One of the cheapest models in the table: $0.15 / $0.60 per million tokens, twice that ($0.30 / $1.20) at weekday peak hours. It scores 32 on the independent Artificial Analysis Intelligence Index (v5): first among models in its class and higher than the larger V4 Pro (23). Fast: about 170 tokens per second, more than twice the average. Use it for simple tasks and drafts. The same censorship restrictions as V4 Pro apply. On long documents it's mid-pack: 84% on AA-LCR, but that was measured on the September build and there is none for the October one, so the EfC score is preliminary. Downsides: it's very wordy. The AA index run used 170 million tokens (the median for models at a similar price is 100 million), so a task costs $0.60 — four times more than GPT-6 Luna ($0.15), even though the per-token prices are comparable. |
| Qwen 3.8 Max $$ EfC 2 · Sp 0 · Th 1 |
Alibaba's flagship (the September 2 version, first released on August 3). It accepts text, images, and video, with a context of about 1M. It's modestly priced: $2 / $6 per million tokens. It scores 34 on the independent Artificial Analysis Intelligence Index (v5): below average for its class (median 37) and far from the flagships (Opus 5.5 has 58). On long documents it's mid-pack: 80% on AA-LCR. The weights of the text version have been open since August 12 (under a license with restrictions for large services), though AA considers the API variant itself closed. It works well for analysis in Asian languages. Downsides: it's very wordy. The AA index run used 150 million tokens (median 70 million), so a task costs $10.02 — among the most expensive in the ranking, even though the per-token price is moderate. Speed is about 40 tokens per second, half the average — only Kimi K3.1 Pro is slower in the table. |
| GLM 5.3 (NEW) $ EfC 2 · Sp 1 · Th 1 |
An open model — you can deploy it on your own infrastructure. Released August 14 as an update to GLM 5.2 at the same price — $1.40 / $4.40 per million tokens; the weights were published on August 29 (under the GLM-5.3 License: where annual revenue exceeds $10 billion, a safety review by Z.ai is required). 1M context. It scores 32 on the independent Artificial Analysis Intelligence Index (v5): above average for its class (median 23), but far from the flagships (Opus 5.5 has 58) and on par with the cheaper V4.1 Flash. Speed is about 76 tokens per second, around the average. On long documents it's mid-pack: 80% on AA-LCR. Use it if local deployment matters. Limitations: text only — it doesn't accept images. It's very wordy: the AA index run used 140 million tokens (median 99 million), so a task costs $5.01 — the highest in its class — and the per-token price is higher than for most open models. It works well in English and Chinese; other languages are much weaker. |
| MiMo V2.6 Pro (NEW) $ EfC 2 · Sp 0 · Th 1 |
Xiaomi's open model (MIT license), released September 21 as the replacement for MiMo V2.5 Pro: 1 trillion parameters, 42 billion active, 1M context; it accepts text, images, audio, and video. Independent benchmarks confirm middling reasoning and modest speed: it scores 30 on the Artificial Analysis Intelligence Index (v5), above average for open models in its class (median 23) but noticeably below the leaders (Opus 5.5 has 58); about 49 tokens per second. The price is among the lowest in the table: $0.435 / $0.87 per million tokens. Use it for simple tasks on a tight budget. On long documents it's at the top of the main group: 86% on AA-LCR. Downsides: it's very wordy. The AA index run used 130 million tokens (median for open models of the same class: 99 million), but a task still costs just $0.26 thanks to the low per-token price. It takes about three seconds before it starts answering. |
| Mistral Large 4 (NEW) $ Sp 3 · Th 1 no long-context data yet |
A European model, Mistral Large 4 offers broad language support and strict privacy rules from servers in the EU. Released October 6 in public preview as the replacement for Mistral Large 3; the weights were published on October 29 (under the Mistral Ecosystem Open-Weight License: commercial use with restrictions). 1.1 trillion parameters, 52 billion active, 1M context; it accepts text and images. The price is $0.70 / $2.10 per million tokens. Its independent Artificial Analysis Intelligence Index (v5) score is low: 26 in high mode (Opus 5.5 has 58). Fast: about 161 tokens per second. Use it when language coverage and privacy are the priority. Downsides: on the Mistral documentation page the model is still listed as a public preview. It's wordy: the AA index run used 110 million tokens (median 99 million), and a task costs $2.06. |
| Step 5 Preview (NEW) $ EfC 3 · Sp 2 · Th 1 |
StepFun's open model (600 billion parameters, 27 billion active): released September 18, with the weights published on Hugging Face under StepFun Model License 1.0 — Apache 2.0-style, but with additional terms on attribution and commercial use (read the license text before production use). 1M context; it accepts text and images. The price is $1 / $2.70 per million tokens. It scores 29 on the independent Artificial Analysis Intelligence Index (v5): noticeably below the flagships (Opus 5.5 has 58). Fast: about 146 tokens per second. On long documents it's in the top group: 88.3% on AA-LCR, one of the best results among models with a measurement. Use it for work with long documents when you need an open model at a moderate price. Downsides: it's very wordy. The AA index run used 130 million tokens (median 99 million), so a task costs $2.35, and the per-token price is higher than for most open models (median: $0.47 / $1.89). The word Preview is in the name: StepFun has released the model as a preliminary version. |
Three models are left out of the table. Gemini 4 Carbon tops the Artificial Analysis index (59), but we never figured out whether its public API is open: AA's own data on this point contradict each other. Muse Wonder Full (50) is available only through a Muse Code subscription, with no API. And on Artificial Analysis, GLM 5 Turbo is marked "deprecated," and it's already gone from Z.ai's price list.
Behind the table
Every metric in the table above has a specific source behind it. Artificial Analysis measures intelligence index, AA-LCR long-context retrieval test, speed, and latency. Arena provides blind comparison: real people pick the best answer without knowing which model produced it. LiveBench offers objective tests designed to be leak-proof, while OpenRouter enables current API price verification. We consider providers’ own figures unverified and did not use them in the scores.
What didn't fit in the cards but matters
Our review unearthed a few surprising facts. Among fully tested models, the best long-context results belong to Kimi K3.1 Pro (86.3%) and MiMo V2.6 Pro (86%), at a lower price than the top Claude models. Only Step 5 Preview ranks higher (88.3%), but we found that measurement in a single source. DeepSeek V4 Pro holds long context at flagship level (75%) for minimal money, but the new intelligence index sharply lowered it (23 points), while the smaller V4.1 Flash scores 32. Mistral Large 4 also scores low on the new index.
On long-context performance, the flagships cluster tightly together. Nearly all score between 75–86% on the AA-LCR test, with gaps within the group comparable to the margin of error. That's why only a model with a clear lead earned EfC 3: Step 5 Preview. The rest honestly get EfC 2 — including both top-tier Claude models: their reputation as the best long-context models wasn't confirmed by independent testing.
The AA-LCR test simulates professional document work: finding and assembling an answer from multiple fragments scattered across a large corpus. Each question draws on roughly one hundred thousand tokens of reports, scientific papers, legal texts, and reviews. There are currently no other independent platforms that have measured long-context performance for this generation of models — Fiction.LiveBench, RULER, and LongBench don't cover it. That's a gap worth knowing about. For some of the new models (Sonnet 5.5, GPT-6.1 Terra, Gemini 3.8 Flash, Mistral Large 4) there is no AA-LCR measurement yet, so we haven't given them EfC scores.
Results vary by effort mode. At maximum reasoning effort, some models pick up a few points on long-context tasks. The scores in the table are for the best mode Artificial Analysis measured. If a measurement was taken in a different mode (for example, low for Fable 5.1 or xhigh for Haiku 5.5), the score is marked preliminary.
Why humans and benchmarks pick different leaders
Benchmarks measure specific skills like math, code, science, and instruction-following: narrow tasks with correct answers. Arena, on the other hand, tracks whether a real person actually likes the model's response. That puts style, structure, tone, the feeling of "intelligence," and readability front and center. On October 27, Gemini 4 Carbon (preliminary) and Argon lead Text Arena, even though on the Artificial Analysis Intelligence Index (v5) Argon scores 50 against Opus 5.5's 58 and GPT-6 Astra's 57. Claude Fable 5.1 paints a similar picture: fifth place with human voters at the same 50 on the index. Fable is a model Anthropic clearly tuned, in part, for the sheer pleasure of conversation. It writes beautifully, thinks out loud, and lays out its reasoning coherently — it's genuinely easy to talk to.
Arena also has a documented verbosity bias, as people tend to prefer longer answers even when shorter ones are more accurate. OpenAI's models appear to exploit this actively.
Independent research confirms that the correlation between benchmark ratings and human perception is low. The model you pick as "best" in a blind test may objectively perform worse on benchmarks — it just feels better to talk to. This human-centered approach has real strengths of its own. Which car would you rather drive: the one that gets you from A to B more comfortably and quickly, or the one that posts better lap times at the track?
That doesn't mean benchmarks are bad or Arena's rankings are wrong. It just means "best model" depends on what you're measuring. That's precisely what this article is about — a single composite score doesn't tell the whole story.
What this means
Which model for which job
For analysis and complex decisions, use Claude Fable 5.1 or Opus 5.5 if budget matters. For large documents on a tight budget, go with Kimi K3.1 Pro or MiMo V2.6 Pro. For a steady stream of daily tasks, pick Sonnet 5.5 or GPT-6.1 Terra. For drafts and quick turnarounds, you can choose any model from the Fast and light section.
Independent testing upends the conventional wisdom
Some results diverge from what's generally taken for granted. Claude doesn't lead on long context. The new intelligence index sharply lowered the scores of many budget models, including DeepSeek V4 Pro. For Grok 4.5, AA recorded 54% hallucinations — more than GPT-6 Astra (51%) and Gemini 4 Argon (15%); there is no comparable measurement for 4.7.
Limitations worth keeping in mind
We drew on four independent sources. For long context, AA-LCR is the only test available for this year's models — there simply aren't others. Data on niche and newer models is thin, which is reflected directly in the model cards. On October 27 the Artificial Analysis Intelligence Index switched to version 5, so Th scores can't be directly compared with earlier data.
The table is a starting point
No table replaces testing on your own workload. Prices shift (GPT and DeepSeek got cheaper this summer, for instance), models get updated, and each has its own quirks. Where you go from here is up to you — your own experience is what counts most.
Data updated in November 2026.
Sources
References cited in this piece. Last verified on the published or revision date.
- 01
- 02
- 03
- 04
- 05
- 06
- 07
- 08
- 09
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20