Skip to main content

AI models ranked by task

Public human votes, one task at a time, next to our own tests of how far each model goes.

As of Oct 19, 2026, the public-vote leaders by task are Wan 3.0 for text-to-video, MiniMax H3 Max for image-to-video, Seedance 2.5 for reference-to-video, GPT Image 2.5 Sunburst for text-to-image, and GPT Image 2.5 Sunburst for image editing.

Speech quality rankings

Independent human listening votes for text-to-speech. These Elo scores do not enter the Spicy Index.

Observed Sep 29, 2026 · 5 catalog models matched against a source leaderboard of 92 models.

  1. 1Gemini 3.8 Flash TTS
    1,267 Elo95% CI 1,251–1,283

    2,211 listening votes

  2. 2Gemini 3.8 Flash-Lite TTS
    1,241 Elo95% CI 1,225–1,257

    2,180 listening votes

  3. 3MiniMax Speech 2.8 HD
    1,171 Elo95% CI 1,160–1,182

    4,498 listening votes

  4. 4MiniMax Speech 2.8 Turbo
    1,151 Elo95% CI 1,140–1,162

    4,391 listening votes

  5. 5Chatterbox
    1,023 Elo95% CI 1,012–1,034

    4,603 listening votes

Ranks use point estimates. Overlapping 95% confidence intervals do not establish a clear quality difference.

Artificial Analysis

Long-document task compliance

Six fixed shared documents, four questions each: retrieval, multi-step reasoning, updated facts and missing information. Scores require correct answers in strict JSON; they do not enter the Spicy Index.

  1. Claude Opus 5.5

    Updated Sep 30, 2026

    Measured input: 47,180–47,180 tokens

    100.0 / 100

    24/24 answers correct in the required format

  2. DeepSeek V4 Flash

    Updated Sep 30, 2026

    Measured input: 32,087–32,087 tokens

    100.0 / 100

    24/24 answers correct in the required format

  3. DeepSeek V4 Pro

    Updated Sep 30, 2026

    Measured input: 32,087–32,087 tokens

    100.0 / 100

    24/24 answers correct in the required format

  4. GLM 5.2

    Updated Sep 30, 2026

    Measured input: 38,336–38,429 tokens

    100.0 / 100

    24/24 answers correct in the required format

  5. GLM 5.3

    Updated Sep 30, 2026

    Measured input: 38,336–38,429 tokens

    100.0 / 100

    24/24 answers correct in the required format

  6. GLM 5.3 Flash

    Updated Sep 30, 2026

    Measured input: 38,336–38,429 tokens

    100.0 / 100

    24/24 answers correct in the required format

  7. Kimi K3

    Updated Sep 30, 2026

    Measured input: 32,084–32,086 tokens

    100.0 / 100

    24/24 answers correct in the required format

  8. Gemini 3.5 Flash-Lite

    Updated Sep 30, 2026

    Measured input: 53,282–53,282 tokens

    95.8 / 100

    23/24 answers correct in the required format

  9. Claude Opus 4.6

    Updated Sep 30, 2026

    Measured input: 33,716–33,716 tokens

    Samples: 0 refused · 1 JSON-format failures · 0 reached the output limit. Categories may overlap.

    83.3 / 100

    20/24 answers correct in the required format

  10. Gemini 3 Flash Preview

    Updated Sep 30, 2026

    Measured input: 53,282–53,283 tokens

    Samples: 0 refused · 1 JSON-format failures · 0 reached the output limit. Categories may overlap.

    83.3 / 100

    20/24 answers correct in the required format

  11. Grok 4.6

    Updated Sep 30, 2026

    Measured input: 32,211–32,211 tokens

    Samples: 0 refused · 1 JSON-format failures · 0 reached the output limit. Categories may overlap.

    83.3 / 100

    20/24 answers correct in the required format

  12. Claude Sonnet 5

    Updated Sep 30, 2026

    Measured input: 47,178–47,178 tokens

    Samples: 0 refused · 2 JSON-format failures · 0 reached the output limit. Categories may overlap.

    66.7 / 100

    16/24 answers correct in the required format

  13. Gemini 3.1 Pro Preview

    Updated Sep 30, 2026

    Measured input: 53,283–53,283 tokens

    Samples: 0 refused · 2 JSON-format failures · 0 reached the output limit. Categories may overlap.

    66.7 / 100

    16/24 answers correct in the required format

  14. Grok 4.5

    Updated Sep 30, 2026

    Measured input: 32,211–32,211 tokens

    Samples: 0 refused · 4 JSON-format failures · 0 reached the output limit. Categories may overlap.

    33.3 / 100

    8/24 answers correct in the required format

  15. Claude Opus 4.5

    Updated Sep 30, 2026

    Measured input: 33,716–33,745 tokens

    Samples: 0 refused · 5 JSON-format failures · 0 reached the output limit. Categories may overlap.

    16.7 / 100

    4/24 answers correct in the required format

  16. Gemini 3.7 Flash

    Updated Sep 30, 2026

    Measured input: 53,283–53,283 tokens

    Samples: 0 refused · 5 JSON-format failures · 0 reached the output limit. Categories may overlap.

    16.7 / 100

    4/24 answers correct in the required format

  17. Claude Opus 5

    Updated Sep 30, 2026

    Measured input: 47,178–47,178 tokens

    Samples: 6 refused · 6 JSON-format failures · 0 reached the output limit. Categories may overlap.

    0.0 / 100

    0/24 answers correct in the required format

  18. Claude Sonnet 4.5

    Updated Sep 30, 2026

    Measured input: 33,745–33,745 tokens

    Samples: 0 refused · 6 JSON-format failures · 0 reached the output limit. Categories may overlap.

    0.0 / 100

    0/24 answers correct in the required format

  19. Claude Sonnet 4.6

    Updated Sep 30, 2026

    Measured input: 33,716–33,716 tokens

    Samples: 0 refused · 6 JSON-format failures · 0 reached the output limit. Categories may overlap.

    0.0 / 100

    0/24 answers correct in the required format

  20. Gemini 2.5 Flash

    Updated Sep 30, 2026

    Measured input: 53,282–53,282 tokens

    Samples: 0 refused · 6 JSON-format failures · 0 reached the output limit. Categories may overlap.

    0.0 / 100

    0/24 answers correct in the required format

All models receive the same texts: 31,998 reference tokens per document (o200k_base, tiktoken 0.14.0), with a 4,096-token output limit. Code fences count as format failures, even when the enclosed answers are correct. Only six completed samples are published. This does not measure maximum context length or the full long-context suite.

Test method

Short English speech transcription

Six fixed read-speech clips from six speakers: 53.6 seconds and 146 reference words. This small diagnostic does not measure lyrics, noisy speech or overall transcription ability, and does not enter the Spicy Index.

  • Grok STT

    Updated Sep 30, 2026

    2 substitutions · 0 deletions · 0 insertions / 146 reference words

    1.37% WER

    Lower is better

  • HeartMuLa Transcribe

    Updated Sep 30, 2026

    4 substitutions · 0 deletions · 2 insertions / 146 reference words

    4.11% WER

    Lower is better

Word error rate (WER) is total substitutions, deletions and insertions divided by all reference words. Text is normalized consistently for case and punctuation; internal apostrophes are kept. Only complete six-clip results are shown. WER can exceed 100%.

Transcription method

Leaderboard questions