Modality: llm_chat · full deep dive — every ranked model, test result, and artifact.
Ranked by confidence-adjusted score (single/zero-sample, non-curated scores floored; curated empirical scores trusted as-is).
| # | Model | Provider | Adj. Score | Raw | Evidence |
|---|---|---|---|---|---|
| 1 | claude-sonnet-4-6 | claude_desktop | 0.773 | 0.850 | n=50 |
| 2 | claude-sonnet-4-6::think=fast | claude_code | 0.545 | 1.000 | n=6 |
| 3 | claude-sonnet-4-6::think=deep | claude_code | 0.545 | 1.000 | n=6 |
| 4 | gpt-5.5::think=deep | codex | 0.525 | 0.963 | n=6 |
| 5 | gpt-5.5::think=fast | codex | 0.525 | 0.963 | n=6 |
| 6 | claude-opus-4-7::think=fast | claude_code | 0.477 | 0.875 | n=6 |
| 7 | claude-opus-4-7::think=deep | claude_code | 0.477 | 0.875 | n=6 |
| 8 | claude-opus-4-8::think=fast | claude_code | 0.430 | 0.788 | n=6 |
| 9 | gpt-5.3-codex-spark::think=deep | codex | 0.382 | 0.700 | n=6 |
| 10 | claude-opus-4-8::think=deep | claude_code | 0.382 | 0.700 | n=6 |
| 11 | gpt-5.4::think=fast | codex | 0.334 | 0.612 | n=6 |
| 12 | gpt-5.4::think=deep | codex | 0.334 | 0.612 | n=6 |
| 13 | codex-auto-review | codex | 0.286 | 0.525 | n=6 |
| 14 | gpt-5.3-codex-spark::think=fast | codex | 0.286 | 0.525 | n=6 |
| 15 | gpt-5.4 | codex | 0.150 | 1.000 | provisional (n≤1) |
| 16 | gpt-5.5 | codex | 0.150 | 1.000 | provisional (n≤1) |
| 17 | claude-sonnet-4-5-20250929 | claude_code | 0.119 | 0.796 | provisional (n≤1) |
| 18 | claude-opus-4-1-20250805 | claude_code | 0.107 | 0.716 | provisional (n≤1) |
| 19 | claude-opus-4-6 | claude_code | 0.101 | 0.675 | provisional (n≤1) |
| 20 | claude-sonnet-4-20250514 | claude_code | 0.096 | 0.643 | provisional (n≤1) |
| 21 | claude-opus-4-20250514 | claude_code | 0.092 | 0.612 | provisional (n≤1) |
| 22 | claude-opus-4-5-20251101 | claude_code | 0.089 | 0.596 | provisional (n≤1) |
| 23 | claude-opus-4-8 | claude_code | 0.063 | 0.422 | provisional (n≤1) |
| 24 | gemini-2.5-flash | google_gemini | 0.043 | 0.287 | provisional (n≤1) |
| 25 | gpt-5.3-codex-spark | codex | 0.032 | 0.210 | provisional (n≤1) |
| 26 | claude-sonnet-4-6 | claude_code | 0.029 | 0.196 | provisional (n≤1) |
| 27 | claude-opus-4-7 | claude_code | 0.019 | 0.126 | provisional (n≤1) |
| 28 | gemini-2.5-pro | google_gemini | 0.000 | 0.002 | provisional (n≤1) |
| 29 | openai.gpt-4 | 0.000 | 0.000 | provisional (n≤1) | |
| 30 | gpt-4-turbo | openai | 0.000 | 0.000 | provisional (n≤1) |
| 31 | openai/gpt-4 | openrouter | 0.000 | 0.000 | provisional (n≤1) |
| 32 | openai_gpt4 | 0.000 | 0.000 | provisional (n≤1) | |
| 33 | model:openai/gpt-4o | 0.000 | 0.000 | provisional (n≤1) | |
| 34 | model_gpt4 | 0.000 | 0.000 | provisional (n≤1) | |
| 35 | claude-3.5-sonnet | 0.000 | 0.000 | provisional (n≤1) | |
| 36 | anthropic_claude3 | 0.000 | 0.000 | provisional (n≤1) | |
| 37 | claude-3-5-sonnet | 0.000 | 0.000 | provisional (n≤1) | |
| 38 | claude-sonnet-4 | 0.000 | 0.000 | provisional (n≤1) | |
| 39 | gpt4o | 0.000 | 0.000 | provisional (n≤1) | |
| 40 | model-gpt-4o-mini | 0.000 | 0.000 | provisional (n≤1) | |
| 41 | llm_gpt4 | 0.000 | 0.000 | provisional (n≤1) | |
| 42 | model_claude3 | 0.000 | 0.000 | provisional (n≤1) | |
| 43 | claude-sonnet-4-6 | anthropic | 0.000 | 0.000 | provisional (n≤1) |
| 44 | gpt-4o | openai | 0.000 | 0.000 | provisional (n≤1) |
| 45 | candidate_openai_large | 0.000 | 0.000 | provisional (n≤1) | |
| 46 | anthropic.claude-3-opus | 0.000 | 0.000 | provisional (n≤1) | |
| 47 | claude-3-opus-20240229 | 0.000 | 0.000 | provisional (n≤1) | |
| 48 | claude-3.5-haiku | 0.000 | 0.000 | provisional (n≤1) | |
| 49 | google.gemini-1.5-pro | 0.000 | 0.000 | provisional (n≤1) | |
| 50 | fal-ai/gpt-4o-mini | 0.000 | 0.000 | provisional (n≤1) | |
| 51 | model:anthropic/claude-3-5-sonnet | 0.000 | 0.000 | provisional (n≤1) | |
| 52 | anthropic/claude-3 | 0.000 | 0.000 | provisional (n≤1) | |
| 53 | claude-3-5-sonnet-20241022 | 0.000 | 0.000 | provisional (n≤1) | |
| 54 | m_001 | 0.000 | 0.000 | provisional (n≤1) | |
| 55 | claude-opus-4 | 0.000 | 0.000 | provisional (n≤1) | |
| 56 | anthropic/claude-3-opus-20240229 | 0.000 | 0.000 | provisional (n≤1) | |
| 57 | claude-3-5-haiku-20241022 | 0.000 | 0.000 | provisional (n≤1) | |
| 58 | llama-3-70b-chat | 0.000 | 0.000 | provisional (n≤1) | |
| 59 | gemini-ultra | 0.000 | 0.000 | provisional (n≤1) | |
| 60 | google_gemini_pro | 0.000 | 0.000 | provisional (n≤1) | |
| 61 | anthropic.claude-3-sonnet | 0.000 | 0.000 | provisional (n≤1) | |
| 62 | gemini-pro-1.5 | 0.000 | 0.000 | provisional (n≤1) | |
| 63 | anthropic/claude-2 | 0.000 | 0.000 | provisional (n≤1) | |
| 64 | model-claude-3-haiku | 0.000 | 0.000 | provisional (n≤1) | |
| 65 | model:google/gemini-1-5-pro | 0.000 | 0.000 | provisional (n≤1) | |
| 66 | anthropic/claude-3-haiku | openrouter | 0.000 | 0.000 | provisional (n≤1) |
| 67 | llm_gemini1.5 | 0.000 | 0.000 | provisional (n≤1) | |
| 68 | model_gemini | 0.000 | 0.000 | provisional (n≤1) | |
| 69 | claude-3-opus | 0.000 | 0.000 | provisional (n≤1) | |
| 70 | claude-opus-4-8 | anthropic | 0.000 | 0.000 | provisional (n≤1) |
| 71 | candidate_anthropic_claude3 | 0.000 | 0.000 | provisional (n≤1) | |
| 72 | gemini-2.0-pro | 0.000 | 0.000 | provisional (n≤1) | |
| 73 | mistral-large-2407 | 0.000 | 0.000 | provisional (n≤1) | |
| 74 | mistral-large | 0.000 | 0.000 | provisional (n≤1) | |
| 75 | gemini-2.0-flash | 0.000 | 0.000 | provisional (n≤1) | |
| 76 | google/gemini-1.5-pro-001 | 0.000 | 0.000 | provisional (n≤1) | |
| 77 | claude-3.5-sonnet-20241022 | 0.000 | 0.000 | provisional (n≤1) | |
| 78 | gemini-pro | 0.000 | 0.000 | provisional (n≤1) | |
| 79 | google/gemini-pro | 0.000 | 0.000 | provisional (n≤1) | |
| 80 | llama-3.1-70b | 0.000 | 0.000 | provisional (n≤1) | |
| 81 | fal-ai/llama-3.1-8b | 0.000 | 0.000 | provisional (n≤1) | |
| 82 | model_openai_gpt4 | 0.000 | 0.000 | provisional (n≤1) | |
| 83 | llama-3.3-70b | 0.000 | 0.000 | provisional (n≤1) | |
| 84 | claude-3-haiku | 0.000 | 0.000 | provisional (n≤1) | |
| 85 | gpt-4 | openai | 0.000 | 0.000 | provisional (n≤1) |
| 86 | model-llama-3-8b | 0.000 | 0.000 | provisional (n≤1) | |
| 87 | claude-v3 | 0.000 | 0.000 | provisional (n≤1) | |
| 88 | default_chat | 0.000 | 0.000 | provisional (n≤1) | |
| 89 | gpt-3.5-turbo | openai | 0.000 | 0.000 | provisional (n≤1) |
| 90 | claude-2 | 0.000 | 0.000 | provisional (n≤1) | |
| 91 | llm_claude3 | 0.000 | 0.000 | provisional (n≤1) | |
| 92 | llm_mistral_large | 0.000 | 0.000 | provisional (n≤1) | |
| 93 | llama-3-1-405b | 0.000 | 0.000 | provisional (n≤1) | |
| 94 | <model_id_placeholder2> | 0.000 | 0.000 | provisional (n≤1) | |
| 95 | gemini-1-5-pro | 0.000 | 0.000 | provisional (n≤1) | |
| 96 | claude-3-5-haiku | 0.000 | 0.000 | provisional (n≤1) | |
| 97 | together/llama-2-70b | 0.000 | 0.000 | provisional (n≤1) | |
| 98 | anthropic/claude-3-sonnet | 0.000 | 0.000 | provisional (n≤1) | |
| 99 | claude-3-sonnet-20240229 | 0.000 | 0.000 | provisional (n≤1) | |
| 100 | gemini-1.5-pro-001 | 0.000 | 0.000 | provisional (n≤1) | |
| 101 | gpt-4o-mini | openai | 0.000 | 0.000 | provisional (n≤1) |
| 102 | claude-3-sonnet | 0.000 | 0.000 | provisional (n≤1) | |
| 103 | m_002 | 0.000 | 0.000 | provisional (n≤1) | |
| 104 | gpt4 | 0.000 | 0.000 | provisional (n≤1) | |
| 105 | claude3 | 0.000 | 0.000 | provisional (n≤1) | |
| 106 | gemini-1.5-pro | 0.000 | 0.000 | provisional (n≤1) | |
| 107 | llama-3.5 | 0.000 | 0.000 | provisional (n≤1) | |
| 108 | model_001 | 0.000 | 0.000 | provisional (n≤1) | |
| 109 | candidate_google_palm | 0.000 | 0.000 | provisional (n≤1) | |
| 110 | cohere/command-r-plus-08-2024 | openrouter | 0.000 | 0.000 | provisional (n≤1) |
| 111 | cohere/command-r-plus | 0.000 | 0.000 | provisional (n≤1) | |
| 112 | meta/llama-3-70b | 0.000 | 0.000 | provisional (n≤1) | |
| 113 | llama-3.1-8b | 0.000 | 0.000 | provisional (n≤1) | |
| 114 | command-r | 0.000 | 0.000 | provisional (n≤1) | |
| 115 | anthropic/claude-3-opus | 0.000 | 0.000 | provisional (n≤1) | |
| 116 | meta.llama-3-70b | 0.000 | 0.000 | provisional (n≤1) | |
| 117 | model_anthropic_claude3 | 0.000 | 0.000 | provisional (n≤1) | |
| 118 | openai/gpt-4o | replicate | 0.000 | 0.000 | provisional (n≤1) |
| 119 | mixtral-8x7b | 0.000 | 0.000 | provisional (n≤1) | |
| 120 | meta/llama-3-70b-chat | 0.000 | 0.000 | provisional (n≤1) | |
| 121 | claude-haiku-4-5 | 0.000 | 0.000 | provisional (n≤1) | |
| 122 | gemini-1.5-flash | 0.000 | 0.000 | curated | |
| 123 | llama-3-70b | 0.000 | 0.000 | provisional (n≤1) | |
| 124 | m_003 | 0.000 | 0.000 | provisional (n≤1) | |
| 125 | llama3-70b | 0.000 | 0.000 | provisional (n≤1) | |
| 126 | model_002 | 0.000 | 0.000 | provisional (n≤1) | |
| 127 | mistral-large-2 | 0.000 | 0.000 | provisional (n≤1) | |
| 128 | claude-3-haiku-20240307 | anthropic | 0.000 | 0.000 | provisional (n≤1) |
| 129 | mistral-large-latest | mistral | 0.000 | 0.000 | provisional (n≤1) |
| 130 | meta-llama/llama-3.1-70b-instruct | openrouter | 0.000 | 0.000 | provisional (n≤1) |
| 131 | claude-haiku-4-5-20251001 | anthropic | 0.000 | 0.000 | provisional (n≤1) |
| 132 | llama-3 | 0.000 | 0.000 | provisional (n≤1) | |
| 133 | mistral.mixtral-8x22b | 0.000 | 0.000 | provisional (n≤1) | |
| 134 | meta-llama-3-70b | 0.000 | 0.000 | provisional (n≤1) | |
| 135 | llama-3.1-70b-instruct | 0.000 | 0.000 | provisional (n≤1) | |
| 136 | m_004 | 0.000 | 0.000 | provisional (n≤1) | |
| 137 | default | 0.000 | 0.000 | curated | |
| 138 | llama-3-8b | 0.000 | 0.000 | provisional (n≤1) | |
| 139 | llama-2-70b | 0.000 | 0.000 | provisional (n≤1) | |
| 140 | mistral-7b | 0.000 | 0.000 | provisional (n≤1) | |
| 141 | claude-haiku | 0.000 | 0.000 | provisional (n≤1) | |
| 142 | mixtral-8x22b | 0.000 | 0.000 | provisional (n≤1) | |
| 143 | GOLD_ENTRY | 0.000 | 0.000 | provisional (n≤1) | |
| 144 | mistral-small-latest | mistral | 0.000 | 0.000 | provisional (n≤1) |
| 145 | claude-haiku-3 | 0.000 | 0.000 | provisional (n≤1) | |
| 146 | llama-3-70b-instruct | 0.000 | 0.000 | provisional (n≤1) | |
| 147 | claude-sonnet-3-5 | 0.000 | 0.000 | provisional (n≤1) | |
| 148 | llama-3.1-70b-versatile | 0.000 | 0.000 | provisional (n≤1) | |
| 149 | claude-haiku-3-5 | 0.000 | 0.000 | provisional (n≤1) | |
| 150 | gemini-flash | 0.000 | 0.000 | provisional (n≤1) |
No benchmark outputs recorded for this niche yet.