TAAS
Overview Niche Catalog
Generated Objects Routing & Audit ← Niche Catalog
coding Modality: llm_chat · full deep dive — every ranked model, test result, and artifact.
Ranked Models Ranked by confidence-adjusted score (single/zero-sample, non-curated scores floored; curated empirical scores trusted as-is).
# Model Provider Adj. Score Raw Evidence 1 deepseek-chat deepseek 0.996 0.997 n=12836 2 claude-sonnet-4-6 anthropic 0.920 0.920 curated 3 claude-opus-4-6 anthropic 0.840 0.840 curated 4 deepseek-ai/DeepSeek-R1 together 0.753 0.753 curated 5 command-r-08-2024 cohere 0.753 0.753 curated 6 Qwen/Qwen2.5-7B-Instruct-Turbo together 0.753 0.753 curated 7 deepseek-ai/DeepSeek-V3.1 together 0.753 0.753 curated 8 claude-haiku-4-5-20251001 anthropic 0.734 0.734 n=16760 9 deepseek-reasoner deepseek 0.700 0.700 curated 10 magistral-small-latest mistral 0.583 0.583 curated 11 mistral-large-latest mistral 0.417 0.417 curated 12 claude-sonnet-4-5-20250929 anthropic 0.363 0.363 n=12137 13 gpt-4.1-mini openai 0.344 0.344 curated 14 gpt-4o-mini openai 0.344 0.344 curated 15 grok-3-mini xai 0.178 0.400 n=4 16 sonar perplexity 0.167 0.377 n=4 17 sonar-pro perplexity 0.167 0.377 n=4 18 grok-3 xai 0.167 0.377 n=4 19 gpt-4o openai 0.150 0.150 curated 20 meta-llama/Llama-3.3-70B-Instruct-Turbo together 0.150 0.150 curated 21 gemini-2.5-pro google_gemini 0.109 0.109 curated 22 gemini-2.5-flash google_gemini 0.102 0.102 curated 23 o3 openai 0.000 0.000 provisional (n≤1) 24 o3-mini openai 0.000 0.000 provisional (n≤1)
Test Results Showing top 40 of 342 benchmark outputs by weighted score.
deepseek-chat · deepseek · weighted 10.0 · syntactic_validity=10.0 · $8e-06 · 787ms ```python
def add(a, b):
return a + b
```
claude-haiku-4-5-20251001 · anthropic · weighted 10.0 · syntactic_validity=10.0, naming_conventions=10.0 · $0.000238 · 633ms ```python
class UserProfile:
def __init__(self):
self.email = None
def update_email(self, new_email):
self.email = new_email
```
mistral-small-latest · mistral · weighted 10.0 · syntactic_validity=10.0 · $0.0 · 899ms ```python
def add(a, b):
return a + b
```
deepseek-chat · deepseek · weighted 10.0 · syntactic_validity=10.0, naming_conventions=10.0 · $1.2e-05 · 979ms ```python
class UserProfile:
def update_email(self, new_email):
self.email = new_email
```
gpt-4o · openai · weighted 10.0 · syntactic_validity=10.0, naming_conventions=10.0 · $0.000485 · 744ms ```python
class UserProfile:
def __init__(self, email):
self.email = email
def update_email(self, new_email):
self.email = new_email
```
gemini-2.5-flash · google_gemini · weighted 10.0 · syntactic_validity=10.0 · $6.6e-05 · 633ms ```python
def add(a, b):
return a + b
```
Qwen/Qwen2.5-7B-Instruct-Turbo · together · weighted 10.0 · syntactic_validity=10.0, naming_conventions=10.0 · $0.0 · 657ms ```python
class UserProfile:
def __init__(self, email):
self.email = email
def update_email(self, new_email):
self.email = new_email
```
This class includes an initializer to set the initial email and a method to update the email.
gemini-2.5-flash · google_gemini · weighted 10.0 · syntactic_validity=10.0 · $6.6e-05 · 684ms ```python
def add(a, b):
return a + b
```
gemini-2.5-flash · google_gemini · weighted 10.0 · syntactic_validity=10.0 · $6.6e-05 · 863ms ```python
def add(a, b):
return a + b
```
command-r-08-2024 · cohere · weighted 10.0 · syntactic_validity=10.0 · $0.0 · 963ms ```python
def add(a, b):
return a + b
```
sonar · perplexity · weighted 10.0 · syntactic_validity=10.0 · $4.5e-05 · 6096ms ```python
def add(a, b):
return a + b
```[1][3][5]
mistral-small-latest · mistral · weighted 10.0 · syntactic_validity=10.0 · $0.0 · 896ms ```python
def add(a, b):
return a + b
```
gpt-4o-mini · openai · weighted 10.0 · syntactic_validity=10.0 · $1.4e-05 · 894ms ```python
def add(a, b):
return a + b
```
gpt-4o-mini · openai · weighted 10.0 · syntactic_validity=10.0 · $1.4e-05 · 745ms ```python
def add(a, b):
return a + b
```
command-r-08-2024 · cohere · weighted 10.0 · syntactic_validity=10.0, naming_conventions=10.0 · $0.0 · 671ms ```python
class UserProfile:
def update_email(self, new_email):
self.email = new_email
```
claude-sonnet-4-6 · anthropic · weighted 10.0 · syntactic_validity=10.0, naming_conventions=10.0 · $0.0 · 1420ms ```python
class UserProfile:
def update_email(self, new_email):
self.email = new_email
```
mistral-large-latest · mistral · weighted 10.0 · syntactic_validity=10.0, naming_conventions=10.0 · $0.0 · 745ms ```python
class UserProfile:
def update_email(self, new_email):
self.email = new_email
```
command-r-08-2024 · cohere · weighted 10.0 · syntactic_validity=10.0 · $0.0 · 769ms ```python
def add(a, b):
return a + b
```
deepseek-reasoner · deepseek · weighted 10.0 · syntactic_validity=10.0 · $0.000123 · 1284ms ```python
def add(a, b):
return a + b
```
grok-3 · xai · weighted 10.0 · syntactic_validity=10.0, naming_conventions=10.0 · $0.000396 · 1812ms class UserProfile:
def update_email(self, new_email):
self.email = new_email
grok-3-mini · xai · weighted 10.0 · syntactic_validity=10.0, naming_conventions=10.0 · $2.3e-05 · 4812ms ```python
class UserProfile:
def update_email(self, new_email):
self.email = new_email
```
deepseek-chat · deepseek · weighted 10.0 · syntactic_validity=10.0 · $8e-06 · 872ms ```python
def add(a, b):
return a + b
```
sonar-pro · perplexity · weighted 10.0 · syntactic_validity=10.0, naming_conventions=10.0 · $0.000513 · 2247ms ```python
class UserProfile:
def update_email(self, new_email):
self.email = new_email
```[2][5]
claude-haiku-4-5-20251001 · anthropic · weighted 10.0 · syntactic_validity=10.0 · $0.000107 · 881ms ```python
def add(a, b):
return a + b
```
mistral-large-latest · mistral · weighted 10.0 · syntactic_validity=10.0, naming_conventions=10.0 · $0.0 · 819ms ```python
class UserProfile:
def update_email(self, new_email):
self.email = new_email
```
gpt-4.1-mini · openai · weighted 10.0 · syntactic_validity=10.0, naming_conventions=10.0 · $0.0 · 1556ms ```python
class UserProfile:
def update_email(self, new_email):
self.email = new_email
```
gpt-4o-mini · openai · weighted 10.0 · syntactic_validity=10.0, naming_conventions=10.0 · $2.9e-05 · 939ms ```python
class UserProfile:
def __init__(self, email):
self.email = email
def update_email(self, new_email):
self.email = new_email
```
gemini-2.5-flash · google_gemini · weighted 10.0 · syntactic_validity=10.0, naming_conventions=10.0 · $0.000205 · 668ms ```python
class UserProfile:
def __init__(self, username, email):
self.username = username
self.email = email
def update_email(self, new_email):
self.email = new_email
```
claude-opus-4-6 · anthropic · weighted 10.0 · syntactic_validity=10.0, naming_conventions=10.0 · $0.00327 · 1714ms
```python
class UserProfile:
def update_email(self, new_email):
self.email = new_email
```
meta-llama/Llama-3.3-70B-Instruct-Turbo · together · weighted 10.0 · syntactic_validity=10.0, naming_conventions=10.0 · $0.0 · 1254ms ```python
class UserProfile:
def __init__(self, email):
self.email = email
def update_email(self, new_email):
self.email = new_email
```
deepseek-ai/DeepSeek-V3.1 · together · weighted 10.0 · syntactic_validity=10.0, naming_conventions=10.0 · $0.0 · 1402ms ```python
class UserProfile:
def __init__(self):
self.email = None
def update_email(self, new_email):
self.email = new_email
```
gemini-2.5-flash · google_gemini · weighted 10.0 · syntactic_validity=10.0, naming_conventions=10.0 · $0.000205 · 910ms ```python
class UserProfile:
def __init__(self, username, email):
self.username = username
self.email = email
def update_email(self, new_email):
self.email = new_email
```
claude-haiku-4-5-20251001 · anthropic · weighted 10.0 · syntactic_validity=10.0 · $0.000107 · 709ms ```python
def add(a, b):
return a + b
```
deepseek-reasoner · deepseek · weighted 10.0 · syntactic_validity=10.0, naming_conventions=10.0 · $0.001169 · 7214ms ```python
class UserProfile:
def update_email(self, new_email):
self.email = new_email
```
gemini-2.5-pro · google_gemini · weighted 10.0 · syntactic_validity=10.0 · $0.004649 · 6438ms ```python
def add(a, b):
return a + b
```
claude-sonnet-4-6 · anthropic · weighted 10.0 · syntactic_validity=10.0 · $0.0 · 1479ms ```python
def add(a, b):
return a + b
```
sonar · perplexity · weighted 10.0 · syntactic_validity=10.0, naming_conventions=10.0 · $5.4e-05 · 2068ms ```python
class UserProfile:
def update_email(self, new_email):
self.email = new_email
```
mistral-small-latest · mistral · weighted 10.0 · syntactic_validity=10.0, naming_conventions=10.0 · $0.0 · 1790ms ```python
class UserProfile:
def update_email(self, new_email):
self.email = new_email
```
sonar · perplexity · weighted 10.0 · syntactic_validity=10.0 · $4.3e-05 · 2045ms ```python
def add(a, b):
return a + b
```[3][5]
grok-3-mini · xai · weighted 10.0 · syntactic_validity=10.0 · $1.4e-05 · 2937ms def add(a, b):
return a + b
Resolved Here Requests semantically routed into this niche.
Requested Method Confidence When code_writinglexical 0.99 2026-05-20 17:25 code_refactorlexical 0.99 2026-05-20 17:25 code_refactorlexical 0.99 2026-05-20 17:24