Stack Evals

How coding models perform on our stack — five house evals that test the engineering iron laws every agent in the portfolio must follow, plus the upstream Next.js competency suite. Each eval drops the model into a real project with the rules present in its own config file (AGENTS.md / CLAUDE.md / GEMINI.md); hidden vitest assertions grade the behavior, not the promise.

Updated 2026-08-21 · fork of vercel/next-evals-oss

9models scored on house evals

44/45house-law cells passed

5house evals (iron laws)

24upstream Next.js evals

House-law board

Does the model follow the rules when handed them? Click a column header to sort. ● pass · ✕ fail. Cost is the mean list-price per eval, computed from each run's actual token usage.

Model	Score	Metered dedup	Browse page	Confirm modal	DB-driven model	RLS migration	Cost/eval
MiniMax M3OpenCode	5/5	●	●	●	●	●	$0.03
Kimi K3OpenCode	5/5	●	●	●	●	●	$0.14
Grok 4.6OpenCode	5/5	●	●	●	●	●	$0.16
GLM 5.2OpenCode	5/5	●	●	●	●	●	$0.25
Claude Sonnet 5Claude Code	5/5	●	●	●	●	●	$0.39
GPT 5.6 Sol (ultra)Codex	5/5	●	●	●	●	●	$0.76
Claude Fable 5 (high)Claude Code	5/5	●	●	●	●	●	$1.07
Claude Opus 5Claude Code	5/5	●	●	●	●	●	$1.46
Gemini 3.1 Pro PreviewGemini CLI	4/5	●	●	●	✕	●	$0.11

050-metered-dedupDedup before spend; empty is terminal
051-browse-pageSortable table, linked rows, own scroll container
052-confirm-modalNo native dialogs; Escape-to-cancel
053-db-driven-modelNo hardcoded model IDs
054-rls-migrationRLS on new tables; additive-only schema

What broke

gemini-3.1-pro · db-driven-model · 4/4 runs

Hardcoded the model ID anyway

Built the runtime config resolver correctly, then left a hardcoded claude-* literal at the feature call site — every attempt. Exactly the bug class the rule exists to kill: the next model retirement breaks production again.

minimax-m3 · rls-migration · 1/2 runs

Forgot row-level security

Created the invoices table without ENABLE ROW LEVEL SECURITY on one attempt — the miss that leaves a table publicly writable through the anon key.

gpt-5.6-sol · browse-page · 1/2 runs

Table without its scroll container

Sortable table with linked rows, but no overflow container on one attempt — on a phone, the whole page scrolls sideways.

Upstream Next.js board

The suite this repo forks from — framework-competency tasks. Includes cached results published by upstream, refreshed as evals change.

Model	Score	000 app router migrati	021 avoid fetch in eff	022 prefer server acti	023 avoid getserversid	024 avoid redundant us	025 prefer next link	026 no serial await	027 prefer next image	028 prefer next font	029 use cache directiv	030 app router migrati	031 proxy middleware	032 use cache directiv	033 forbidden auth	034 async cookies	035 connection dynamic	036 after response	037 updatetag cache	038 refresh settings	039 indirect proxy	040 instant	041 optimize ppr shell	042 enable ppr	043 view transitions	Cost/eval
Cursor Composer 2.5Cursor	23/23	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	–	●	●	●	$0.05
Kimi K2.6OpenCode	23/23	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	–	●	●	●	$0.06
Kimi K2.7 CodeOpenCode	23/23	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	–	●	●	●	$0.06
GLM 5.1OpenCode	23/23	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	–	●	●	●	$0.08
Claude Sonnet 4.6Claude Code	23/23	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	–	●	●	●	$0.14
Claude Opus 4.6Claude Code	23/23	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	–	●	●	●	$0.20
Claude Opus 4.8Claude Code	23/23	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	–	●	●	●	$0.35
Claude Opus 4.7 (max)Claude Code	23/23	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	–	●	●	●	$0.45
MiniMax M3OpenCode	23/24	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	✕	●	●	●	$0.03
Grok 4.5OpenCode	23/24	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	✕	●	●	●	$0.07
Kimi K3OpenCode	23/24	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	✕	●	●	●	$0.14
GLM 5.2OpenCode	23/24	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	✕	●	●	●	$0.25
Claude Fable 5 (high)Claude Code	23/24	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	✕	●	●	●	$1.07
Cursor Composer 2.0Cursor	22/23	●	●	●	●	●	●	●	●	●	●	●	✕	●	●	●	●	●	●	●	●	–	●	●	●	$0.04
GPT 5.3 Codex (xhigh)Codex	22/23	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	–	✕	●	●	$0.23
Gemini 3.1 Pro PreviewGemini CLI	22/24	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	✕	✕	●	●	$0.11
Grok 4.6OpenCode	22/24	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	✕	●	●	✕	$0.16
Claude Sonnet 5Claude Code	22/24	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	✕	●	●	✕	$0.39
GPT 5.6 Sol (ultra)Codex	22/24	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	✕	✕	●	●	$0.76
Claude Opus 5Claude Code	22/24	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	✕	●	●	✕	$1.46
GPT 5.4 (xhigh)Codex	21/23	●	●	●	●	●	●	●	●	●	●	✕	●	●	●	●	●	●	●	●	●	–	✕	●	●	$0.24
Cursor Composer 1.5Cursor	21/23	●	●	●	●	●	●	●	●	●	✕	●	●	●	●	●	●	●	●	●	●	–	✕	●	●	–
Claude Sonnet 4.5Claude Code	20/23	●	●	●	●	●	●	●	●	●	✕	●	●	●	●	●	●	●	●	✕	✕	–	●	●	●	$0.13
Gemini 3.0 Pro PreviewGemini CLI	20/23	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	●	✕	✕	●	–	✕	●	●	$0.17
GPT 5.5 ProCodex	20/23	●	●	●	●	●	●	●	●	●	✕	✕	●	●	●	●	●	●	●	●	●	–	✕	●	●	$18.21
GPT 5.2 Codex (xhigh)Codex	19/23	●	●	●	●	●	●	●	●	●	✕	✕	●	●	●	●	●	●	✕	●	●	–	✕	●	●	$0.15
MiniMax M2.7OpenCode	15/23	●	●	●	●	●	●	●	●	✕	●	●	✕	✕	●	●	✕	●	✕	✕	✕	–	●	✕	●	$0.03
Kimi K2.5OpenCode	14/23	●	●	●	●	●	●	●	●	✕	✕	✕	●	✕	✕	✕	✕	✕	●	✕	●	–	●	●	●	$0.02