{"harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "description": "Claude Code is Anthropic's official agentic coding product: a terminal-first CLI that plans, searches the repository, edits files, and runs shell commands in a loop until the task is done. It is designed for day-to-day engineering work rather than a minimal benchmark scaffold.\n\nOn public leaderboards, \"Claude Code\" rows usually mean this product (or a pinned release) paired with a specific Claude model \u2014 not a stripped-down bash-only loop. Artificial Analysis publishes a composite Coding Agent Index across DeepSWE, SWE-bench Pro, and Terminal-Bench components; SWE-bench Verified community submissions use vendor-specific harness versions.\n\nHarnessRL validates Claude Code end-to-end: scaffold code `cc`, Harbor agent `BuiltinCCAgentLoop`, Anthropic Messages API protocol, and in-process proxy integration for RL rollouts. Scores tagged `is_harnessrl_measured` in this catalog are from fixed harness versions under the HarnessRL eval protocol.\n\nWhen comparing Claude Code to other vendor CLIs (Codex, Cursor CLI, Gemini CLI), look at rows with the same benchmark slug and similar model tier. When comparing models inside Anthropic's stack, prefer rows that share the same harness version and eval protocol rather than mixing AA component scores with Verified resolution %.", "card_summary": "Anthropic's agentic coding CLI \u2014 terminal edits, search, and multi-step repo workflows via the Messages API.", "homepage_url": "https://github.com/anthropics/claude-code", "repo_url": "https://github.com/anthropics/claude-code", "docs_url": "https://harness-rl.pages.dev/docs/architecture/in-process-proxy", "scaffold_code": "cc", "harbor_agent_name": "BuiltinCCAgentLoop", "api_protocol": "anthropic", "status": "validated", "supports_rl": true, "score_count": 74, "best_success_rate": 89.14, "github_stars": 18400, "popularity_tier": "popular", "hrl_score": 63.0, "hrl_rank": 6, "catalog_token_volume": 12676236218475, "openrouter_icon_url": "https://claude.ai/favicon.ico", "leaderboard_icon_url": null, "aa_coding_index": 0.6814975428587517, "aa_mean_cost_usd": 3.1718671924846644, "aa_mean_total_tokens": 7999358}, "scores": [{"success_rate": 89.14, "resolved_count": null, "total_count": null, "cost_usd": 8.170783921523505, "input_tokens": 10747525, "output_tokens": 73273, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-5-opus-xhigh", "name": "Claude 5 Opus (xhigh)", "vendor": "Anthropic"}, "benchmark": {"slug": "terminal-bench-v2-1", "name": "Terminal-Bench v2.1 (AA component)"}}, {"success_rate": 88.76, "resolved_count": null, "total_count": null, "cost_usd": 8.943547018021487, "input_tokens": 11810977, "output_tokens": 81385, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-5-opus-max", "name": "Claude 5 Opus (max)", "vendor": "Anthropic"}, "benchmark": {"slug": "terminal-bench-v2-1", "name": "Terminal-Bench v2.1 (AA component)"}}, {"success_rate": 86.89, "resolved_count": null, "total_count": null, "cost_usd": 3.918602446063396, "input_tokens": 4937399, "output_tokens": 37331, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-5-opus-high", "name": "Claude 5 Opus (high)", "vendor": "Anthropic"}, "benchmark": {"slug": "terminal-bench-v2-1", "name": "Terminal-Bench v2.1 (AA component)"}}, {"success_rate": 86.52, "resolved_count": null, "total_count": null, "cost_usd": 11.687407620910031, "input_tokens": 6901095, "output_tokens": 74265, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "fable-5-max-with-fallback", "name": "Fable 5 Max With Fallback", "vendor": "Fable"}, "benchmark": {"slug": "terminal-bench-v2-1", "name": "Terminal-Bench v2.1 (AA component)"}}, {"success_rate": 85.02, "resolved_count": null, "total_count": null, "cost_usd": 3.1718671924846644, "input_tokens": 3984815, "output_tokens": 30206, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-5-opus-medium", "name": "Claude 5 Opus (medium)", "vendor": "Anthropic"}, "benchmark": {"slug": "terminal-bench-v2-1", "name": "Terminal-Bench v2.1 (AA component)"}}, {"success_rate": 84.0, "resolved_count": null, "total_count": null, "cost_usd": null, "input_tokens": null, "output_tokens": null, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "SWE-rebench resolved rate (best rolling window)", "source": "swe_rebench_site", "source_url": "https://swe-rebench.com/", "observed_at": null, "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "swe-rebench-agent-product", "name": "SWE-rebench agent (product)", "vendor": null}, "benchmark": {"slug": "swe-rebench", "name": "SWE-rebench"}}, {"success_rate": 83.9, "resolved_count": null, "total_count": null, "cost_usd": 3.2306109660020477, "input_tokens": 6469207, "output_tokens": 63117, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "qwen3-8-max", "name": "Qwen3.8 Max", "vendor": "Qwen"}, "benchmark": {"slug": "terminal-bench-v2-1", "name": "Terminal-Bench v2.1 (AA component)"}}, {"success_rate": 83.9, "resolved_count": null, "total_count": null, "cost_usd": 7.724725797009217, "input_tokens": 8919829, "output_tokens": 89374, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-4-8-opus-max", "name": "Claude 4.8 Opus (max)", "vendor": "Anthropic"}, "benchmark": {"slug": "terminal-bench-v2-1", "name": "Terminal-Bench v2.1 (AA component)"}}, {"success_rate": 83.82, "resolved_count": null, "total_count": null, "cost_usd": 1.2419550561797752, "input_tokens": 20480925, "output_tokens": 9954945, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": 445, "is_harnessrl_measured": false, "eval_protocol": null, "source": "tbench_official", "source_url": "https://github.com/harbor-framework/terminal-bench-2-1/pull/75", "observed_at": "2026-06-07T00:00:00+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "fable-5", "name": "Fable 5", "vendor": "Anthropic"}, "benchmark": {"slug": "terminal-bench-2-1-official", "name": "Terminal-Bench 2.1 (official)"}}, {"success_rate": 83.15, "resolved_count": null, "total_count": null, "cost_usd": 5.665366469580767, "input_tokens": 6783846, "output_tokens": 62541, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-4-8-opus-xhigh", "name": "Claude 4.8 Opus (xhigh)", "vendor": "Anthropic"}, "benchmark": {"slug": "terminal-bench-v2-1", "name": "Terminal-Bench v2.1 (AA component)"}}, {"success_rate": 82.4, "resolved_count": null, "total_count": null, "cost_usd": 3.7823013803680996, "input_tokens": 4574601, "output_tokens": 40682, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-4-8-opus-high", "name": "Claude 4.8 Opus (high)", "vendor": "Anthropic"}, "benchmark": {"slug": "terminal-bench-v2-1", "name": "Terminal-Bench v2.1 (AA component)"}}, {"success_rate": 82.02, "resolved_count": null, "total_count": null, "cost_usd": 2.2951577448875273, "input_tokens": 2618171, "output_tokens": 22632, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-5-opus-low", "name": "Claude 5 Opus (low)", "vendor": "Anthropic"}, "benchmark": {"slug": "terminal-bench-v2-1", "name": "Terminal-Bench v2.1 (AA component)"}}, {"success_rate": 82.02, "resolved_count": null, "total_count": null, "cost_usd": 3.3038416339468313, "input_tokens": 3898584, "output_tokens": 34668, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-4-8-opus-medium", "name": "Claude 4.8 Opus (medium)", "vendor": "Anthropic"}, "benchmark": {"slug": "terminal-bench-v2-1", "name": "Terminal-Bench v2.1 (AA component)"}}, {"success_rate": 78.88, "resolved_count": null, "total_count": null, "cost_usd": 0.6448089887640449, "input_tokens": 12873472, "output_tokens": 8089069, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": 445, "is_harnessrl_measured": false, "eval_protocol": null, "source": "tbench_official", "source_url": "https://github.com/harbor-framework/terminal-bench-2-1/pull/92", "observed_at": "2026-07-09T00:00:00+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-4-8-opus", "name": "Claude 4.8 Opus", "vendor": "Anthropic"}, "benchmark": {"slug": "terminal-bench-2-1-official", "name": "Terminal-Bench 2.1 (official)"}}, {"success_rate": 77.9, "resolved_count": null, "total_count": null, "cost_usd": 2.1825946797034774, "input_tokens": 2595804, "output_tokens": 23270, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-4-8-opus-low", "name": "Claude 4.8 Opus (low)", "vendor": "Anthropic"}, "benchmark": {"slug": "terminal-bench-v2-1", "name": "Terminal-Bench v2.1 (AA component)"}}, {"success_rate": 77.78, "resolved_count": null, "total_count": null, "cost_usd": 1.9369, "input_tokens": null, "output_tokens": null, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": 45, "is_harnessrl_measured": false, "eval_protocol": "HAL CORE-Bench Hard (HAL) accuracy", "source": "hal_site", "source_url": "https://hal.cs.princeton.edu/corebench_hard", "observed_at": null, "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-4-5-opus", "name": "Claude 4.5 Opus", "vendor": "Anthropic"}, "benchmark": {"slug": "hal-corebench-hard", "name": "CORE-Bench Hard (HAL)"}}, {"success_rate": 77.53, "resolved_count": null, "total_count": null, "cost_usd": 5.915424259100206, "input_tokens": 8042457, "output_tokens": 47224, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-4-7-opus-max", "name": "Claude 4.7 Opus (max)", "vendor": "Anthropic"}, "benchmark": {"slug": "terminal-bench-v2-1", "name": "Terminal-Bench v2.1 (AA component)"}}, {"success_rate": 77.15, "resolved_count": null, "total_count": null, "cost_usd": 1.8006617447341524, "input_tokens": 2310268, "output_tokens": 17168, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-4-7-opus-medium", "name": "Claude 4.7 Opus (medium)", "vendor": "Anthropic"}, "benchmark": {"slug": "terminal-bench-v2-1", "name": "Terminal-Bench v2.1 (AA component)"}}, {"success_rate": 74.61, "resolved_count": null, "total_count": null, "cost_usd": 0.6475955056179775, "input_tokens": 20137729, "output_tokens": 11216322, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": 445, "is_harnessrl_measured": false, "eval_protocol": null, "source": "tbench_official", "source_url": "https://github.com/harbor-framework/terminal-bench-2-1/pull/98", "observed_at": "2026-07-09T00:00:00+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-5-sonnet", "name": "Claude 5 Sonnet", "vendor": "Anthropic"}, "benchmark": {"slug": "terminal-bench-2-1-official", "name": "Terminal-Bench 2.1 (official)"}}, {"success_rate": 72.28, "resolved_count": null, "total_count": null, "cost_usd": 6.657531358159516, "input_tokens": 5529735, "output_tokens": 28567, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "glm-5-2", "name": "GLM 5 2", "vendor": "Zhipu"}, "benchmark": {"slug": "terminal-bench-v2-1", "name": "Terminal-Bench v2.1 (AA component)"}}, {"success_rate": 71.54, "resolved_count": null, "total_count": null, "cost_usd": 6.3006588057259885, "input_tokens": 4715248, "output_tokens": 32611, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "qwen3-7-plus-thinking", "name": "Qwen3.7 Plus (thinking)", "vendor": "Qwen"}, "benchmark": {"slug": "terminal-bench-v2-1", "name": "Terminal-Bench v2.1 (AA component)"}}, {"success_rate": 70.04, "resolved_count": null, "total_count": null, "cost_usd": 0.2547256318476481, "input_tokens": 5181244, "output_tokens": 41975, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "deepseek-v4-pro-high", "name": "Deepseek V4 Pro High", "vendor": "DeepSeek"}, "benchmark": {"slug": "terminal-bench-v2-1", "name": "Terminal-Bench v2.1 (AA component)"}}, {"success_rate": 69.29, "resolved_count": null, "total_count": null, "cost_usd": 1.2171000741104296, "input_tokens": 5891966, "output_tokens": 36930, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "kimi-k2-6", "name": "KIMI K2.6", "vendor": "Moonshot"}, "benchmark": {"slug": "terminal-bench-v2-1", "name": "Terminal-Bench v2.1 (AA component)"}}, {"success_rate": 68.9, "resolved_count": null, "total_count": null, "cost_usd": 1.3412080536912752, "input_tokens": 31873388, "output_tokens": 14896548, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": 447, "is_harnessrl_measured": false, "eval_protocol": null, "source": "tbench_official", "source_url": "https://github.com/harbor-framework/terminal-bench-2-1/pull/44", "observed_at": "2026-05-01T00:00:00+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-4-7-opus", "name": "Claude 4.7 Opus", "vendor": "Anthropic"}, "benchmark": {"slug": "terminal-bench-2-1-official", "name": "Terminal-Bench 2.1 (official)"}}, {"success_rate": 67.42, "resolved_count": null, "total_count": null, "cost_usd": 2.0415130214723938, "input_tokens": 4205086, "output_tokens": 40070, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-4-6-sonnet-medium", "name": "Claude 4.6 Sonnet (medium)", "vendor": "Anthropic"}, "benchmark": {"slug": "terminal-bench-v2-1", "name": "Terminal-Bench v2.1 (AA component)"}}, {"success_rate": 67.42, "resolved_count": null, "total_count": null, "cost_usd": 4.282000252842536, "input_tokens": 13139446, "output_tokens": 49883, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "glm-5-1", "name": "GLM 5 1", "vendor": "Zhipu"}, "benchmark": {"slug": "terminal-bench-v2-1", "name": "Terminal-Bench v2.1 (AA component)"}}, {"success_rate": 66.08, "resolved_count": null, "total_count": null, "cost_usd": 11.687407620910031, "input_tokens": 6901095, "output_tokens": 74265, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "fable-5-max-with-fallback", "name": "Fable 5 Max With Fallback", "vendor": "Fable"}, "benchmark": {"slug": "deepswe", "name": "DeepSWE (AA component)"}}, {"success_rate": 63.13, "resolved_count": null, "total_count": null, "cost_usd": 8.943547018021487, "input_tokens": 11810977, "output_tokens": 81385, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-5-opus-max", "name": "Claude 5 Opus (max)", "vendor": "Anthropic"}, "benchmark": {"slug": "deepswe", "name": "DeepSWE (AA component)"}}, {"success_rate": 62.83, "resolved_count": null, "total_count": null, "cost_usd": 3.1718671924846644, "input_tokens": 3984815, "output_tokens": 30206, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-5-opus-medium", "name": "Claude 5 Opus (medium)", "vendor": "Anthropic"}, "benchmark": {"slug": "deepswe", "name": "DeepSWE (AA component)"}}, {"success_rate": 62.22, "resolved_count": null, "total_count": null, "cost_usd": 1.5184, "input_tokens": null, "output_tokens": null, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": 45, "is_harnessrl_measured": false, "eval_protocol": "HAL CORE-Bench Hard (HAL) accuracy", "source": "hal_site", "source_url": "https://hal.cs.princeton.edu/corebench_hard", "observed_at": null, "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-4-5-sonnet", "name": "Claude 4.5 Sonnet", "vendor": "Anthropic"}, "benchmark": {"slug": "hal-corebench-hard", "name": "CORE-Bench Hard (HAL)"}}, {"success_rate": 60.77, "resolved_count": null, "total_count": null, "cost_usd": 3.918602446063396, "input_tokens": 4937399, "output_tokens": 37331, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-5-opus-high", "name": "Claude 5 Opus (high)", "vendor": "Anthropic"}, "benchmark": {"slug": "deepswe", "name": "DeepSWE (AA component)"}}, {"success_rate": 60.47, "resolved_count": null, "total_count": null, "cost_usd": 8.170783921523505, "input_tokens": 10747525, "output_tokens": 73273, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-5-opus-xhigh", "name": "Claude 5 Opus (xhigh)", "vendor": "Anthropic"}, "benchmark": {"slug": "deepswe", "name": "DeepSWE (AA component)"}}, {"success_rate": 58.65, "resolved_count": null, "total_count": null, "cost_usd": 0.6227865168539326, "input_tokens": 13570352, "output_tokens": 10635681, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": 445, "is_harnessrl_measured": false, "eval_protocol": null, "source": "tbench_official", "source_url": "https://github.com/harbor-framework/terminal-bench-2-1/pull/67", "observed_at": "2026-05-01T00:00:00+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "glm-5-1", "name": "GLM 5 1", "vendor": "Zhipu"}, "benchmark": {"slug": "terminal-bench-2-1-official", "name": "Terminal-Bench 2.1 (official)"}}, {"success_rate": 56.93, "resolved_count": null, "total_count": null, "cost_usd": 2.2951577448875273, "input_tokens": 2618171, "output_tokens": 22632, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-5-opus-low", "name": "Claude 5 Opus (low)", "vendor": "Anthropic"}, "benchmark": {"slug": "deepswe", "name": "DeepSWE (AA component)"}}, {"success_rate": 55.75, "resolved_count": null, "total_count": null, "cost_usd": 7.724725797009217, "input_tokens": 8919829, "output_tokens": 89374, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-4-8-opus-max", "name": "Claude 4.8 Opus (max)", "vendor": "Anthropic"}, "benchmark": {"slug": "deepswe", "name": "DeepSWE (AA component)"}}, {"success_rate": 54.84, "resolved_count": null, "total_count": null, "cost_usd": 8.170783921523505, "input_tokens": 10747525, "output_tokens": 73273, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-5-opus-xhigh", "name": "Claude 5 Opus (xhigh)", "vendor": "Anthropic"}, "benchmark": {"slug": "swe-atlas-qna", "name": "SWE-Atlas-QnA (AA component)"}}, {"success_rate": 51.92, "resolved_count": null, "total_count": null, "cost_usd": 3.2306109660020477, "input_tokens": 6469207, "output_tokens": 63117, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "qwen3-8-max", "name": "Qwen3.8 Max", "vendor": "Qwen"}, "benchmark": {"slug": "deepswe", "name": "DeepSWE (AA component)"}}, {"success_rate": 51.62, "resolved_count": null, "total_count": null, "cost_usd": 3.7823013803680996, "input_tokens": 4574601, "output_tokens": 40682, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-4-8-opus-high", "name": "Claude 4.8 Opus (high)", "vendor": "Anthropic"}, "benchmark": {"slug": "deepswe", "name": "DeepSWE (AA component)"}}, {"success_rate": 51.33, "resolved_count": null, "total_count": null, "cost_usd": 5.665366469580767, "input_tokens": 6783846, "output_tokens": 62541, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-4-8-opus-xhigh", "name": "Claude 4.8 Opus (xhigh)", "vendor": "Anthropic"}, "benchmark": {"slug": "deepswe", "name": "DeepSWE (AA component)"}}, {"success_rate": 49.26, "resolved_count": null, "total_count": null, "cost_usd": 3.3038416339468313, "input_tokens": 3898584, "output_tokens": 34668, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-4-8-opus-medium", "name": "Claude 4.8 Opus (medium)", "vendor": "Anthropic"}, "benchmark": {"slug": "deepswe", "name": "DeepSWE (AA component)"}}, {"success_rate": 49.19, "resolved_count": null, "total_count": null, "cost_usd": 8.943547018021487, "input_tokens": 11810977, "output_tokens": 81385, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-5-opus-max", "name": "Claude 5 Opus (max)", "vendor": "Anthropic"}, "benchmark": {"slug": "swe-atlas-qna", "name": "SWE-Atlas-QnA (AA component)"}}, {"success_rate": 49.19, "resolved_count": null, "total_count": null, "cost_usd": 3.918602446063396, "input_tokens": 4937399, "output_tokens": 37331, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-5-opus-high", "name": "Claude 5 Opus (high)", "vendor": "Anthropic"}, "benchmark": {"slug": "swe-atlas-qna", "name": "SWE-Atlas-QnA (AA component)"}}, {"success_rate": 48.92, "resolved_count": null, "total_count": null, "cost_usd": 11.687407620910031, "input_tokens": 6901095, "output_tokens": 74265, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "fable-5-max-with-fallback", "name": "Fable 5 Max With Fallback", "vendor": "Fable"}, "benchmark": {"slug": "swe-atlas-qna", "name": "SWE-Atlas-QnA (AA component)"}}, {"success_rate": 48.4, "resolved_count": null, "total_count": null, "cost_usd": 0.61, "input_tokens": null, "output_tokens": null, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": 60, "is_harnessrl_measured": false, "eval_protocol": "WildClawBench overall score (60 tasks)", "source": "wildclawbench_site", "source_url": "https://github.com/internlm/WildClawBench", "observed_at": null, "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "gpt-5-4", "name": "GPT-5.4", "vendor": "openai"}, "benchmark": {"slug": "wildclawbench", "name": "WildClawBench"}}, {"success_rate": 48.12, "resolved_count": null, "total_count": null, "cost_usd": 3.2306109660020477, "input_tokens": 6469207, "output_tokens": 63117, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "qwen3-8-max", "name": "Qwen3.8 Max", "vendor": "Qwen"}, "benchmark": {"slug": "swe-atlas-qna", "name": "SWE-Atlas-QnA (AA component)"}}, {"success_rate": 46.77, "resolved_count": null, "total_count": null, "cost_usd": 7.724725797009217, "input_tokens": 8919829, "output_tokens": 89374, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-4-8-opus-max", "name": "Claude 4.8 Opus (max)", "vendor": "Anthropic"}, "benchmark": {"slug": "swe-atlas-qna", "name": "SWE-Atlas-QnA (AA component)"}}, {"success_rate": 46.67, "resolved_count": null, "total_count": null, "cost_usd": 1.4573, "input_tokens": null, "output_tokens": null, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": 45, "is_harnessrl_measured": false, "eval_protocol": "HAL CORE-Bench Hard (HAL) accuracy", "source": "hal_site", "source_url": "https://hal.cs.princeton.edu/corebench_hard", "observed_at": null, "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-4-sonnet", "name": "Claude 4 Sonnet", "vendor": "Anthropic"}, "benchmark": {"slug": "hal-corebench-hard", "name": "CORE-Bench Hard (HAL)"}}, {"success_rate": 44.35, "resolved_count": null, "total_count": null, "cost_usd": 3.1718671924846644, "input_tokens": 3984815, "output_tokens": 30206, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-5-opus-medium", "name": "Claude 5 Opus (medium)", "vendor": "Anthropic"}, "benchmark": {"slug": "swe-atlas-qna", "name": "SWE-Atlas-QnA (AA component)"}}, {"success_rate": 42.74, "resolved_count": null, "total_count": null, "cost_usd": 5.665366469580767, "input_tokens": 6783846, "output_tokens": 62541, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-4-8-opus-xhigh", "name": "Claude 4.8 Opus (xhigh)", "vendor": "Anthropic"}, "benchmark": {"slug": "swe-atlas-qna", "name": "SWE-Atlas-QnA (AA component)"}}, {"success_rate": 42.22, "resolved_count": null, "total_count": null, "cost_usd": 7.3731, "input_tokens": null, "output_tokens": null, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": 45, "is_harnessrl_measured": false, "eval_protocol": "HAL CORE-Bench Hard (HAL) accuracy", "source": "hal_site", "source_url": "https://hal.cs.princeton.edu/corebench_hard", "observed_at": null, "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-4-1-opus", "name": "Claude 4.1 Opus", "vendor": "Anthropic"}, "benchmark": {"slug": "hal-corebench-hard", "name": "CORE-Bench Hard (HAL)"}}, {"success_rate": 41.0, "resolved_count": null, "total_count": null, "cost_usd": 2.1825946797034774, "input_tokens": 2595804, "output_tokens": 23270, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-4-8-opus-low", "name": "Claude 4.8 Opus (low)", "vendor": "Anthropic"}, "benchmark": {"slug": "deepswe", "name": "DeepSWE (AA component)"}}, {"success_rate": 40.12, "resolved_count": null, "total_count": null, "cost_usd": 5.915424259100206, "input_tokens": 8042457, "output_tokens": 47224, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-4-7-opus-max", "name": "Claude 4.7 Opus (max)", "vendor": "Anthropic"}, "benchmark": {"slug": "deepswe", "name": "DeepSWE (AA component)"}}, {"success_rate": 39.25, "resolved_count": null, "total_count": null, "cost_usd": 2.2951577448875273, "input_tokens": 2618171, "output_tokens": 22632, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-5-opus-low", "name": "Claude 5 Opus (low)", "vendor": "Anthropic"}, "benchmark": {"slug": "swe-atlas-qna", "name": "SWE-Atlas-QnA (AA component)"}}, {"success_rate": 38.71, "resolved_count": null, "total_count": null, "cost_usd": 3.7823013803680996, "input_tokens": 4574601, "output_tokens": 40682, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-4-8-opus-high", "name": "Claude 4.8 Opus (high)", "vendor": "Anthropic"}, "benchmark": {"slug": "swe-atlas-qna", "name": "SWE-Atlas-QnA (AA component)"}}, {"success_rate": 37.1, "resolved_count": null, "total_count": null, "cost_usd": 5.915424259100206, "input_tokens": 8042457, "output_tokens": 47224, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-4-7-opus-max", "name": "Claude 4.7 Opus (max)", "vendor": "Anthropic"}, "benchmark": {"slug": "swe-atlas-qna", "name": "SWE-Atlas-QnA (AA component)"}}, {"success_rate": 36.02, "resolved_count": null, "total_count": null, "cost_usd": 3.3038416339468313, "input_tokens": 3898584, "output_tokens": 34668, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-4-8-opus-medium", "name": "Claude 4.8 Opus (medium)", "vendor": "Anthropic"}, "benchmark": {"slug": "swe-atlas-qna", "name": "SWE-Atlas-QnA (AA component)"}}, {"success_rate": 32.0, "resolved_count": null, "total_count": null, "cost_usd": 0.09, "input_tokens": null, "output_tokens": null, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": 60, "is_harnessrl_measured": false, "eval_protocol": "WildClawBench overall score (60 tasks)", "source": "wildclawbench_site", "source_url": "https://github.com/internlm/WildClawBench", "observed_at": null, "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "minimax-m2-7", "name": "MiniMax M2.7", "vendor": "minimax"}, "benchmark": {"slug": "wildclawbench", "name": "WildClawBench"}}, {"success_rate": 31.0, "resolved_count": null, "total_count": null, "cost_usd": 0.21, "input_tokens": null, "output_tokens": null, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": 60, "is_harnessrl_measured": false, "eval_protocol": "WildClawBench overall score (60 tasks)", "source": "wildclawbench_site", "source_url": "https://github.com/internlm/WildClawBench", "observed_at": null, "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "glm-5", "name": "GLM 5", "vendor": "z-ai"}, "benchmark": {"slug": "wildclawbench", "name": "WildClawBench"}}, {"success_rate": 29.9, "resolved_count": null, "total_count": null, "cost_usd": 0.15, "input_tokens": null, "output_tokens": null, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": 60, "is_harnessrl_measured": false, "eval_protocol": "WildClawBench overall score (60 tasks)", "source": "wildclawbench_site", "source_url": "https://github.com/internlm/WildClawBench", "observed_at": null, "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "mimo-v2-pro", "name": "MiMo V2 Pro", "vendor": "Xiaomi"}, "benchmark": {"slug": "wildclawbench", "name": "WildClawBench"}}, {"success_rate": 29.03, "resolved_count": null, "total_count": null, "cost_usd": 6.657531358159516, "input_tokens": 5529735, "output_tokens": 28567, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "glm-5-2", "name": "GLM 5 2", "vendor": "Zhipu"}, "benchmark": {"slug": "swe-atlas-qna", "name": "SWE-Atlas-QnA (AA component)"}}, {"success_rate": 28.91, "resolved_count": null, "total_count": null, "cost_usd": 2.0415130214723938, "input_tokens": 4205086, "output_tokens": 40070, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-4-6-sonnet-medium", "name": "Claude 4.6 Sonnet (medium)", "vendor": "Anthropic"}, "benchmark": {"slug": "deepswe", "name": "DeepSWE (AA component)"}}, {"success_rate": 28.61, "resolved_count": null, "total_count": null, "cost_usd": 6.657531358159516, "input_tokens": 5529735, "output_tokens": 28567, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "glm-5-2", "name": "GLM 5 2", "vendor": "Zhipu"}, "benchmark": {"slug": "deepswe", "name": "DeepSWE (AA component)"}}, {"success_rate": 28.23, "resolved_count": null, "total_count": null, "cost_usd": 2.1825946797034774, "input_tokens": 2595804, "output_tokens": 23270, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-4-8-opus-low", "name": "Claude 4.8 Opus (low)", "vendor": "Anthropic"}, "benchmark": {"slug": "swe-atlas-qna", "name": "SWE-Atlas-QnA (AA component)"}}, {"success_rate": 27.43, "resolved_count": null, "total_count": null, "cost_usd": 1.8006617447341524, "input_tokens": 2310268, "output_tokens": 17168, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-4-7-opus-medium", "name": "Claude 4.7 Opus (medium)", "vendor": "Anthropic"}, "benchmark": {"slug": "deepswe", "name": "DeepSWE (AA component)"}}, {"success_rate": 24.73, "resolved_count": null, "total_count": null, "cost_usd": 4.282000252842536, "input_tokens": 13139446, "output_tokens": 49883, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "glm-5-1", "name": "GLM 5 1", "vendor": "Zhipu"}, "benchmark": {"slug": "swe-atlas-qna", "name": "SWE-Atlas-QnA (AA component)"}}, {"success_rate": 23.66, "resolved_count": null, "total_count": null, "cost_usd": 6.3006588057259885, "input_tokens": 4715248, "output_tokens": 32611, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "qwen3-7-plus-thinking", "name": "Qwen3.7 Plus (thinking)", "vendor": "Qwen"}, "benchmark": {"slug": "swe-atlas-qna", "name": "SWE-Atlas-QnA (AA component)"}}, {"success_rate": 22.58, "resolved_count": null, "total_count": null, "cost_usd": 1.8006617447341524, "input_tokens": 2310268, "output_tokens": 17168, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-4-7-opus-medium", "name": "Claude 4.7 Opus (medium)", "vendor": "Anthropic"}, "benchmark": {"slug": "swe-atlas-qna", "name": "SWE-Atlas-QnA (AA component)"}}, {"success_rate": 19.89, "resolved_count": null, "total_count": null, "cost_usd": 0.2547256318476481, "input_tokens": 5181244, "output_tokens": 41975, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "deepseek-v4-pro-high", "name": "Deepseek V4 Pro High", "vendor": "DeepSeek"}, "benchmark": {"slug": "swe-atlas-qna", "name": "SWE-Atlas-QnA (AA component)"}}, {"success_rate": 19.62, "resolved_count": null, "total_count": null, "cost_usd": 2.0415130214723938, "input_tokens": 4205086, "output_tokens": 40070, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "claude-4-6-sonnet-medium", "name": "Claude 4.6 Sonnet (medium)", "vendor": "Anthropic"}, "benchmark": {"slug": "swe-atlas-qna", "name": "SWE-Atlas-QnA (AA component)"}}, {"success_rate": 19.17, "resolved_count": null, "total_count": null, "cost_usd": 6.3006588057259885, "input_tokens": 4715248, "output_tokens": 32611, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "qwen3-7-plus-thinking", "name": "Qwen3.7 Plus (thinking)", "vendor": "Qwen"}, "benchmark": {"slug": "deepswe", "name": "DeepSWE (AA component)"}}, {"success_rate": 18.58, "resolved_count": null, "total_count": null, "cost_usd": 4.282000252842536, "input_tokens": 13139446, "output_tokens": 49883, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "glm-5-1", "name": "GLM 5 1", "vendor": "Zhipu"}, "benchmark": {"slug": "deepswe", "name": "DeepSWE (AA component)"}}, {"success_rate": 16.52, "resolved_count": null, "total_count": null, "cost_usd": 1.2171000741104296, "input_tokens": 5891966, "output_tokens": 36930, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "kimi-k2-6", "name": "KIMI K2.6", "vendor": "Moonshot"}, "benchmark": {"slug": "deepswe", "name": "DeepSWE (AA component)"}}, {"success_rate": 15.86, "resolved_count": null, "total_count": null, "cost_usd": 1.2171000741104296, "input_tokens": 5891966, "output_tokens": 36930, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "kimi-k2-6", "name": "KIMI K2.6", "vendor": "Moonshot"}, "benchmark": {"slug": "swe-atlas-qna", "name": "SWE-Atlas-QnA (AA component)"}}, {"success_rate": 8.55, "resolved_count": null, "total_count": null, "cost_usd": 0.2547256318476481, "input_tokens": 5181244, "output_tokens": 41975, "latency_ms": null, "temperature": null, "max_turns": null, "context_budget_tokens": null, "harness_version": null, "trial_count": null, "is_harnessrl_measured": false, "eval_protocol": "Artificial Analysis Coding Agent Index component (pass@1)", "source": "artificialanalysis", "source_url": "https://artificialanalysis.ai/agents/coding-agents", "observed_at": "2026-08-27T21:54:11.189561+00:00", "harness": {"slug": "claude-code", "name": "Claude Code", "vendor": "Anthropic", "repo_url": "https://github.com/anthropics/claude-code", "openrouter_icon_url": "https://claude.ai/favicon.ico"}, "model": {"slug": "deepseek-v4-pro-high", "name": "Deepseek V4 Pro High", "vendor": "DeepSeek"}, "benchmark": {"slug": "deepswe", "name": "DeepSWE (AA component)"}}], "benchmarks": [{"slug": "hal-corebench-hard", "name": "CORE-Bench Hard (HAL)", "category": "eval"}, {"slug": "deepswe", "name": "DeepSWE (AA component)", "category": "eval"}, {"slug": "swe-atlas-qna", "name": "SWE-Atlas-QnA (AA component)", "category": "eval"}, {"slug": "swe-rebench", "name": "SWE-rebench", "category": "eval"}, {"slug": "terminal-bench-2-1-official", "name": "Terminal-Bench 2.1 (official)", "category": "eval"}, {"slug": "terminal-bench-v2-1", "name": "Terminal-Bench v2.1 (AA component)", "category": "eval"}, {"slug": "wildclawbench", "name": "WildClawBench", "category": "eval"}]}