Releases·Tool Releasefeatured
Anthropic ships `claude plugin eval init` for AI-drafted plugin eval suites
Anthropic released `claude plugin eval`, a Claude Code feature that evaluates plugins and skills against user-defined test cases. It runs each case with and without the plugin, scores both, and reports the delta in the terminal plus an HTML report. Practitioners get a built-in way to measure whether a plugin adds value before shipping it.
3578 engagement4 sourcesFri, Sep 11, 2026, 07:56 PM
AI Overview
- `claude plugin eval init` drafts test cases and checks from user-supplied good/bad output examples, pilots the suite, and estimates full-run cost
- `claude plugin eval` scores plugin runs against test cases and reruns each case without the plugin to compare differences
- `claude plugin eval` generates an HTML report with full per-case detail, published as a private artifact where the account supports it
- Anthropic advises piloting with `--runs 1` since evals consume tokens and results vary, and warns plugin hooks and MCP servers run as the user, so only evaluate trusted plugins
Related
Products·launched
100xeval — open-source tool that automates evals for Claude plugins
100xeval is an open-source Claude Code plugin that automates evaluation of other Claude plugins. It runs a static structural check without model calls, then a headless eval where Claude grades plugin output against user-defined test cases. It targets developers maintaining plugins who would otherwise skip regular evals.
100xeval is an open-source Claude Code plugin that automates evaluation of other Claude plugins. It runs a static structural check without model calls, then a headless eval where Claude grades plugin output against user-defined test cases. It targets developers maintaining plugins who would otherwise skip regular evals.
2 engagement1 sourceredditFri, Sep 18, 2026, 02:08 PM
News·Company News
Enforcing data residency with single-Region Claude Code on Amazon Bedrock
A US-headquartered global organization required Claude Code on Amazon Bedrock to process all inference in London (eu-west-2), not just be called from there. The team tried Anthropic's Mantle endpoint first but hit a limitation, then fell back to an alternative approach. The post details how they achieved single-Region data residency for Claude Code.
A US-headquartered global organization required Claude Code on Amazon Bedrock to process all inference in London (eu-west-2), not just be called from there. The team tried Anthropic's Mantle endpoint first but hit a limitation, then fell back to an alternative approach. The post details how they achieved single-Region data residency for Claude Code.
0 engagement2 sourcesrssThu, Aug 6, 2026, 04:21 PM
Social·Community
Users complain about frequent Claude Code updates
Users are complaining that Claude Code, Anthropic's coding tool, prompts for updates more than once per day, which they find distracting and unprofessional. They suggest Anthropic should limit updates to about twice per week.
Users are complaining that Claude Code, Anthropic's coding tool, prompts for updates more than once per day, which they find distracting and unprofessional. They suggest Anthropic should limit updates to about twice per week.
2.4k engagement1 sourceredditTue, Aug 18, 2026, 12:30 AM
Social·Community
User finds Claude Code desktop shows Opus 4.6 thinking blocks as empty despite hidden reasoning
A Reddit user noticed Claude Opus 4.6 in the Claude Code desktop app sometimes appeared to skip extended thinking on hard questions while thinking on trivial ones. Inspecting session JSONL files revealed thinking blocks exist on every turn — the 'missing' ones simply have empty text, with large signatures and output token counts. The finding suggests the reasoning runs but isn't always surfaced to the user.
A Reddit user noticed Claude Opus 4.6 in the Claude Code desktop app sometimes appeared to skip extended thinking on hard questions while thinking on trivial ones. Inspecting session JSONL files revealed thinking blocks exist on every turn — the 'missing' ones simply have empty text, with large signatures and output token counts. The finding suggests the reasoning runs but isn't always surfaced to the user.
61 engagement2 sourcesredditMon, Aug 31, 2026, 03:06 AM