Releases·Tool Release
Creator launches open-source Agent Amusement Park with three stateful agent-evaluation environments
An open-source experiment called Agent Amusement Park introduces three deterministic, stateful environments for evaluating AI agents beyond binary task completion. The environments preserve what the agent observed, chose, triggered, and changed, scoring behavior against evidence steps rather than just outcomes. This matters to practitioners building agent benchmarks where process quality and safety matter as much as final success.
6 engagement1 sourceTue, Sep 1, 2026, 06:52 PM
AI Overview
- Three environments: a bureaucracy with conflicting instructions and delayed approval, a negotiation market with verification and escrow traps, and a browser refund flow with shifting controls and permission hazards
- Each run keeps a complete trace and scores rules against evidence steps rather than only final outcomes
- Nominal task success is weighted at 60, implying non-task behavioral factors carry substantial scoring weight
- The project is open-source and created by the poster
Related
Products·launched
Seahaven: open source framework for building RL environments for LLM agents
Seahaven is an open source framework for building RL/eval environments for optimizing long-running LLM agents. It provides realistic data, stateful writes, and resettable starting points so agent iterations (prompts, tools, subagents) can be scored consistently. It targets developers tuning agents who find production/staging and hand-written mocks inadequate.
Seahaven is an open source framework for building RL/eval environments for optimizing long-running LLM agents. It provides realistic data, stateful writes, and resettable starting points so agent iterations (prompts, tools, subagents) can be scored consistently. It targets developers tuning agents who find production/staging and hand-written mocks inadequate.
45 engagement1 sourceredditThu, Oct 8, 2026, 02:19 PM
Developments·Paper
Thinkingbox: a sandbox and benchmark for agents in stateful business workflows
Researchers introduced Thinkingbox, a sandbox and benchmark for evaluating agents in stateful business workflows. It provides isolated MCP-compatible tool sessions and emphasizes persistent state transitions over single-turn correctness.
Researchers introduced Thinkingbox, a sandbox and benchmark for evaluating agents in stateful business workflows. It provides isolated MCP-compatible tool sessions and emphasizes persistent state transitions over single-turn correctness.
0 engagement1 sourcearxivThu, Aug 20, 2026, 07:37 AM
Social·Community
Developer builds open-source analytics separating agent execution success from task success
Developer shares open-source analytics separating agent execution success from task success
Developer shares open-source analytics separating agent execution success from task success
3 engagement1 sourceredditMon, Sep 21, 2026, 11:00 AM
Products·building
Battle Agents: Arena to stress-test AI agents in controlled scenarios
Battle Agents is a platform that runs AI agents through controlled, adversarial scenarios to expose failures beyond the happy path. It provides tools, constraints, and detailed inspection of decisions, tool calls, and recovery, with scoring. It helps developers evaluate agent robustness and identify weaknesses.
Battle Agents is a platform that runs AI agents through controlled, adversarial scenarios to expose failures beyond the happy path. It provides tools, constraints, and detailed inspection of decisions, tool calls, and recovery, with scoring. It helps developers evaluate agent robustness and identify weaknesses.
3 engagement1 sourceredditSun, Aug 16, 2026, 03:29 PM
Developments·Paper
WebPageBench: Event-Level Verification and Controlled UI-Variant Generation for Web Agents
Researchers introduce WebPageBench, an open framework that verifies web-agent task success from the interface's own typed event log rather than judge models or page scraping. It pairs 152 tasks with controlled UI-variant generation to isolate sensitivity to interface form, and ships with a public leaderboard of 24 model-harness pairs. On the leaderboard, agents' declared completion diverges from log-confirmed success by up to 41 points, highlighting overreporting in current web agents.
Researchers introduce WebPageBench, an open framework that verifies web-agent task success from the interface's own typed event log rather than judge models or page scraping. It pairs 152 tasks with controlled UI-variant generation to isolate sensitivity to interface form, and ships with a public leaderboard of 24 model-harness pairs. On the leaderboard, agents' declared completion diverges from log-confirmed success by up to 41 points, highlighting overreporting in current web agents.
0 engagement1 sourcearxivMon, Sep 28, 2026, 12:24 PM