Releases·Tool Release
Wontopos dev nears completion of Glasshouse, an open memory benchmark for LLMs, seeking companies and devs
A developer at Wontopos, a memory API company, reports their open-source memory benchmark 'Glasshouse' is near completion. The repo currently contains rules and structure rather than benchmark files, with the benchmark itself to be built via public PRs. The author is soliciting companies and developers to contribute and is transparent about their conflict of interest.
5 engagement1 sourceSun, Sep 13, 2026, 12:05 PM
AI Overview
- Wontopos engineer, whose company builds a memory API, says the Glasshouse benchmark is near completion with judge, prompts, and per-question logs public
- Glasshouse repo currently contains only rules and structure; the benchmark itself will be built out through public PRs with submission and change policies
- Builder is recruiting companies and developers via comments or DMs to help build Glasshouse out
Entities
Related
Developments·Benchmark
Glasshouse v0.1 memory benchmark released: 2,847 questions, 1.97M-token conversations, 10 languages
A developer released Glasshouse v0.1, an open-source memory benchmark addressing complaints that vendor-published memory numbers don't replicate and that grader-model choice shifts results more than the systems being compared. It tests long-conversation memory across scale, language, and multimodal axes with no headline score. The benchmark is live with real files but has no submissions yet, and the author is soliciting community results and PRs.
A developer released Glasshouse v0.1, an open-source memory benchmark addressing complaints that vendor-published memory numbers don't replicate and that grader-model choice shifts results more than the systems being compared. It tests long-conversation memory across scale, language, and multimodal axes with no headline score. The benchmark is live with real files but has no submissions yet, and the author is soliciting community results and PRs.
6 engagement1 sourceredditTue, Sep 22, 2026, 08:12 AM
Social·Community
Memory-infrastructure dev proposes open LongMemEval/Beam benchmark with disclosed readers, judges, and Apache 2.0 terms
A Reddit user invites others to jointly build a fair, open benchmark for long-memory/long-context evaluation, criticizing the status quo where LongMemEval and Beam ship datasets and scripts with no leaderboard or submission path, leaving each team to self-report scores. The proposal includes draft rules on disclosure and reproducibility; nothing is built yet and the poster is soliciting input.
A Reddit user invites others to jointly build a fair, open benchmark for long-memory/long-context evaluation, criticizing the status quo where LongMemEval and Beam ship datasets and scripts with no leaderboard or submission path, leaving each team to self-report scores. The proposal includes draft rules on disclosure and reproducibility; nothing is built yet and the poster is soliciting input.
8 engagement1 sourceredditTue, Sep 8, 2026, 02:43 AM
Social·Community
Memory API startup founder asks devs where they'd use a memory layer, aiming to rival Mem0 and Zep
A founder building a memory API startup, competing in the same space as Mem0 and Zep, asked developers on Reddit where they would use such an API and what it would need to do. After two months running the product, the founder is questioning whether a hosted memory layer is actually what developers want. The long-term ambition is to become standard B2B infrastructure for AI memory, but the immediate goal is real developer adoption and visibility.
A founder building a memory API startup, competing in the same space as Mem0 and Zep, asked developers on Reddit where they would use such an API and what it would need to do. After two months running the product, the founder is questioning whether a hosted memory layer is actually what developers want. The long-term ambition is to become standard B2B infrastructure for AI memory, but the immediate goal is real developer adoption and visibility.
39 engagement2 sourcesredditWed, Sep 30, 2026, 10:14 AM
Social·Community
Wontopos launches memory API for AI agents, seeks feedback on $2/$8 per million token pricing
A Wontopos employee asked developers for candid feedback on the company's agent memory API, launched roughly three months earlier with little traction. The post discloses pricing tiers and self-measured benchmark scores and asks how it compares to Mem0, Zep, or a plain file. It functions as feedback-seeking rather than a launch announcement.
A Wontopos employee asked developers for candid feedback on the company's agent memory API, launched roughly three months earlier with little traction. The post discloses pricing tiers and self-measured benchmark scores and asks how it compares to Mem0, Zep, or a plain file. It functions as feedback-seeking rather than a launch announcement.
5 engagement1 sourceredditThu, Oct 15, 2026, 08:33 AM
Social·Community
Community call to build a harness-only benchmark
A Reddit user proposed a community-driven benchmark focused solely on harness performance, aiming to create a leaderboard across diverse real-world tasks grouped by underlying models and reasoning efforts. The project would be open to contributions and decisions made by a group rather than a single person.
A Reddit user proposed a community-driven benchmark focused solely on harness performance, aiming to create a leaderboard across diverse real-world tasks grouped by underlying models and reasoning efforts. The project would be open to contributions and decisions made by a group rather than a single person.
77 engagement2 sourcesredditWed, Aug 5, 2026, 10:54 AM