Developments·Benchmark
Benchmarking retrieval for agents on messy real-world company knowledge
Benchmarking retrieval for agents on messy real-world company knowledge
28 engagement1 sourceFri, Oct 2, 2026, 01:37 PM
Entities
Related
Social·Community
kapa.ai details a year of incremental sync across 20+ enterprise RAG source types: what worked and what still breaks
A kapa.ai founder (YC S23) posted a retrospective on moving thousands of hosted RAG knowledge indexes from nightly reindexing to incremental sync across 20+ source types. The post explains per-source sync cadences, content-vs-metadata change detection, delayed deletion handling, and human review for mass-change crawls. Practitioners building retrieval pipelines get a concrete accounting of what still breaks at scale.
A kapa.ai founder (YC S23) posted a retrospective on moving thousands of hosted RAG knowledge indexes from nightly reindexing to incremental sync across 20+ source types. The post explains per-source sync cadences, content-vs-metadata change detection, delayed deletion handling, and human review for mass-change crawls. Practitioners building retrieval pipelines get a concrete accounting of what still breaks at scale.
7 engagement1 sourceredditTue, Sep 29, 2026, 12:30 PM
Developments·Benchmark
Show HN: An AI office-work benchmark, and 4 bugs we found in our own judge
A developer shared an AI office-work benchmark on Hacker News, revealing four bugs discovered in their own evaluation judge. The post highlights the challenges of building reliable benchmarks for AI agents in office tasks.
A developer shared an AI office-work benchmark on Hacker News, revealing four bugs discovered in their own evaluation judge. The post highlights the challenges of building reliable benchmarks for AI agents in office tasks.
1 engagement1 sourcehackernewsWed, Aug 5, 2026, 07:00 AM
Developments·Benchmark
Percona publishes benchmarking of vector indexes
Percona published a benchmark of vector indexes, drawing discussion on Hacker News. The work compares vector index implementations, offering practitioners data for choosing similarity-search infrastructure.
Percona published a benchmark of vector indexes, drawing discussion on Hacker News. The work compares vector index implementations, offering practitioners data for choosing similarity-search infrastructure.
23 engagement1 sourcehackernewsFri, Aug 28, 2026, 08:27 AM
Developments·Benchmark
Terminal-Bench-Science launches to evaluate AI agents on scientific research workflows
Terminal-Bench-Science, a new benchmark for evaluating AI agents on scientific research workflows, has been released. It aims to measure agent performance on tasks like literature review, experiment design, and data analysis, providing a standardized test for AI in research settings.
Terminal-Bench-Science, a new benchmark for evaluating AI agents on scientific research workflows, has been released. It aims to measure agent performance on tasks like literature review, experiment design, and data analysis, providing a standardized test for AI in research settings.
147 engagement1 sourcehackernewsFri, Aug 28, 2026, 12:06 AM
Developments·Benchmark
Real-SWE: Benchmarking AI models on private, real-world, enterprise codebases
Real-SWE, a new benchmark from Specific, evaluates AI coding models on private, real-world enterprise codebases rather than public repositories. It surfaced via Hacker News and a dedicated site on 2026-09-12, drawing community attention as a harder, more realistic test of coding agents.
Real-SWE, a new benchmark from Specific, evaluates AI coding models on private, real-world enterprise codebases rather than public repositories. It surfaced via Hacker News and a dedicated site on 2026-09-12, drawing community attention as a harder, more realistic test of coding agents.
583 engagement2 sourcesreddit, hackernewsSat, Sep 12, 2026, 07:50 PM