Releases·Tool Release
Dev open-sources OpticParse toolkits for LangChain and LlamaIndex adding zero-CSS web scraping and phishing detection to agents
A developer released an open-source toolkit for LangChain and LlamaIndex aimed at giving autonomous agents and RAG pipelines live web access. It targets three stated bottlenecks: context bloat from raw HTML, brittle CSS/XPath selectors, and agents falling into malicious link traps.
3 engagement1 sourceFri, Sep 4, 2026, 04:44 AM
AI Overview
- OpticParse converts JavaScript-heavy pages to token-efficient Markdown with 96% noise reduction, eliminating brittle CSS/XPath selectors.
- PhishVisionTool heuristically scans URLs for brand impersonation, zero-day phishing kits, and crypto wallet drainers before agents interact.
- Installs via pip as langchain-opticparse and llama-index-tools-opticparse, and works with Claude Desktop/Cursor (MCP) and ElizaOS.
- Includes a 3-agent research swarm demo (Scout, Sentinel, Analyst) in examples/autonomous_market_researcher.py, plus a Hugging Face benchmark space.
Related
Releases·Tool Release
Developer releases LangChain package for structured web extraction
A developer published a LangChain tool package that wraps a RapidAPI service to return structured fields (emails, security headers, SEO score, tech stack) instead of raw Markdown or HTML. The package includes four tools for extraction, markdown conversion, and contact discovery.
A developer published a LangChain tool package that wraps a RapidAPI service to return structured fields (emails, security headers, SEO score, tech stack) instead of raw Markdown or HTML. The package includes four tools for extraction, markdown conversion, and contact discovery.
1 engagement1 sourceredditSun, Aug 23, 2026, 03:29 PM
Social·Community
web scraping for langchain agents: DIY (playwright/bs4) vs managed APIs (contextdev, firecrawl), my experience
A Reddit user shares hands-on experience building web-scraping tools for LangChain agents, comparing DIY Playwright/BeautifulSoup stacks with managed APIs like Firecrawl and context.dev. The writeup finds DIY viable for fixed workflows but costly and maintenance-heavy for open-ended agent web search, where managed APIs' built-in rendering, proxy rotation, and LLM-ready markdown outweigh per-request costs.
A Reddit user shares hands-on experience building web-scraping tools for LangChain agents, comparing DIY Playwright/BeautifulSoup stacks with managed APIs like Firecrawl and context.dev. The writeup finds DIY viable for fixed workflows but costly and maintenance-heavy for open-ended agent web search, where managed APIs' built-in rendering, proxy rotation, and LLM-ready markdown outweigh per-request costs.
27 engagement1 sourceredditSun, Sep 13, 2026, 08:33 PM
Products·launched
LookaCrawler: local, token-efficient web crawler for LLM workflows with native MCP server
LookaCrawler is an open-source local web crawler that strips HTML boilerplate (SVGs, navs, ads, scripts) before Markdown extraction to cut token usage for LLMs. It ships a native MCP server so Claude Desktop, Cursor, and agent frameworks can call it directly. It targets developers who currently waste thousands of tokens per scraped page on boilerplate.
LookaCrawler is an open-source local web crawler that strips HTML boilerplate (SVGs, navs, ads, scripts) before Markdown extraction to cut token usage for LLMs. It ships a native MCP server so Claude Desktop, Cursor, and agent frameworks can call it directly. It targets developers who currently waste thousands of tokens per scraped page on boilerplate.
2 engagement1 sourceredditFri, Aug 28, 2026, 11:47 PM
Products·launched
AST-based Web-to-Markdown Crawler for RAG pipelines
An Apify Actor that converts web pages to clean Markdown using AST parsing, preserving document hierarchy for better RAG retrieval. It removes noise like navbars and scripts, and outputs structured Markdown that improves chunking and retrieval accuracy. Built for developers building RAG pipelines or AI agents that consume web content.
An Apify Actor that converts web pages to clean Markdown using AST parsing, preserving document hierarchy for better RAG retrieval. It removes noise like navbars and scripts, and outputs structured Markdown that improves chunking and retrieval accuracy. Built for developers building RAG pipelines or AI agents that consume web content.
0304 engagement3 sourcesredditThu, Jul 23, 2026, 03:01 PM
Releases·Tool Release
SkillShield: Open-source static analysis tool for LangChain tool security
A developer released SkillShield, an open-source static analysis tool that scans LangChain agent tools and skills for malicious code, prompt injections, and excessive permissions before installation. It parses SKILL.md manifests and repository contents to catch hidden bash scripts and other risks, addressing the challenge of vetting third-party tools from open-source repos.
A developer released SkillShield, an open-source static analysis tool that scans LangChain agent tools and skills for malicious code, prompt injections, and excessive permissions before installation. It parses SKILL.md manifests and repository contents to catch hidden bash scripts and other risks, addressing the challenge of vetting third-party tools from open-source repos.
1 engagement1 sourceredditTue, Jul 28, 2026, 09:39 AM