AI & Agents
23 posts tagged “AI & Agents”.
Web scraping in Dify, with the Zenrows plugin
The official Zenrows plugin for Dify adds Fetch, Extract and Batch to the tool picker, so your workflows and agents read the real page instead of a block page.
Yuvraj Chandra4 min readBuild an AI Lead Generation Agent With OpenAI Agents SDK and Zenrows
Most lead agents only rank what a database already holds. This one reads a bot-protected directory live, pulls 78 companies off one page, enriches each from its own site, and scores it against your ICP.
Ifeanyi Iheagwara27 min readBest Web Scraping API for AI Agents in 2026
We put four scraping APIs through 800 requests against DataDome, Cloudflare and Akamai. Two of them never returned a single usable page from the DataDome target.
Goodness E. Eboh18 min readBuild a Web-Aware TypeScript Agent With Mastra and Zenrows
A custom Zenrows Fetch tool gives a Mastra agent reliable access to dynamic and protected pages, returning Markdown to reason over or structured JSON when you need fields. Includes the typed tool, the agent, and a workflow for bulk URLs.
Ifeanyi Iheagwara18 min readWeb data for LLM fine-tuning, a practical 2026 guide
A fine-tuning seed is a few hundred rows, so one bad extraction is a measurable share of the training signal. Collect it from sources that block standard crawlers.
Yuvraj Chandra15 min readHow to scrape protected sites in Cursor with Zenrows MCP
Cursor's built-in web access reads cached snapshots, so it misses JS-rendered data and gets blocked on protected pages. Add Zenrows MCP and run three workflows on live sites.
Yuvraj Chandra10 min readIntegrating Zenrows into smolagents for production web access
smolagents' VisitWebpageTool fetches with plain requests, so protected pages return a bot check your agent cannot tell from real data. Replace it with a Zenrows @tool.
Yuvraj Chandra10 min readBuild a web-aware agent with OpenAI Agents SDK and Zenrows
Register Zenrows Fetch as a @function_tool next to OpenAI's built-in WebSearchTool, and let one agent pick discovery or full-page retrieval per turn.
Yuvraj Chandra11 min readBuild a web research multi-agent system with AG2 and Zenrows
In an AG2 group chat every tool response enters the shared history. Register Zenrows Fetch as a typed tool so clean content, not a failed fetch, is what circulates.
Yuvraj Chandra15 min readHow to Pull Live Web Data into Claude, Cursor, and Any AI Agent with Zenrows MCP
Learn how to connect Zenrows MCP to Claude Desktop, Cursor, and other AI agents to fetch live web data, render JavaScript, and scrape protected websites.
The Zenrows team9 min readWhat is an MCP Server? And Why AI Agents Need Web Scraping
Discover what MCP servers are and why web scraping is the top tool for AI agents, including a practical guide to adding a scraping MCP server.
Denis Kuria9 min readPuppeteer MCP Server: LLM Web Scraping Guide
Learn how to set up the open-source Puppeteer MCP for AI web scraping, its limitations on protected sites, and how to solve them.
Yuvraj Chandra13 min readPlaywright MCP: AI Web Scraping Guide
Learn how to set up Playwright MCP for AI web scraping, including its use cases and limitations, and how to overcome anti-bots for reliable, AI-ready data extraction.
Denis Kuria11 min read7 Best MCP Servers for Web Scraping in 2026
How Zenrows, Firecrawl, Bright Data, Oxylabs and more compare as MCP servers for web scraping: anti-bot bypass, JS rendering, AI-ready output.
Idowu Omisola21 min readBest AI Web Scraping Tools for LLM & RAG Pipelines (2026)
Compare the 7 best AI web scraping tools for LLM and RAG pipelines, including Crawl4AI, Firecrawl, Zenrows, and more, with pros, cons, and when to use each.
Denis Kuria16 min readIntroducing Zenrows MCP: The Universal Connector for AI Web Scraping
Stop hitting anti-bot block messages in your AI tool. Zenrows MCP brings our scraping infrastructure directly into your AI tool's conversation. 1,000 free URLs to start.
Yuvraj Chandra5 min readHow to Give Your AI Agent Real Web Access with Zenrows MCP
Connect your AI agents to the live web with Zenrows MCP. Bypass anti-bots, render JavaScript, and deliver LLM-ready data to any MCP-compatible tool.
Idowu Omisola13 min readWeb Scraping for RAG with Crawl4AI
Learn RAG data ingestion with Crawl4AI. Convert messy HTML to LLM-ready Markdown, implement semantic chunking, and bypass anti-bots at production scale.
Denis Kuria25 min readHow to Extract Web Data for AI Training Pipelines
Learn to slash LLM costs using Markdown extraction, implement production-ready data provenance, and scale your training pipelines with web data.
Idowu Omisola20 min readHow to Build a Real Estate AI Agent Using LangChain and Zenrows
Build an AI agent to scrape real estate listings automatically using LangChain and Zenrows. Collect property data without getting blocked.
Denis Kuria18 min readHow to Build an AI Agent with Web Access Using Flowise and Zenrows
Learn to build an AI agent with access to web data. Avoid blocks and enhance your agent’s performance with rich, up-to-date context via web scraping.
Yuvraj Chandra9 min readHow to Integrate Web Scraping Into Your n8n Workflows
Learn to enhance your n8n workflow with real-time web data. Optimize your pipeline for consistent web access, scalability, and high-quality results.
Idowu Omisola12 min readWeb Scraping With LangChain and Zenrows
Learn how to integrate web scraping into your LangChain workflows. Discover step-by-step guides and real-world examples for enhancing your AI agents with up-to-date web data.
Idowu Omisola8 min read