---
title: "Build a Live Web Data Agent With Agno and Zenrows"
description: "Register Zenrows Fetch as a typed Agno tool, keep the API key out of the model's view, and pass a validated page from a web data agent to a market intelligence agent. On a Cloudflare-protected page, the tool returns the content a direct request is refused."
---
# Build a Live Web Data Agent With Agno and Zenrows

> Register Zenrows Fetch as a typed Agno tool, keep the API key out of the model's view, and pass a validated page from a web data agent to a market intelligence agent. On a Cloudflare-protected page, the tool returns the content a direct request is refused.

Source: https://www.zenrows.com/blog/live-web-data-agent-agno-zenrows

[Agno](https://docs.agno.com/agents/overview) manages the workflow between your agents, but the quality of their output still depends on the web data they receive. A normal HTTP request may return a blocked response or miss product values added by JavaScript, leaving the agents with incomplete information.

[Zenrows](https://www.zenrows.com/) solves these access problems for your agents. It retrieves content from protected websites and renders JavaScript before returning the page data. You can connect it to Agno through a typed Python tool, giving the agent [a direct way to request current web content](/glossary/ai-agents-and-the-web/how-to-give-an-agent-live-web-access).

In this tutorial, you'll use this setup to build a two-agent market intelligence workflow. The web data agent will collect current product information through Zenrows, after which the processing agent will return a validated market snapshot. A direct-request comparison makes the difference visible: on a Cloudflare-protected page, the direct request gets HTTP `403` and a "Just a moment..." challenge, while the same URL sent through the Agno tool returns HTTP `200` with the page content.

**The short version: Agno is an open-source Python framework for building agents. To give an Agno agent live web data, register a Python function that calls Zenrows Fetch with `mode=auto` as a tool, keep the API key inside that function so only `url` reaches the model, and validate what the web data agent returns before a second agent extracts the fields you need.** The Zenrows outputs below come from runs on October 21, 2026, and the complete project is in the [agno-zenrows-live-web-agent repository on GitHub](https://github.com/ZenRows/agno-zenrows-live-web-agent).

## Why Use Zenrows as an Agno Tool

Zenrows exposes web access through an HTTP endpoint, which maps cleanly to an Agno function tool. Your Python function receives the target URL and calls Zenrows with an API key stored in the application environment. Only the URL appears in the tool schema, keeping the credential outside the model context.

With `mode=auto`, Fetch selects the request configuration for each target and can apply [JavaScript rendering](/glossary/web-data-access/what-is-javascript-rendering) or Premium Proxies when required. The Agno tool only needs to send the target URL and API key, leaving the access configuration to Fetch.

Zenrows handles page retrieval and browser rendering in this project. The first Agno agent calls the tool to collect the product data, then the processing agent converts the returned content into the `MarketSnapshot` structure used by the application.

Agno agents can also reach Zenrows through the [Zenrows MCP server](https://docs.zenrows.com/mcp/overview). Agno's `MCPTools` connects to the remote server at `https://mcp.zenrows.com/mcp` and hands the agent the full set of [Zenrows MCP](/mcp) tools, so you can ask it in plain English to read a page, extract product details, or queue a list of URLs as one job. This tutorial uses a function tool instead because the workflow needs the typed `FetchedPage` handoff between its two agents. If your agent runs without a Zenrows account, it can also [pay for Fetch per request on its own](/blog/ai-agents-can-pay-for-zenrows) over x402, MPP or a card.

To connect Zenrows to the web data agent, you need to configure the project and wrap the Fetch request in an Agno tool.

## Setting Up Zenrows as an Agno Tool

### Prerequisites

Before you begin, make sure you have:

- Python 3.11, 3.12 or 3.13 installed.
- A [Zenrows account](https://app.zenrows.com/register) and API key.
- An OpenAI API key.
- A public page that you're authorized to access.

The Zenrows free plan currently includes 5,000 monthly credits, so you can run the tutorial without starting a paid subscription. The project pins Agno 3.0.10, the version this tutorial was tested with.

### Create the Project

Create a project directory, then set up a virtual environment:

```bash
mkdir agno-zenrows-live-web-agent
cd agno-zenrows-live-web-agent
python -m venv .venv
source .venv/bin/activate
```

Use the following activation command on Windows PowerShell:

```powershell
.venv\Scripts\Activate.ps1
```

Create a source directory and an empty `__init__.py` file:

```bash
mkdir src
mkdir src/live_web_agent
python -c "from pathlib import Path; Path('src/live_web_agent/__init__.py').touch()"
```

Create `pyproject.toml` in the project root:

```toml
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "agno-zenrows-live-web-agent"
version = "0.1.0"
description = "A two-agent Agno workflow that retrieves protected web data through Zenrows."
readme = "README.md"
requires-python = ">=3.11,<3.14"
dependencies = [
  "agno==3.0.10",
  "beautifulsoup4==4.14.2",
  "openai==3.16.2",
  "pydantic==2.12.5",
  "python-dotenv==1.2.1",
  "requests==2.32.5",
]

[project.optional-dependencies]
dev = [
  "pytest==9.0.2",
  "pytest-cov==7.0.0",
  "ruff==0.15.6",
]

[project.scripts]
live-web-agent = "live_web_agent.cli:main"

[tool.hatch.build.targets.wheel]
packages = ["src/live_web_agent"]

[tool.pytest.ini_options]
testpaths = ["tests"]

[tool.ruff]
line-length = 88
target-version = "py311"
```

Install the project dependencies:

```bash
python -m pip install --upgrade pip
python -m pip install -e ".[dev]"
```

### Create the Project Structure

The finished application will use the following structure:

```text
agno-zenrows-live-web-agent/
├── src/
│   └── live_web_agent/
│       ├── __init__.py
│       ├── agents.py
│       ├── cli.py
│       ├── config.py
│       ├── schemas.py
│       ├── web.py
│       └── workflow.py
├── tests/
├── .env
├── .env.example
├── .gitignore
└── pyproject.toml
```

The `tests/` folder and `.env.example` come with the GitHub repository; the steps below create everything else.

### Configure the API Keys

Create a `.env` file in the project root:

```text
ZENROWS_API_KEY=replace_with_your_zenrows_api_key
OPENAI_API_KEY=replace_with_your_openai_api_key
OPENAI_MODEL=gpt-5-mini
TARGET_URL=https://www.scrapingcourse.com/cloudflare-challenge
```

Add `.env` to your `.gitignore` file so neither API key enters version control.

### Define the Zenrows Request

Create `src/live_web_agent/web.py` and add the following code:

```python
from __future__ import annotations

from datetime import UTC, datetime
from typing import Any

import requests
from bs4 import BeautifulSoup

ZENROWS_ENDPOINT = "https://api.zenrows.com/v1/"


def page_to_text(html: str, limit: int = 12_000) -> tuple[str | None, str]:
    soup = BeautifulSoup(html, "html.parser")
    for element in soup(["script", "style", "noscript", "svg"]):
        element.decompose()
    title = soup.title.get_text(" ", strip=True) if soup.title else None
    text = " ".join(soup.get_text(" ", strip=True).split())
    metadata: list[str] = []
    for element in soup.select("[itemprop][content]"):
        itemprop = element.get("itemprop")
        content = element.get("content")
        if itemprop in {"priceCurrency", "availability"} and content:
            value = f"{itemprop}: {content}"
            if value not in metadata:
                metadata.append(value)
    if metadata:
        text = f"{text} Structured metadata: {'; '.join(metadata)}"
    return title, text[:limit]


def fetch_with_zenrows(
    url: str,
    api_key: str,
    *,
    session: Any = requests,
) -> dict[str, object]:
    """Fetch a public page through Zenrows and return compact page text."""
    response = session.get(
        ZENROWS_ENDPOINT,
        params={
            "url": url,
            "apikey": api_key,
            "mode": "auto",
        },
        timeout=60,
    )
    response.raise_for_status()
    title, text = page_to_text(response.text)
    return {
        "source_url": url,
        "status_code": response.status_code,
        "title": title,
        "text": text,
        "fetched_at": datetime.now(UTC).isoformat(),
    }


def fetch_direct(url: str, *, session: Any = requests) -> dict[str, object]:
    """Fetch the same page without Zenrows for a transparent comparison."""
    response = session.get(
        url,
        headers={"User-Agent": "Mozilla/5.0 (compatible; tutorial-validator/1.0)"},
        timeout=30,
    )
    title, text = page_to_text(response.text, limit=500)
    return {
        "source_url": url,
        "status_code": response.status_code,
        "title": title,
        "text_preview": text,
    }
```

The request uses `mode=auto`, allowing Fetch to select the access configuration for the target and apply JavaScript rendering or Premium Proxies when needed. Before the response reaches the agent, `page_to_text()` strips non-content elements and shortens the text to 12,000 characters. It also appends the Schema.org currency and availability values the processing agent needs. The tool reads HTML rather than a Markdown response because those Schema.org values sit in `itemprop` attributes, which only the HTML carries.

`fetch_direct()` sends the same URL without Zenrows so the CLI can print both responses side by side.

Now that the request function returns compact page data, you can register it with the web data agent through a wrapper that keeps the API key inside the application.

## Building a Two-Agent Market Intelligence System

You'll use one agent to fetch the page and another to extract product data from the returned content. A Python function calls them in that order every time and passes a validated `FetchedPage` from the first agent to the second. The complete project and test suite are available in the [GitHub repository](https://github.com/ZenRows/agno-zenrows-live-web-agent).

| Agent | Tools | Output schema | Job |
| --- | --- | --- | --- |
| Web data agent | `fetch_with_zenrows` | `FetchedPage` | Fetches the supplied URL through Zenrows and returns the page data |
| Market intelligence agent | None | `MarketSnapshot` | Extracts product name, price, currency and availability, with evidence |

![The two-agent workflow: the web data agent fetches the page through Zenrows Fetch with mode=auto, the workflow validates the FetchedPage, the market intelligence agent extracts the fields with no tools, and the workflow validates the MarketSnapshot](/blog/_img/agno-two-agent-market-intelligence-workflow.png)

### Load the Application Settings

Create `src/live_web_agent/config.py`:

```python
from __future__ import annotations

import os
from dataclasses import dataclass


@dataclass(frozen=True)
class Settings:
    zenrows_api_key: str
    openai_api_key: str
    openai_model: str = "gpt-5-mini"

    @classmethod
    def from_env(cls) -> "Settings":
        # stop before creating either agent when a credential is missing
        missing = [
            name
            for name in ("ZENROWS_API_KEY", "OPENAI_API_KEY")
            if not os.getenv(name)
        ]
        if missing:
            names = ", ".join(missing)
            raise RuntimeError(f"Missing required environment variables: {names}")
        # use gpt-5-mini unless OPENAI_MODEL overrides it
        return cls(
            zenrows_api_key=os.environ["ZENROWS_API_KEY"],
            openai_api_key=os.environ["OPENAI_API_KEY"],
            openai_model=os.getenv("OPENAI_MODEL", "gpt-5-mini"),
        )
```

`Settings.from_env()` stops the application early if either required credential is missing, while the frozen dataclass keeps the loaded values unchanged.

### Define the Agent Output Schemas

Create `src/live_web_agent/schemas.py`:

```python
from __future__ import annotations

from datetime import datetime
from urllib.parse import urlparse

from pydantic import BaseModel, ConfigDict, Field, field_validator


def _validate_http_url(value: str) -> str:
    parsed = urlparse(value)
    if parsed.scheme not in {"http", "https"} or not parsed.netloc:
        raise ValueError("source_url must be a valid HTTP or HTTPS URL")
    return value


def _keep_source_url_schema_plain(schema: dict[str, object]) -> None:
    properties = schema.get("properties")
    if isinstance(properties, dict):
        source_url = properties.get("source_url")
        if isinstance(source_url, dict):
            # remove metadata that the structured-output schema does not need
            source_url.pop("title", None)


class FetchedPage(BaseModel):
    # keep source_url as a plain string in the OpenAI output schema
    model_config = ConfigDict(json_schema_extra=_keep_source_url_schema_plain)

    source_url: str
    status_code: int
    title: str | None = None
    text: str
    fetched_at: datetime

    # validate the URL in Python without adding format: "uri" to the schema
    _source_url_is_http = field_validator("source_url")(_validate_http_url)


class MarketSnapshot(BaseModel):
    model_config = ConfigDict(json_schema_extra=_keep_source_url_schema_plain)

    product_name: str | None = None
    current_price: float | None = None
    currency: str | None = None
    availability: str | None = None
    source_url: str
    collected_at: datetime
    evidence: list[str] = Field(default_factory=list, max_length=5)

    _source_url_is_http = field_validator("source_url")(_validate_http_url)
```

Before passing the first agent's response downstream, the workflow validates it against `FetchedPage`, so malformed data stops at retrieval. After extraction, the workflow checks the final record against `MarketSnapshot`; any product field absent from the source page can remain null.

The `source_url` field uses `str` instead of Pydantic's `HttpUrl`. In the tested Agno 3.0.10 setup with gpt-5-mini, `HttpUrl` added `format: "uri"` to the generated JSON Schema, which caused OpenAI to reject the structured-output request. The `_validate_http_url()` function retains HTTP and HTTPS validation without adding that schema keyword.

### Create Both Agents

Create `src/live_web_agent/agents.py`:

```python
from __future__ import annotations

from agno.agent import Agent
from agno.models.openai import OpenAIChat

from .config import Settings
from .schemas import FetchedPage, MarketSnapshot
from .web import fetch_with_zenrows


def build_agents(settings: Settings) -> tuple[Agent, Agent]:
    # both agents use the model selected in the application settings
    model = OpenAIChat(
        id=settings.openai_model,
        api_key=settings.openai_api_key,
    )

    # capture the API key here so the model supplies only the URL
    def fetch_with_zenrows_tool(url: str) -> dict[str, object]:
        """Fetch a public page through Zenrows and return compact page text."""
        try:
            return fetch_with_zenrows(
                url,
                api_key=settings.zenrows_api_key,
            )
        except Exception:
            # request exceptions can include the complete query string
            # keep Zenrows credentials out of logs and model tool results
            raise RuntimeError("Zenrows request failed") from None

    # expose a stable public tool name to Agno
    fetch_with_zenrows_tool.__name__ = "fetch_with_zenrows"

    # require the first agent to fetch the supplied page through Zenrows
    fetch_agent = Agent(
        name="Web data agent",
        model=model,
        tools=[fetch_with_zenrows_tool],
        output_schema=FetchedPage,
        instructions=[
            "Always call fetch_with_zenrows for the supplied URL.",
            "Return only the fetched page data. Do not invent missing content.",
        ],
    )
    # give the second agent no tools because it only processes fetched data
    processing_agent = Agent(
        name="Market intelligence agent",
        model=model,
        output_schema=MarketSnapshot,
        instructions=[
            "Extract only values supported by the supplied page data.",
            "Use null for missing fields and quote short evidence snippets.",
        ],
    )
    return fetch_agent, processing_agent
```

The nested `fetch_with_zenrows_tool()` function exposes only `url: str` to Agno. It reads the Zenrows API key from settings when called, keeping the credentials out of the generated tool schema. This is the schema Agno 3.0.10 generates for the tool:

```json
{
  "type": "object",
  "properties": {
    "url": {
      "type": "string"
    }
  },
  "required": [
    "url"
  ]
}
```

The instructions make the web data agent fetch the supplied URL and return a `FetchedPage`, which the processing agent converts into a `MarketSnapshot`.

### Connect the Agents

Create `src/live_web_agent/workflow.py`:

```python
from __future__ import annotations

import json
from typing import Any

from pydantic import ValidationError

from .schemas import FetchedPage, MarketSnapshot


class WorkflowAgentError(RuntimeError):
    """Raised when an Agno agent or its model provider fails."""


def _run_agent(agent: Any, prompt: str, role: str) -> Any:
    try:
        response = agent.run(prompt)
    except Exception as exc:
        # include the failed agent's role in the application error
        raise WorkflowAgentError(f"{role} failed: {exc}") from exc

    status = getattr(response, "status", None)
    status_value = getattr(status, "value", status)
    # catch provider errors returned as Agno responses
    if isinstance(status_value, str) and status_value.upper() == "ERROR":
        detail = getattr(response, "content", None) or "unknown provider error"
        raise WorkflowAgentError(f"{role} failed: {detail}")
    return response


def _content(
    response: Any,
    schema: type[FetchedPage] | type[MarketSnapshot],
) -> Any:
    value = response.content
    if isinstance(value, schema):
        return value
    try:
        # parse JSON strings before handling dictionary-like values
        if isinstance(value, str):
            return schema.model_validate_json(value)
        return schema.model_validate(value)
    except ValidationError as exc:
        raise WorkflowAgentError(
            f"Agent returned invalid {schema.__name__} data: {value!r}"
        ) from exc


def run_workflow(
    url: str,
    fetch_agent: Any,
    processing_agent: Any,
) -> MarketSnapshot:
    fetch_response = _run_agent(
        fetch_agent,
        f"Fetch this public product page through Zenrows: {url}",
        "Web data agent",
    )
    # validate the fetch result before sending it to the processing agent
    page = _content(fetch_response, FetchedPage)
    # pass the validated page as JSON instead of raw HTML
    processing_response = _run_agent(
        processing_agent,
        "Extract a market snapshot from this fetched page data:\n"
        + json.dumps(page.model_dump(mode="json"), ensure_ascii=False),
        "Market intelligence agent",
    )
    return _content(processing_response, MarketSnapshot)
```

`run_workflow()` starts with the web data agent and validates its response as `FetchedPage`. It serializes that model into the next prompt, then validates the processing agent's response as `MarketSnapshot`.

The helper functions prevent provider errors from reaching Pydantic as though they were agent output. If either agent fails, the CLI receives a `WorkflowAgentError` that identifies the failed stage.

### Add the Command-Line Entry Point

Create `src/live_web_agent/cli.py`:

```python
from __future__ import annotations

import argparse
import json
import os

from dotenv import load_dotenv

from .agents import build_agents
from .config import Settings
from .web import fetch_direct, fetch_with_zenrows
from .workflow import run_workflow


def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(
        description="Run the Agno and Zenrows workflow"
    )
    # use TARGET_URL unless the reader passes --url
    parser.add_argument("--url", default=os.getenv("TARGET_URL"))
    parser.add_argument("--skip-direct", action="store_true")
    parser.add_argument("--fetch-only", action="store_true")
    return parser.parse_args()


def main() -> None:
    # load the local environment before reading TARGET_URL or API keys
    load_dotenv()
    args = parse_args()
    if not args.url:
        raise SystemExit("Pass --url or set TARGET_URL")

    # print the direct response unless --skip-direct is present
    if not args.skip_direct:
        print("Direct request comparison:")
        print(
            json.dumps(
                fetch_direct(args.url),
                indent=2,
                ensure_ascii=False,
            )
        )

    # print the Zenrows tool result without running the agents
    if args.fetch_only:
        api_key = os.getenv("ZENROWS_API_KEY")
        if not api_key:
            raise SystemExit("Set ZENROWS_API_KEY to run --fetch-only")
        print("\nZenrows tool result:")
        page = fetch_with_zenrows(args.url, api_key=api_key)
        print(json.dumps(page, indent=2, ensure_ascii=False))
        return

    settings = Settings.from_env()
    fetch_agent, processing_agent = build_agents(settings)
    result = run_workflow(args.url, fetch_agent, processing_agent)
    print("\nTwo-agent result:")
    print(result.model_dump_json(indent=2))


if __name__ == "__main__":
    main()
```

The CLI loads `.env` before reading the target URL and credentials. It runs the direct comparison unless you pass `--skip-direct`, then sends the target through the two-agent workflow and prints the validated JSON result. With `--fetch-only`, it prints the Zenrows tool result and stops before either agent runs, which lets you check the web access layer before spending model tokens.

## Zenrows Returns the Cloudflare-Protected Page the Direct Request Could Not

The URL in `.env` points to ScrapingCourse's [Cloudflare challenge](/glossary/web-data-access/what-is-a-javascript-challenge) page, which answers automated requests with a challenge instead of the page. Use the installed CLI to test both access paths and print what the Zenrows tool hands the web data agent:

```bash
live-web-agent --fetch-only
```

The run produced:

```text
Direct request comparison:
{
  "source_url": "https://www.scrapingcourse.com/cloudflare-challenge",
  "status_code": 403,
  "title": "Just a moment...",
  "text_preview": "Just a moment..."
}

Zenrows tool result:
{
  "source_url": "https://www.scrapingcourse.com/cloudflare-challenge",
  "status_code": 200,
  "title": "Cloudflare Challenge - ScrapingCourse.com",
  "text": "Cloudflare Challenge - ScrapingCourse.com Scraping Course Cloudflare Challenge You bypassed the Cloudflare challenge! :D",
  "fetched_at": "2026-10-21T08:11:16.464568+00:00"
}
```

![The same Cloudflare challenge page two ways: the direct request gets HTTP 403 and a Just a moment challenge page, while the fetch_with_zenrows tool, Zenrows Fetch with mode=auto, gets HTTP 200 and the page content](/blog/_img/agno-tool-cloudflare-direct-vs-zenrows.png)

The two requests produced different status codes. The direct request returned HTTP `403`, as shown above, while the request sent through Zenrows returned HTTP `200`. The page content received by the web data agent included `You bypassed the Cloudflare challenge! :D`.

| Access path | Status | Page title | What the agent receives |
| --- | --- | --- | --- |
| Direct request | `403` | Just a moment... | A challenge page |
| `fetch_with_zenrows` (Zenrows Fetch, `mode=auto`) | `200` | Cloudflare Challenge - ScrapingCourse.com | The page content |

### Run Both Agents on a Product Page

With access confirmed, point the workflow at a product page and let both agents run:

```bash
live-web-agent --url "https://www.scrapingcourse.com/ecommerce/product/chaz-kangeroo-hoodie/"
```

The web data agent calls the same tool. This is the tool result it receives for that page, printed with `--fetch-only --skip-direct`:

```text
Zenrows tool result:
{
  "source_url": "https://www.scrapingcourse.com/ecommerce/product/chaz-kangeroo-hoodie/",
  "status_code": 200,
  "title": "Chaz Kangeroo Hoodie – Ecommerce Test Site to Learn Web Scraping",
  "text": "Chaz Kangeroo Hoodie – Ecommerce Test Site to Learn Web Scraping Skip to navigation Skip to content Ecommerce Test Site to Learn Web Scraping ScrapingCourse.com Search for: Search Menu Shop Home Cart Checkout My account $ 0.00 0 items Home / Clothing / Men / Tops / Hoodies & Sweatshirts|Clothing / Collections / Eco Friendly|Clothing / Chaz Kangeroo Hoodie Chaz Kangeroo Hoodie $ 52.00 This is a variable product called a Chaz Kangeroo Hoodie Size Choose an option XS S M L XL Color Choose an option Black Gray Orange Clear Chaz Kangeroo Hoodie quantity Add to cart SKU: MH01 Category: Eco Friendly|Clothing Description Additional information Description Ideal for cold-weather training or work outdoors, the Chaz Hoodie promises superior warmth with every wear. Thick material blocks out the wind as ribbed cuffs and bottom band seal in body heat. • Two-tone gray heather hoodie. • Drawstring-adjustable hood. • Machine wash/dry. Additional information Size XS, S, M, L, XL Color Black, Gray, Orange Related products Frankie Sweatshirt $ 60.00 Select options Bruno Compete Hoodie $ 63.00 Select options Frankie Sweatshirt Bruno Compete Hoodie Search Search © Ecommerce Test Site to Learn Web Scraping 2026 Built with WooCommerce . My Account Search Search for: Search Cart 0",
  "fetched_at": "2026-10-21T08:11:24.399910+00:00"
}
```

The web data agent returns that data as a `FetchedPage`. The processing agent then reads it and returns a `MarketSnapshot` with `product_name`, `current_price`, `currency`, `availability`, `source_url`, `collected_at`, and up to five `evidence` snippets quoted from the page text. Fields the page does not support stay `null`.

### Project Checks

The repository includes a test suite that covers the request parameters, the tool schema, the schemas and the agent handoff without spending API credits. Run it from the cloned project:

```bash
pytest -q
ruff check .
```

The project produced:

```text
21 passed in 1.75s
All checks passed!
```

## Zenrows Adds Reliable Web Access to Agno

You registered Zenrows as a typed Agno tool so the web data agent can retrieve rendered page content. The workflow validates the response structure before the processing agent extracts a market snapshot, and the agent instructions restrict extraction to values supported by the fetched page.

On the Cloudflare page, the direct request received HTTP `403`, while the Agno tool retrieved the protected content through Zenrows with HTTP `200`.

You can adapt the same pattern to another public market-data source by changing the target URL and updating the `MarketSnapshot` fields to match the data you want to collect. To snapshot many product pages on a schedule, submit the URLs as one [Batch](/blog/introducing-batch) job and hand each result to the processing agent. For the same idea in other frameworks, see the [Mastra web-aware agent](/blog/web-aware-typescript-agent-mastra-zenrows), the [Google ADK research agent](/blog/web-research-agent-google-adk-zenrows), and the [AG2 multi-agent research team](/blog/web-research-multi-agent-ag2-zenrows).

[Start building free](https://app.zenrows.com/register) with 5,000 credits every month.

## FAQ

### How Do You Add a Custom Tool to an Agno Agent?

Pass a typed Python function in the agent's `tools` list. Agno reads the function's name, docstring and type hints to build the tool schema the model sees. In this tutorial, the function calls Zenrows Fetch with `mode=auto`, so the Agno agent gets reliable web access while Fetch chooses the access configuration and returns the requested page content. For how web data APIs compare for agents, see [the best web scraping API for AI agents](/blog/best-web-scraping-api-for-ai-agents).

### How Do You Keep an API Key Out of an Agno Tool Schema?

Define the tool as a nested function that captures the key from your settings, and give it only the arguments the model should fill. Here, `fetch_with_zenrows_tool()` takes `url: str` alone, so the schema Agno generates contains `url` and nothing else, and the Zenrows API key stays inside the application. The wrapper also replaces request errors with a generic message, so the key never reaches a tool result or a log.

### Can Agno Agents Use the Zenrows MCP Server?

Yes. Agno's `MCPTools` connects to the remote Zenrows MCP server at `https://mcp.zenrows.com/mcp` with your API key as a Bearer token, and the agent gets the Zenrows tools for reading pages, extracting structured data, running browser sessions and queuing URL lists. You then describe the task in plain English, for example "Get the price and availability from this product page", and the agent picks the tool. See the [Zenrows MCP documentation](https://docs.zenrows.com/mcp/overview) for setup.

### Why Does the Workflow Use Two Agents?

The web data agent requests the page through Zenrows, while the processing agent extracts the product fields from the returned content. Validating the first agent's response as a `FetchedPage` stops invalid retrieval output before the second agent receives it.

### Can the Workflow Extract Multiple Products From One Page?

The current `MarketSnapshot` model represents one product. To return every product found on a page, define a response model containing a list of product records, revise the processing agent's instructions, and update the code that reads the final result.

### How Do You Track Prices Across Many Product Pages?

Submit the product URLs as one Zenrows [Batch](/products/batch) job instead of calling the tool once per page. Batch queues every URL, applies `mode=auto` to each one, and returns the results for collection, so the processing agent can turn each page into a `MarketSnapshot` on whatever schedule you run the job.