CrewAI crews are only as good as the information their agents can reach. Out of the box they reason over the model’s training data; to research, fact-check, or monitor anything live they need web tools. Here’s how to give a crew both search and scraping through a single Auxiliar key — so you never wire up (or pay for) providers one at a time.
One key, two tools
Auxiliar is a gateway: one bearer token reaches every search and scraping provider at https://api.auxiliar.ai/<provider>/..., with the upstream keys injected server-side. We’ll build two CrewAI tools on top of it — one to search, one to fetch a page as clean markdown.
import os, requests
from crewai.tools import tool
AUX = "https://api.auxiliar.ai"
H = {"Authorization": f"Bearer {os.environ['AUXILIAR_API_KEY']}"}
@tool("Web Search")
def web_search(query: str) -> str:
"""Search the live web for recent or factual information."""
r = requests.post(f"{AUX}/serper/search", headers=H, json={"q": query}, timeout=30)
r.raise_for_status()
return "\n".join(f"{o['title']} — {o['link']}" for o in r.json().get("organic", [])[:6])
@tool("Read Page")
def read_page(url: str) -> str:
"""Fetch a web page and return clean, LLM-ready markdown."""
r = requests.post(f"{AUX}/firecrawl/v1/scrape", headers=H,
json={"url": url, "formats": ["markdown"]}, timeout=60)
r.raise_for_status()
return r.json().get("data", {}).get("markdown", "")[:8000]
The search tool uses Serper; the reader uses Firecrawl, which topped our scraping benchmark for markdown quality. Both are the same key.
Give the tools to an agent
from crewai import Agent, Task, Crew
researcher = Agent(
role="Research Analyst",
goal="Find and summarize accurate, up-to-date information",
backstory="A meticulous analyst who always grounds claims in live sources.",
tools=[web_search, read_page],
)
task = Task(
description="Research the current state of open-source autonomous agents and cite sources.",
expected_output="A short brief with 3-5 findings, each with a source URL.",
agent=researcher,
)
print(Crew(agents=[researcher], tasks=[task]).kickoff())
The agent will search, pick promising results, read the pages, and synthesize — all through one Auxiliar key and one bill.
Swapping providers
Every provider keeps its native API; Auxiliar just forwards the request. To search with an agent-native index instead, change /serper/search to /tavily/search ({"query": ...}). To scrape hard, anti-bot-protected targets, point the reader at a stealth scraper — see how to scrape without getting blocked and the best anti-bot scraping API ranking. Because it’s all one key, you can even fall back from one provider to another when a fetch fails.