CrewAI Live Web Research Workflow

Empowering an autonomous AI Agent with real-time web search capabilities using Google Gemini and SerperDevTool.


crewai_web_research.py
import os
from crewai import Agent, Task, Crew, Process, LLM
from crewai_tools import SerperDevTool

# 1. Setup Environment Variables
# Gemini provides the reasoning LLM engine; Serper provides live Google Search access
os.environ["GEMINI_API_KEY"] = "YOUR_GEMINI_API_KEY"
os.environ["SERPER_API_KEY"] = "YOUR_SERPER_API_KEY"

# 2. Define the Brain (LLM)
gemini_brain = LLM(
    model="gemini/gemini-1.5-flash",
    temperature=0.3  # Lower temperature ensures factual adherence during web research
)

# 3. Initialize the Web Search Tool
search_tool = SerperDevTool()

# 4. Define the Research Agent with Tools attached
research_agent = Agent(
    role="Senior Market Research Analyst",
    goal="Uncover cutting-edge industry trends, technical advancements, and live market data.",
    backstory=(
        "You are a relentless tech researcher. You don't rely on old knowledge; "
        "you use search engines to cross-reference multiple modern sources, weed out "
        "marketing fluff, and isolate hard, technical facts."
    ),
    verbose=True,
    allow_delegation=False,
    llm=gemini_brain,
    tools=[search_tool]  # <-- Attaching web search capability directly to the Agent
)

# 5. Define the Research Task
topic = "Mahindra electric vehicle platforms like INGLO and competition comparison"

research_task = Task(
    description=(
        f"Conduct a thorough online search regarding: '{topic}'. "
        "Find the key technical specs, target launch timeline, and architecture benefits. "
        "Filter out generic press release fluff and synthesize the core technical details."
    ),
    expected_output="A structured markdown report with key technical bullet points and clear headings.",
    agent=research_agent
)

# 6. Assemble the Crew and Kick off
crew = Crew(
    agents=[research_agent],
    tasks=[research_task],
    process=Process.sequential
)

print(f"--- Starting Live Web Research on '{topic}' ---")
result = crew.kickoff()

print("\n--- Final Result ---")
print(result)

How Tool-Augmented Agents Work

Unlike basic static prompt workflows, this script grants your agent **real-time internet access**. The agent autonomously forms Google search queries, parses the returned web pages, filters out marketing buzzwords, and structures a concise technical report.

1. Dual API Configuration
APIs & Setup

This pipeline uses two distinct services:

  • GEMINI_API_KEY: Powers the AI reasoning engine (Gemini).
  • SERPER_API_KEY: Connects to Serper.dev, enabling the agent to query real-time Google search results programmatically.
2. Low Temperature LLM
Factual Brain

Setting temperature=0.3 lowers randomness in the model's outputs. For research and data retrieval, lower temperatures ensure the agent sticks strictly to facts retrieved from web pages rather than hallucinating details.

3. Tool Binding
Tool Augmentation

By passing tools=[SerperDevTool()] into the Agent definition, CrewAI gives the agent an iterative execution loop (ReAct loop). The agent evaluates its assigned task, decides when to trigger a Google search, reads the raw web snippets, and runs additional queries if information is missing.

4. Task Instructions & Execution
Execution

The Task specifies the research subject and explicit rules (e.g., "filter out marketing fluff", "output structured markdown"). Calling crew.kickoff() manages the search loop automatically until the structured report is fully compiled.

Note on EV Platform Terminology in the Query

In your research topic variable, note that INGLO is Mahindra's flagship born-electric skateboard architecture (powering models like the BE 6 and XEV 9e), while acti.ev and EMA are platforms associated with Tata Motors and JLR respectively. The tool-augmented agent will automatically cross-reference these terms during its search execution and clarify these architectural relationships in the final report!