Build an Autonomous Research Agent with Claude: Web Search, PDF Downloads, and Conversational Memory

Clearing Up a Common Misconception

Before writing a single line of code, let's address the most common mistake in "Claude Agent" tutorials.

# This code will throw AttributeError
agent = Anthropic().agents.create(
  name="Research Agent",
  memory=True
)
session = agent.create_session()
response = session.query("How...")

# This is what actually works
client = Anthropic()

response = client.messages.create(
  model="claude-sonnet-4-6",
  max_tokens=4096,
  system="You are a research assistant",
  tools=[web_search_tool],
  messages=conversation_history
)

The key insight: There is no magic "agent" object. Autonomy comes from a loop you write yourself calling messages.create(), reading the response, executing tools, feeding results back, and repeating until Claude stops requesting tools.

Architecture: The Four Components

graph TD
  User([User Query]) --> AgentLoop[Agentic Loop]
  
  subgraph AgentSystem[The Research Agent]
    SystemPrompt[System Prompt: 'You are an expert...']
    History[(Conversation History)]
    Tools{Tools}
    
    subgraph ToolSet[Capabilities]
      T1[web_search]
      T2[download_pdf]
      T3[read_file]
    end
  end
  
  AgentLoop --> |1. messages.create| Claude[Claude API]
  Claude --> |2. stop_reason: tool_use| AgentLoop
  AgentLoop --> |3. Execute Tool| Tools
  Tools --> |4. Tool Result| AgentLoop
  AgentLoop --> |5. Repeat| Claude
  Claude --> |6. stop_reason: end_turn| FinalResponse([Final Research Summary])

Step 1: Basic Setup and Stateless Queries

Installation

pip install anthropic tavily-python requests streamlit python-dotenv

Create a .env file (add to .gitignore):

ANTHROPIC_API_KEY=sk-ant-...
TAVILY_API_KEY=tvly-...

Your First Working Claude Query

# step1_basic.py
from anthropic import Anthropic
from dotenv import load_dotenv
import os

load_dotenv()
client = Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY"))

def query_claude(prompt: str, system: str = "You are a helpful research assistant.") -> str:
    """Single stateless query — no memory between calls."""
    response = client.messages.create(
        model="claude-sonnet-4-6",
        max_tokens=2048,
        system=system,
        messages=[{"role": "user", "content": prompt}]
    )
    return response.content[0].text

if __name__ == "__main__":
    result = query_claude("How does the human body digest protein? Explain the key enzymes involved.")
    print(result)

This works, but has a critical limitation: every call is independent. Ask a follow-up and Claude has no idea what you were discussing. That's where memory comes in.

Step 2: Conversational Memory

Memory in Claude is not a setting—it's a list you maintain. Every turn, you append both the user message and the assistant's response to conversation_history, then pass the entire list on the next API call.

# step2_memory.py
from anthropic import Anthropic
from dotenv import load_dotenv
import os

load_dotenv()
client = Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY"))

SYSTEM_PROMPT = """You are an expert research assistant specializing in biology and biochemistry.
When answering questions, be thorough and scientific. Reference previous context in the conversation."""

class ResearchAgent:
    """Research agent with multi-turn conversational memory."""
    def __init__(self, system: str = SYSTEM_PROMPT):
        self.system = system
        self.conversation_history: list[dict] = []

    def chat(self, user_message: str) -> str:
        """Send a message and maintain conversation history."""
        # Add user message to history
        self.conversation_history.append({
            "role": "user",
            "content": user_message
        })

        # Call API with full history
        response = client.messages.create(
            model="claude-sonnet-4-6",
            max_tokens=2048,
            system=self.system,
            messages=self.conversation_history  # Full history every time
        )

        assistant_text = response.content[0].text

        # Add assistant response to history
        self.conversation_history.append({
            "role": "assistant",
            "content": assistant_text
        })
        return assistant_text

if __name__ == "__main__":
    agent = ResearchAgent()
    # Turn 1
    r1 = agent.chat("How does the body digest protein?")
    print(f"Turn 1:\n{r1}\n{'='*60}\n")
    # Turn 2 — Claude remembers context
    r2 = agent.chat("What specific enzymes break down the peptide bonds you mentioned?")
    print(f"Turn 2:\n{r2}")

Step 3: Tool Use — Web Search and PDF Download

Now we give the agent real capabilities. Claude's tool use works via function calling: you define tool schemas in JSON Schema format, Claude decides when to call them, and you execute the actual code.

Define the Tools

# tools.py
from tavily import TavilyClient
import requests
import os
import json
from dotenv import load_dotenv

load_dotenv()
tavily = TavilyClient(api_key=os.getenv("TAVILY_API_KEY"))

# --- TOOL SCHEMAS ---
WEB_SEARCH_TOOL = {
    "name": "web_search",
    "description": "Search the web for research papers and scientific articles. Prefer academic sources.",
    "input_schema": {
        "type": "object",
        "properties": {
            "query": {"type": "string", "description": "Search query"},
            "max_results": {"type": "integer", "description": "Number of results (1-10, default 5)"}
        },
        "required": ["query"]
    }
}

DOWNLOAD_PDF_TOOL = {
    "name": "download_pdf",
    "description": "Download a PDF from a URL and save it locally.",
    "input_schema": {
        "type": "object",
        "properties": {
            "url": {"type": "string", "description": "Direct URL to the PDF"},
            "topic": {"type": "string", "description": "Research topic for folder naming"}
        },
        "required": ["url", "topic"]
    }
}

# --- TOOL IMPLEMENTATIONS ---
def execute_web_search(query: str, max_results: int = 5) -> dict:
    try:
        results = tavily.search(query=query, max_results=max_results)
        return {"success": True, "results": results.get("results", [])}
    except Exception as e:
        return {"success": False, "error": str(e)}

def execute_download_pdf(url: str, topic: str) -> dict:
    try:
        folder = os.path.join("papers", topic.replace(" ", "_")[:50])
        os.makedirs(folder, exist_ok=True)
        filename = url.split("/")[-1].split("?")[0] or "paper.pdf"
        filepath = os.path.join(folder, filename)
        
        response = requests.get(url, timeout=30)
        with open(filepath, "wb") as f:
            f.write(response.content)
        return {"success": True, "filepath": filepath}
    except Exception as e:
        return {"success": False, "error": str(e)}

TOOL_REGISTRY = {
    "web_search": execute_web_search,
    "download_pdf": execute_download_pdf
}
ALL_TOOLS = [WEB_SEARCH_TOOL, DOWNLOAD_PDF_TOOL]

The Agentic Loop

This is the core of autonomous behavior. The loop continues as long as Claude's stop_reason is "tool_use":

# step3_agent_with_tools.py
from anthropic import Anthropic
from tools import ALL_TOOLS, TOOL_REGISTRY
import os, json

client = Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY"))

RESEARCH_SYSTEM = """You are an autonomous research assistant with access to web search and PDF downloading.
1. Search for recent, authoritative papers
2. Download relevant PDFs
3. Synthesize findings into a clear, cited summary"""

class AutonomousResearchAgent:
    def __init__(self):
        self.conversation_history = []

    def research(self, topic: str) -> str:
        self.conversation_history.append({"role": "user", "content": f"Research: {topic}"})
        
        while True:
            response = client.messages.create(
                model="claude-sonnet-4-6",
                max_tokens=4096,
                system=RESEARCH_SYSTEM,
                tools=ALL_TOOLS,
                messages=self.conversation_history
            )

            if response.stop_reason == "end_turn":
                final_text = response.content[0].text
                self.conversation_history.append({"role": "assistant", "content": final_text})
                return final_text

            if response.stop_reason == "tool_use":
                self.conversation_history.append({"role": "assistant", "content": response.content})
                
                tool_results = []
                for block in response.content:
                    if block.type == "tool_use":
                        result = TOOL_REGISTRY[block.name](**block.input)
                        tool_results.append({
                            "type": "tool_result",
                            "tool_use_id": block.id,
                            "content": json.dumps(result)
                        })
                
                self.conversation_history.append({"role": "user", "content": tool_results})

Step 4: Streamlit Web Interface

# app.py
import streamlit as st
from step3_agent_with_tools import AutonomousResearchAgent

st.title("🧪 Autonomous Research Agent")
if "agent" not in st.session_state:
    st.session_state.agent = AutonomousResearchAgent()

if topic := st.chat_input("Enter research topic..."):
    with st.chat_message("user"): st.write(topic)
    with st.chat_message("assistant"):
        with st.spinner("Researching..."):
            result = st.session_state.agent.research(topic)
            st.markdown(result)

Security and Best Practices

Key Takeaways