Tools & Features

CleverBee comes equipped with a range of tools and features designed to facilitate comprehensive AI-driven research. This guide details the key capabilities.

Core Features

• LLM-Driven Research

  Utilizes powerful Large Language Models (configured in config.yaml) for reasoning, planning research steps, generating search queries, deciding which tools to use, and synthesizing information into final reports. CleverBee is optimized for Gemini models, specifically using Gemini 2.5 Pro for planning/final report, Gemini 2.5 Flash for agentic flow (next steps, etc.), and Gemini 2.0 for summarization.

• Automated Web Browsing (Playwright)

  Employs Playwright for robust, automated browsing of web pages. Can handle dynamic content, extract text from HTML, and download PDFs for processing.

• Content Cleaning & Summarization

  Raw HTML content is converted to clean Markdown. Large documents can be chunked and summarized (using cloud or local models specified in config.yaml) to fit within context windows and provide concise information to the primary LLM.

• Interactive UI (Chainlit)

  Provides a user-friendly web interface for entering research topics, viewing the agent's progress (thoughts, actions, observations), and receiving the final report.

• Token Tracking & Cost Estimation

  Monitors LLM token usage for each API call (if enabled via TRACK_TOKEN_USAGE). Estimates costs based on configured pricing in config.yaml, providing transparency into resource consumption.

• LLM Caching

  Uses SQLite-based caching (.langchain.db) to store results of previous LLM calls. This significantly speeds up repeated or similar queries and reduces API costs.

• Configurable Settings

  Extensive configuration options in config.yaml, .env, and mcp.json allow tailoring of models, tools, behavior, limits, and appearance.

• Modular LLM Clients

  Designed to easily integrate different LLM providers (currently Gemini and Claude) through a consistent interface.

Built-in Tools

These tools are directly integrated into the CleverBee framework and can be enabled/disabled in the tools: section of config.yaml.

• web_browser

  The primary tool for interacting with the web. It can perform searches (using an underlying search engine like DuckDuckGo), navigate to specific URLs, extract text content from HTML pages.

• reddit_search

  Searches Reddit for posts matching a given query. Returns a list of relevant posts including titles, scores, and permalinks. The number of results is configurable via MIN_POSTS_PER_SEARCH and MAX_POSTS_PER_SEARCH in config.yaml.

• reddit_extract_post

  Takes a Reddit post URL as input and extracts the content of the post and its top comments, providing valuable community insights.

MCP Tools (External)

CleverBee supports external tools via the Model Context Protocol (MCP) standard. These tools run as separate processes and communicate with the agent. They are defined in mcp.json and enabled in the tools: section of config.yaml.

• YouTube Transcript Tool (Example)

  A tool for fetching the transcript of a YouTube video given its URL. Unlike in previous versions, this tool is not automatically installed during setup but can be added by installing the @sinco-lab/mcp-youtube-transcript package and configuring it in mcp.json.

• PubMed Search (Example)

  An example placeholder for a tool to search the PubMed database for biomedical literature.

You can add more MCP tools by defining their server configuration in mcp.json and enabling them in config.yaml under the tools: section.

Browser Enhancements

• Rebrowser Patches

  The setup script applies patches to the Playwright browser driver to enhance its stealth capabilities, helping to avoid detection by some websites.

• Human-like Browsing

  CleverBee uses Chrome in headed mode and is designed to not trigger security gates, browsing like a human would to access content more reliably.

• CAPTCHA Handling (Optional)

  If USE_CAPTCHA_SOLVER is enabled in config.yaml, the system can attempt to use external services (like 2Captcha via the `recognizer` library, if installed and configured) to solve CAPTCHAs encountered during browsing.