Web search (SearXNG) and fetch tools with MCP server

Navigation

Verified details

These details have been verified by PyPI
Project links
GitHub Statistics
Maintainers
Avatar for pvliesdonk from gravatar.com pvliesdonk

Unverified details

These details have not been verified by PyPI
Meta
  • License: MIT License (MIT)
  • Author: pvliesdonk
  • Tags markitdown , mcp , metasearch , searxng , trafilatura , webfetch , websearch
  • Requires: Python >=3.10
  • Provides-Extra: all , dev , docs , extraction , markdown , mcp
Classifiers
Report project as malware

Project description

pvl-webtools

Web search (via SearXNG) and fetch tools with MCP server.

Documentation

Installation

pip install pvl-webtools

# With MCP server support
pip install pvl-webtools[mcp]

# With markitdown for LLM-friendly markdown output (recommended)
pip install pvl-webtools[markdown]

# With trafilatura for article text extraction
pip install pvl-webtools[extraction]

# Everything
pip install pvl-webtools[all]

Usage

Direct Usage

import asyncio
from pvlwebtools import web_search, web_fetch

async def main():
    # Search (requires SEARXNG_URL env var)
    results = await web_search("python async best practices", max_results=5)
    for r in results:
        print(f"{r.title}: {r.url}")

    # Fetch and extract article
    page = await web_fetch("https://example.com/article")
    print(page.content)

asyncio.run(main())

With SearXNG Client

from pvlwebtools import SearXNGClient

client = SearXNGClient(url="http://localhost:8888")
if client.check_health():
    results = await client.search("query", domain_filter="wikipedia.org")

As MCP Server

# Set SearXNG URL
export SEARXNG_URL="http://localhost:8888"

# Optional: enable verbose logging (sent to stderr)
export VERBOSE=1

# Run server
uvx pvl-webtools-mcp

Docker (Streamable HTTP)

# Pull and run
docker run -d -p 8000:8000 \
  -e SEARXNG_URL="http://your-searxng:8888" \
  ghcr.io/pvliesdonk/pvl-webtools:latest

# MCP endpoint available at http://localhost:8000/mcp

See Docker Deployment for full documentation.

Configuration

Environment Variable Description
SEARXNG_URL SearXNG instance URL (required for web_search)
LOG_LEVEL Optional logging level for the MCP server (DEBUG, INFO, TRACE, etc.). TRACE removes dependency filtering so every log (including FastMCP internals) is emitted.
VERBOSE Convenience flag; set to 1/true for debug logging without exposing dependency noise.

Tools

web_search

Search the web via SearXNG metasearch engine.

  • query: Search query
  • max_results: 1-20 (default 5)
  • domain_filter: Limit to domain (e.g., "wikipedia.org")
  • recency: "all_time", "day", "week", "month", "year"

web_fetch

Fetch and extract content from URLs.

  • url: URL to fetch
  • extract_mode:
    • "markdown" (default): LLM-friendly markdown via markitdown
    • "article": Plain text via trafilatura
    • "raw": Raw HTML (truncated)
    • "metadata": Title, description, Open Graph tags

License

MIT

Project details

Verified details

These details have been verified by PyPI
Project links
GitHub Statistics
Maintainers
Avatar for pvliesdonk from gravatar.com pvliesdonk

Unverified details

These details have not been verified by PyPI
Meta
  • License: MIT License (MIT)
  • Author: pvliesdonk
  • Tags markitdown , mcp , metasearch , searxng , trafilatura , webfetch , websearch
  • Requires: Python >=3.10
  • Provides-Extra: all , dev , docs , extraction , markdown , mcp
Classifiers

Release history Release notifications | RSS feed

This version

1.0.1

1.0.0

0.1.3

0.1.1

0.1.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pvl_webtools-1.0.1.tar.gz (23.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

pvl_webtools-1.0.1-py3-none-any.whl (15.9 kB view details)

Uploaded Python 3

File details

Details for the file pvl_webtools-1.0.1.tar.gz.

File metadata

  • Download URL: pvl_webtools-1.0.1.tar.gz
  • Upload date:
  • Size: 23.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for pvl_webtools-1.0.1.tar.gz
Algorithm Hash digest
SHA256 2f084fec8c95fb8d4484d5a00153185ed22ec5cc8922f0f1a43fa2f3539b8103
MD5 0d7cebd77b021651c89d2e1e6fdd91c8
BLAKE2b-256 e1ae8c4d946cb4e00bbf9cb33c922e79040c70bf4750102fa17f5eca544273e6

See more details on using hashes here.

Provenance

The following attestation bundles were made for pvl_webtools-1.0.1.tar.gz:

Publisher: release.yml on pvliesdonk/pvl-webtools

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file pvl_webtools-1.0.1-py3-none-any.whl.

File metadata

  • Download URL: pvl_webtools-1.0.1-py3-none-any.whl
  • Upload date:
  • Size: 15.9 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for pvl_webtools-1.0.1-py3-none-any.whl
Algorithm Hash digest
SHA256 a7e12ef7628787d1edd8e4028955762f3c44e903dda2c4b90139c607662f4294
MD5 768ae3aa9be2df729fec3763abfc6446
BLAKE2b-256 5d5e16b03eeddc24d2fc1e455a0353715f6369127e9b3c7d18070a58b8083966

See more details on using hashes here.

Provenance

The following attestation bundles were made for pvl_webtools-1.0.1-py3-none-any.whl:

Publisher: release.yml on pvliesdonk/pvl-webtools

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page