WaterCrawl · emploidai Marketplace
emploidai Marketplace
Add-onsAppletsPlugins
Search tools, teams, and capabilitiesPublish
MarketplacePluginsWaterCrawl
Plugin
Limited listing

WaterCrawl

by watercrawl · v0.5.0

WaterCrawl is a smart web crawling tool. with this plugin, you can easily crawl websites and extract data from them. this plugin is compatible with WaterCrawl v0.9.*

7.3k installsUpdated Aug 19, 2025
Publisher information is incomplete

This community listing does not yet include every recommended support, privacy, pricing, and permission disclosure. Review the available package permissions before installing.

Capabilities

Tools

Available inside your emploidai workspace after installation.

Data sources

Available inside your emploidai workspace after installation.

Category

tool

Version

0.5.0watercrawl

Requirements

Maximum memory 256MB

Pricing

Not disclosed by publisher

Security & access

Review before installing

CompatibleRequires emploidai 1.0.0+

Permissions

  • Uses endpoint capability
  • Uses storage capability
  • Uses tool capability
  • Requires encrypted tool credentials

Dependencies

No additional dependencies

Resources

Privacy policy
emploidai Marketplace

Discover capabilities. Review access. Install inside your workspace.

DocumentationSecuritySupportPrivacyTerms

WaterCrawl Plugin

Author: watercrawl
Version: 0.5.0 Type: Tool

Description

WaterCrawl is a powerful web crawling tool designed for developers. This plugin allows you to easily crawl websites, extract data, and search the web. It is compatible with WaterCrawl v0.9.* and offers a range of features for efficient data extraction and discovery.

Features

Scrape Tool:

Scrapes a single URL and outputs the content in markdown format with options for including HTML, links, and screenshots.

Crawl Tool:

Initiates a web crawl to extract data from specified URLs with configurable options for including/excluding URL patterns and generating alt text for images.

Crawl Job Tool:

Manage crawl jobs using a crawl request UUID. You can retrieve the job status, get all results, fetch a single result by its UUID, or cancel an ongoing task.

Search Tool:

Search the web for information using WaterCrawl's search API with configurable options for language, country, time range, and search depth.

Search Job Tool:

Retrieve search results or manage search jobs based on a search request UUID. You can get the status and results of a search, or cancel it.

Sitemap Tool:

Generates a sitemap directly from a URL. You can configure it to include subdomains, ignore existing sitemap.xml files, and more.

Installation

To install the WaterCrawl plugin, follow these steps: Download the latest .difypkg file from the GitHub Releases or from the Dify marketplace.

Authentication

Login to your WaterCrawl account here. Or your self-hosted WaterCrawl instance. In the dashboard go to API Keys and create a new key or use an existing key.

then in the Dify plugin management page, go to Plugins and click on the + Install button. and use API Key in the plugin configuration.

Contributing / Development

To contribute to the WaterCrawl plugin, follow these steps:

  1. Clone the repository: git clone https://github.com/watercrawl/watercrawl-dify-plugin.git
  2. Navigate to the project directory: cd watercrawl-dify-plugin
  3. Make virtual environment: python -m venv env
  4. Activate the virtual environment: source env/bin/activate
  5. Install dependencies: pip install -r requirements.txt
  6. Copy the .env.example file to .env and fill in the necessary values.
  7. Run the plugin: python -m main

Support

For support, please contact us at support@watercrawl.dev.

License

This project is licensed under the MIT License - see the LICENSE file for details.