CRW · emploidai Marketplace
emploidai Marketplace
Add-onsAppletsPlugins
Search tools, teams, and capabilitiesPublish
MarketplacePluginsCRW
Plugin
Limited listing

CRW

by us · v0.0.1

CRW — the open-source web scraper built for AI agents. Scrape, crawl, and map websites. Firecrawl-compatible API, 5.5x faster, 75x less memory. Self-hosted or fastcrw.com cloud.

915 installsUpdated Mar 30, 2026
Publisher information is incomplete

This community listing does not yet include every recommended support, privacy, pricing, and permission disclosure. Review the available package permissions before installing.

Capabilities

Tools

Available inside your emploidai workspace after installation.

Data sources

Available inside your emploidai workspace after installation.

Category

tool

Version

0.0.1us

Requirements

Maximum memory 1MB

Pricing

Not disclosed by publisher

Security & access

Review before installing

CompatibleRequires emploidai 1.0.0+

Permissions

  • Uses tool capability
  • Requires encrypted tool credentials

Dependencies

No additional dependencies

Resources

Privacy policy
emploidai Marketplace

Discover capabilities. Review access. Install inside your workspace.

DocumentationSecuritySupportPrivacyTerms

CRW — Dify Tool Plugin

Web scraping, crawling, and URL mapping for Dify workflows and agents.

CRW is an open-source web scraper built for AI agents. Firecrawl-compatible API, 5.5x faster, 75x less memory. Works with fastcrw.com cloud or any self-hosted CRW instance.

Tools

ToolEndpointDescription
ScrapePOST /v1/scrapeScrape a single URL and return clean markdown, HTML, plain text, or structured JSON
CrawlPOST /v1/crawlStart an async BFS crawl with depth and page limits
Crawl StatusGET /v1/crawl/{id}Check crawl job status or cancel a running job
MapPOST /v1/mapDiscover all URLs on a website via link extraction and sitemap parsing

Setup

1. Install the Plugin

Install from the Dify Marketplace, or load locally during development:

# Package the plugin
dify plugin package ./crw

# Install the .difypkg file via Dify Settings > Plugins

2. Configure Credentials — Pick One

Option A: Cloud (fastcrw.com) — Quickest Start

Sign up at fastcrw.com and get 500 free credits:

  • API Key: crw_live_... from fastcrw.com
  • Base URL: (leave empty — defaults to fastcrw.com)

Option B: Self-hosted with binary (free, no limits)

curl -fsSL https://raw.githubusercontent.com/us/crw/main/install.sh | bash
crw  # starts on http://localhost:3000
  • API Key: any-value (or leave empty if no auth)
  • Base URL: http://localhost:3000

Option C: Self-hosted with Docker

docker run -d -p 3000:3000 ghcr.io/us/crw:latest

Same as Option B.

Tool Details

Scrape

Extract clean content from a single URL. Supports:

  • Multiple output formats: markdown, HTML, rawHtml, plainText, links, JSON
  • JavaScript rendering with configurable wait time
  • CSS selectors and XPath for targeted extraction
  • Include/exclude tag filters
  • Custom HTTP headers
  • Stealth mode (browser-like headers, UA rotation)
  • Per-request proxy
  • LLM-based structured extraction via JSON Schema

Crawl

Start an async breadth-first crawl from a URL:

  • Configurable max depth and max pages
  • Sync mode (wait for completion) or async mode (return job ID)
  • Poll for results using the Crawl Status tool

Crawl Status

Check on or cancel a running crawl job:

  • Returns current status, total pages, completed pages
  • Cancel action stops the crawl immediately

Map

Discover all URLs on a website:

  • Combines link extraction with sitemap.xml parsing
  • Configurable crawl depth
  • Returns a complete list of discovered URLs

Links

  • CRW GitHub
  • fastcrw.com
  • Dify Plugin Development Guide