ScrapeUnblocker - Web Scraper & SERP API for Dify
Web scraping and Google SERP API as a Dify tool. Scrape any website, bypass
anti-bot protections, and pull Google search results directly into your Dify
apps, agents and workflows.
ScrapeUnblocker renders every page in a real browser behind anti-bot systems
such as Cloudflare, DataDome, PerimeterX and Akamai, using residential
proxies, and returns the result as rendered HTML or AI-parsed structured
data. Use it whenever a plain HTTP request returns a block page, a captcha, or
an empty shell instead of the real content.
Why use it
- Scrape any website - e-commerce listings, product pages, travel sites,
marketplaces, classifieds, news, real estate, and other pages that only render
for a real browser.
- Bypass anti-bot protections - Cloudflare, DataDome, PerimeterX, Akamai and
similar systems, handled automatically. No flags to set, rendering always runs.
- Google search (SERP) API - scrape Google search engine results and get
clean organic results (title, url, description, position).
- Raw HTML or parsed data - get the full rendered HTML, or AI-parsed
structured JSON when you just want the fields.
- Geo-targeting - route requests through a specific country's residential IP
for geo-restricted or localised content.
Tools
Web Page Scraper (get_page_source)
Scrapes a single web page and returns its fully rendered content.
| Parameter | Required | Description |
|---|
url | yes | Full URL of the page to scrape, including the scheme |
parsed_data | no | Return AI-parsed structured JSON instead of raw HTML |
proxy_country | no | Two-letter country code for the exit IP, for geo-restricted content |
Returns the page content as a text message, plus a JSON message with success,
url, parsed_data and content_length.
Google Search / SERP (search_google)
Scrapes a Google search engine results page (SERP) and returns the organic
results.
| Parameter | Required | Description |
|---|
keyword | yes | The search term to look up on Google |
pages_to_check | no | How many result pages to scrape (default 1) |
proxy_country | no | Two-letter country code for country-specific results |
Each organic result carries title, url, description and position.
Quick start
- Get an API key at scrapeunblocker.com.
- In Dify, go to Plugins -> Marketplace, search for ScrapeUnblocker,
and install it.
- Open the plugin settings and paste your key into ScrapeUnblocker API Key.
- Add the Web Page Scraper or Google Search (SERP) tool to any workflow,
agent or chatflow and connect its output to an LLM node.
Two optional settings are available: API Base URL (override to point at a
different environment) and Timeout Seconds (default 180, since rendering a
protected page in a browser takes longer than a plain HTTP request).
Example uses
- Feed a competitor's product page into an LLM node to extract price and specs.
- Scrape a Google SERP for a keyword, then summarise the top-ranking results.
- Pull a marketplace or classifieds listing that normally shows a block page.
- Fetch geo-restricted content through a specific country's IP.
Notes
- Requests are made by ScrapeUnblocker's infrastructure, so the target site sees
a ScrapeUnblocker IP rather than your Dify host. See PRIVACY.md
for exactly what data leaves your instance.
- Rendering a protected page typically takes several seconds. Keep this in mind
when placing the tool in latency-sensitive workflows.
Links