Skip to content

n8n Node for Web Scraping: Setup, Actions and Workflow Patterns

The web scraping node lets an n8n workflow call scraping API endpoints as ordinary steps — scrape a page, pull structured search or product data, or send a prompt to ChatGPT — without writing a separate HTTP client. This guide covers the one-time install and credential save, the full action set, two practical workflow patterns, the proxy escalation ladder, and how to tell a credential problem apart from a blocked target site.

What the node does and when to use it

n8n is a canvas-based automation tool: you wire nodes together and can drop into JavaScript or Python code nodes for custom logic. The node wraps scraping API endpoints so each one becomes a step you configure with fields rather than code. It is a good fit when a workflow needs live web data — a product page, a search result page, a video's metadata — and you want the result to flow directly into another node such as a filter, a spreadsheet writer or a chat alert.

One structural point matters early: the node exposes actions only, not triggers. It cannot start a workflow by itself. Pair it with a trigger node such as On a Schedule or On a Webhook call; n8n surfaces both in the node picker.

Prerequisites

You need two things before the node will run:

  • An n8n account, either n8n Cloud or a self-hosted instance.
  • A scraping API account and an API key taken from your dashboard.

The node is not bundled with n8n by default, so plan for a one-time install step in addition to the credential.

Setup: install once, reuse the credential

The connection is saved at the n8n instance level, so you enter the API key once and every node of this integration in any workflow can use it afterwards.

  1. Sign in to n8n. On the What do you want to build? screen, choose Build a workflow to open the editor.
  2. On the empty canvas, click Add first step... (or the + in the top right) to open the Nodes panel, search for the scraping integration and select it.
  3. In the Node details panel, click Install node. This is the one-time install that makes the node configurable.
  4. Pick an action, for instance Scrape using HTML API. The node opens with its Resource (for example HTML API) and Operation (for example GET) already chosen, and the Credential field shows "No credentials yet."
  5. Click Set up credential next to the Credential field to open the account dialog.
  6. Paste your API key into the API Key field, leave Allowed HTTP Request Domains set to All, and click Save.
  7. Fill in the remaining parameters (for example URL for Scrape using HTML API), then click Execute step to test, or activate the workflow so it runs on its trigger.

After saving, the credential is available to every node of this integration in the instance and does not need to be re-entered.

Available actions

The node exposes 12 actions, grouped by capability. Each one wraps a scraping API endpoint and appears in n8n under the label in the Action column. The tables below are the reference for matching an n8n action to its underlying API; follow the Underlying API link on any row for the full parameter reference.

Scraping actions cover the HTML API in three forms: a GET-style scrape, a POST request, and a PUT request.

Search and AI actions cover Google SERP data and a ChatGPT prompt response.

E-commerce actions cover Amazon search and product lookups plus the Walmart equivalents.

YouTube actions cover search results and detailed video metadata.

There is also a Get usage data action. It returns your current credit balance, max concurrency and plan limits, which makes it useful as a usage check inside a workflow or as the basis for an alert when credits run low. It maps to the usage endpoint.

Two workflow patterns to start from

Price-drop alerts. Trigger the workflow On a Schedule (for example every morning). Run Get product information using AMAZON PRODUCT API — or the Walmart equivalent — with your tracked ASIN or product ID to pull the current price. Add an IF node that only continues when the price is below your target. Finish with a Slack message, email or Telegram alert.

Sync scraped data to Google Sheets. Trigger on a schedule or webhook, ideally with rows read from a Google Sheets node (for example a list of URLs). Run Scrape using HTML API against each URL. Map the response fields into a second Google Sheets node that appends or updates a row. New rows added to the source sheet get enriched on the next run, with no code and no separate cron job.

Credits, limits and the proxy escalation ladder

Every action draws credits at the same rate as the API call it wraps; n8n adds no surcharge. Get SERP data using GOOGLE SEARCH API therefore costs whatever the Google API costs, and Scrape using HTML API with premium_proxy: true costs whatever the HTML API costs with that flag. The credit costs reference lists the exact per-endpoint figures.

n8n's own billing is separate. On n8n Cloud, the meter is workflow executions: a run counts once no matter how many nodes it passes through. A self-hosted n8n instance carries no execution fee from n8n itself — your only cost there is the infrastructure hosting it.

The HTML API actions (Scrape, Send POST request, Send PUT request) accept premium_proxy, stealth_proxy, custom_google and country_code parameters for geo-targeted and anti-bot scraping. Treat them as a ladder rather than a set of switches:

  1. Start with neither proxy flag.
  2. If the site blocks you, retry with premium_proxy: true.
  3. If you are still blocked, escalate to stealth_proxy: true.

The ladder runs cheap to expensive, so only step up when the previous rung fails.

Troubleshooting: two different failures

Credential fails to save. n8n could not validate the API key against the scraping API. Copy the key fresh from the API key page in your account, make sure there is no surrounding whitespace, and paste it again.

A scrape returns empty or blocked content. The target site is refusing the default scraper. On the HTML API actions, turn on premium_proxy: true as the first move; if the block persists, move up to stealth_proxy: true. Every rung costs more credits — see credit costs.

The distinction is worth keeping straight: a credential failure happens before any request reaches the target site, while a blocked scrape means the request went out and the site refused it. Fixing the key will not unblock a site, and raising the proxy level will not fix a bad key.

Resources

  • Integrations on n8n — marketplace listing and available operations
  • Dashboard — manage your API key, view usage, upgrade your plan
  • Knowledge Base — searchable troubleshooting articles
Scraping actions
ActionUnderlying API
Scrape using HTML APIHTML API
Send POST request using HTML APIHTML API
Send PUT request using HTML APIHTML API

Reference table for matching n8n action labels to the underlying scraping API.

Search and AI actions
ActionUnderlying API
Get SERP data using GOOGLE SEARCH APIGoogle API
Get AI response from CHATGPT for your prompt using our CHATGPT APIChatGPT API

Reference table for matching n8n action labels to the underlying scraping API.

E-commerce actions
ActionUnderlying API
Get products related to search query using AMAZON SEARCH APIAmazon Search API
Get product information using AMAZON PRODUCT APIAmazon Product API
Get products related to search query using WALMART SEARCH APIWalmart Search API
Get product information using WALMART PRODUCT APIWalmart Product API

Reference table for matching n8n action labels to the underlying scraping API.

YouTube actions
ActionUnderlying API
Get search results from YOUTUBE using our YOUTUBE SEARCH APIYouTube Search API
Get detailed video metadata from YOUTUBE using our YOUTUBE METADATA APIYouTube Metadata API

Reference table for matching n8n action labels to the underlying scraping API.

Frequently asked questions

Is the web scraping node included with n8n by default?

No. The node is not bundled, so you install it once from the Node details panel before you can configure it.

Can the web scraping node start a workflow on its own?

No. The node exposes actions only and has no triggers. Pair it with a trigger node such as On a Schedule or On a Webhook call.

Do I need to re-enter the API key for each workflow?

No. The credential is saved in your n8n instance and is available to every node of this integration in any workflow.

How can I check my remaining credits from a workflow?

Use the Get usage data action. It returns your current credit balance, max concurrency and plan limits, and maps to the usage endpoint.

A scrape returns empty content. What should I try first?

Enable premium_proxy: true on the HTML API action. If the site still blocks you, escalate to stealth_proxy: true. Each step up costs more credits.