Skip to content

Random IPs for Web Scraping: VPN, Proxy Pool, Tor, or API

Scrapers turn pages into structured data, and that data feeds dashboards, market analysis, and forecasting models. The same automation that makes collection efficient also leaves a trail: many requests arriving from one address. Sites respond with address-level defenses. Some refuse traffic from IPs that look bot-like; others cap how many requests a single address may send in a given period. Because both defenses key on the address rather than the payload, swapping IPs is often the first fix people try. Below are four ways to do it with Node.js and Puppeteer, plus guidance on which one suits your workload.

Why IP rotation matters, and what it does not fix

A single IP hammering a site is easy to flag. Rotating addresses spreads requests across many sources, which reduces the chance that any one address trips a block or a rate limit. It also helps when you need to see content that is served differently by region.

It is worth being precise about what rotation solves. Changing your IP protects against IP bans. It does not, by itself, defeat CAPTCHAs, user-agent checks, browser fingerprinting, honeypot traps, or login walls. If your scraper is failing for one of those reasons, a new IP will not help. Diagnose the failure first: a clean 403 with a block page usually points to IP reputation; a challenge page points to bot detection; a login redirect points to session requirements.

Step 1: Confirm your current public IP with Puppeteer

You need Node.js (which includes npm) and Puppeteer installed. Initialize a project and add Puppeteer:

  • npm init esnext
  • npm install puppeteer

httpbin.org echoes the caller's IP at its /ip endpoint, which makes it a clean test target. Create scraper.js with the baseline script below, run it with node scraper.js, and you should see a JSON body whose origin key holds your public address.

Running it more than once may show a slightly different address, because many ISPs assign public IPs dynamically or rotate them for security. That natural variation is not the same as deliberate rotation, so treat this run as your starting reference point.

Method A: VPN (simple, but manual)

A VPN hides your address and encrypts traffic in transit. Proton VPN is one example known for unlimited bandwidth and a privacy focus. Install the free open-source client for your OS, sign in (creating a free account if needed), and press Quick Connect. You are then assigned a new exit address, and re-running the baseline script should print that address as origin.

The catch is control. To get a different address you have to reconnect manually through the app's random profile. Proton VPN, like many VPN services, is a standalone product without a public API for programmatic control, so you cannot script the switch. That makes a VPN fine for occasional or small jobs and awkward for large automated runs.

Method B: Rotating proxy pool (scriptable, variable quality)

A proxy service sits between you and the target, forwarding requests while masking your address. Rotating proxies go further: they cycle through a series of addresses, either on a timer or per request, which spreads load and helps dodge IP bans and rate limits.

Proxies come in two tiers. The free kind costs nothing, but expect spotty uptime, sluggish responses, and signatures that bot defenses recognize quickly. Paid proxies generally hold up better and add capabilities the free tier lacks, such as anti-bot handling and stronger security. Free pools also do not rotate often, so if you use them you usually have to rotate across several yourself.

A free proxy entry is written as PROTOCOL://IP_ADDRESS:PORT, where the protocol is typically HTTP, HTTPS, SOCKS4, or SOCKS5. When picking from a free list, favor entries with decent uptime (around 90 percent or higher) and lower latency. The pool below is illustrative only — replace the addresses with ones you have just checked, because free proxies expire quickly.

A helper picks a random entry from the array, and Puppeteer receives it through the --proxy-server launch argument. Running the script several times should show a different origin each run. The trade-offs remain: free proxies are not guaranteed to be up, and maintaining the pool is real work — which is why many developers move on to Tor.

Method C: Tor (programmatic rotation, with caveats)

Tor, short for The Onion Router, is a network of relays operated by volunteers. A request is wrapped in layers of encryption and passed from relay to relay; each hop peels off one layer and learns only the next stop. The final relay, the exit node, sends the request onward, so the destination sees the exit node's address rather than yours. Exit nodes rotate often, which is what makes your apparent IP shift.

To use Tor as a proxy you need a Tor client, ideally the official one, running locally or remotely. Installation is usually a single command: brew install tor on macOS via Homebrew, apt install tor on Debian or Ubuntu, or on Windows download the Tor expert bundle (not the browser), unzip it, and run tor.exe from the terminal.

Unlike most VPNs, Tor can be controlled programmatically, which lets you refresh the session and get a new exit IP without restarting the client. That requires enabling Tor's control port and authenticating with a hashed password. Generate the hash with tor --hash-password PASSWORD, then put the result in a torrc file alongside the scraper.

Key settings to map:

  • ControlPort 9051 — the local port Tor listens on for control commands.
  • HashedControlPassword — the hash you generated, used to authenticate control connections.
  • SOCKS proxy at socks5://127.0.0.1:9050 — the default data path once Tor is running.

Start Tor with tor -f ./torrc; the log should reach Bootstrapped 100% (done). With the tor-request library installed (npm install tor-request), a small renewTorSession() function sets the control port and password, calls newTorSession, and resolves on success or rejects on error. Point Puppeteer at the SOCKS5 proxy, and each run should print a different origin.

Tor has real drawbacks. Encryption covers only the links between relays, so an exit node you do not control can read or alter anything sent in the clear. The anonymity it offers has limits, and address leaks are possible. Busy periods on the network slow connections down, and plenty of sites refuse traffic from known exit nodes. And as noted above, a new IP only addresses IP bans — not the other anti-bot signals.

Method D: Managed scraping service (rotation plus the rest)

A dedicated scraping service bundles the infrastructure: IP rotation, headless browser control, proxy management, and anti-bot handling. That is useful when you want to focus on extracting data rather than maintaining a proxy pool or a Tor client.

With {{OWNER_BRAND_NAME}}, you create an account at {{OWNER_SIGNUP_URL}}, receive {{OWNER_FREE_CREDIT_TERMS}}, and find your API key at {{OWNER_API_KEY_LOCATION}}. The Node.js client is installed with npm install {{OWNER_SDK_PACKAGE}}, and you pass your key to the client constructor. Each request to the same target then comes back from a different address, as the sample responses show. Capabilities such as rotating proxies and headless browser control are documented at {{OWNER_DOCS_URL}}.

Beyond the SDK, the same service can be reached through its API directly or through a proxy frontend at {{OWNER_PROXY_FRONTEND_URL}}, which simplifies wiring it into third-party tools. The full set of code from this walkthrough is collected at {{OWNER_REPO_URL}}.

Decision checklist: which method fits your job?

Use these questions to narrow the choice before you write code.

  1. Do you need to switch IPs without human input? If yes, a VPN is out — there is no public API for programmatic control. Choose a proxy pool, Tor, or a managed service.
  2. How much setup and upkeep can you absorb? A free proxy pool needs constant refreshing and uptime checks. Tor needs a client, a control port, and a hashed password. A managed service trades setup for cost.
  3. How reliable must each request be? Free proxies are not guaranteed to be up. Tor can be slow and is blocked by some sites. Premium proxies and managed services are more dependable.
  4. Is anonymity a hard requirement? Tor offers strong anonymity but has known leak and exit-node risks. A VPN hides your address from the target but is not designed for scripted rotation.
  5. Is the block actually about your IP? If you are seeing CAPTCHAs, fingerprint checks, or login walls, rotation alone will not fix it — you need browser and session handling as well.
  6. What is your scale? A handful of requests can be handled manually. Large automated runs favor a pool or a service that rotates for you.

Worked example: verifying the IP actually changed

The httpbin /ip endpoint is the simplest way to confirm a switch. Run the baseline script once with no proxy and note the origin value. Then launch Puppeteer with --proxy-server set to one proxy from your pool, or to socks5://127.0.0.1:9050 for Tor, and run it again. If the new origin differs from the baseline, rotation is working. If it matches, the proxy was not applied — check the launch argument and that the proxy or Tor client is actually running. Repeating the run several times should produce a spread of addresses rather than one repeated value.

Bottom line

Swapping addresses on the fly buys you several things: a smaller footprint, fewer address-based blocks, requests spread across sources, and access to content that varies by region. A VPN is the simplest but least scriptable option; a free proxy pool is scriptable but fragile; Tor offers programmatic rotation at the cost of speed and exit-node risk. When you need rotation plus handling for CAPTCHAs, fingerprints, and rate limits together, a managed scraping service removes the most operational work.

Frequently asked questions

Does changing my IP stop all scraping blocks?

No. A new IP addresses IP bans and rate limits tied to an address. CAPTCHAs, user-agent detection, browser fingerprinting, honeypot traps, and required logins are separate signals that rotation does not resolve.

Why does Tor need a control port and hashed password?

The control port lets a local program send commands to Tor, and the hashed password authenticates those commands. Together they let you request a new session — and therefore a new exit IP — without restarting the Tor client.

Are free proxies good enough for scraping?

They can work for small or experimental jobs, but they are often unreliable, slow, and easy for sites to detect. They also rotate infrequently, so you usually have to cycle across several yourself. Premium proxies are paid and generally more dependable.

Can I script a VPN to rotate IPs?

Not with services that lack a public control API. Proton VPN, like many VPN products, is a standalone app with no public API for programmatic control, so switching servers is a manual step.

{
  "origin": "your_public_ip_address"
}
{
  "origin": "89.39.107.198"
}
{
  "origin": "201.182.251.142"
}

{
  "origin": "50.174.7.156:80"
}

{
  "origin": "50.174.7.156:80"
}
{
  "origin": "185.220.100.240"
}

{
  "origin": "79.137.198.213"
}

{
  "origin": "104.244.73.136"
}
// …
{
  "origin": "77.83.233.117"
}

{
  "origin": "38.170.8.53"
}

{
  "origin": "66.146.238.60"
}