Node-fetch Proxy Setup: Agent, Dispatcher, Rotation, and TLS
node-fetch and Node's built-in fetch both leave proxy routing to you. This guide shows the working paths: an HttpsProxyAgent passed as { agent } for node-fetch, an Undici dispatcher or NODE_USE_ENV_PROXY for native fetch, plus practical handling for authentication, TLS, rotation, and sticky sessions. It also explains when a managed scraping API is the cleaner option.
Quick answer
For node-fetch, create an HttpsProxyAgent from your proxy URL and pass it as { agent } in the fetch call. That works for HTTP and HTTPS targets and supports authenticated proxies in the form http://user:pass@host:port.
For Node's native fetch, the route is different. On Node 18–23 it ignores HTTP(S)_PROXY and has no agent option; use an Undici dispatcher such as ProxyAgent or EnvHttpProxyAgent. On Node 24+ you can enable NODE_USE_ENV_PROXY=1 with HTTP(S)_PROXY / NO_PROXY. https-proxy-agent is designed for node-fetch and http(s).request, not native fetch.
What node-fetch is and what it is not
node-fetch brought the browser fetch API to Node before Node shipped its own fetch. It mirrors browser request/response shapes and streaming behavior, which is why many existing tools and tutorials still use it. Version 3 is ESM-only; CommonJS projects either stay on v2 or add setup.
Neither node-fetch nor native fetch includes built-in proxy configuration. Passing a proxy URL as a fetch option does nothing. You must wire an Agent (node-fetch) or a dispatcher (native fetch) that tunnels the request through the proxy.
Use https-proxy-agent with node-fetch
The standard node-fetch proxy setup is an HttpsProxyAgent passed through { agent }. Despite the name, it works for both HTTP and HTTPS targets and supports authenticated proxies.
A quick warning: avoid free proxies unless you accept that unknown operators can inspect your traffic. They are slow, unreliable, and often unsafe. A local proxy also does not change your outgoing IP, because the traffic still leaves from your own machine.
ESM and CommonJS examples
The ESM example (node-fetch v3) imports fetch and HttpsProxyAgent, builds the agent from your proxy URL, and passes it into fetch. The target https://httpbin.org/ip returns the IP your proxy exposes, so you can confirm the route.
The CommonJS example (node-fetch v2) uses require for both packages and wraps the await calls in an async IIFE. The agent is still passed as { agent }, which forces traffic through the proxy.
Local proxy with Docker and environment variables
For a stable test environment, you can run a local proxy container and point your proxy URL at http://localhost:8888. Remember that the outgoing IP will not change; the proxy is on your own machine.
Keeping credentials out of source code is safer. Read PROXY_URL and TARGET_URL from process.env, build the HttpsProxyAgent from PROXY_URL, and pass it into fetch. TARGET_URL can default to https://httpbin.org/ip for checking the exposed IP. Use a .env file or CI secrets and never commit proxy credentials.
Native fetch in Node
Node 18+ includes a built-in fetch powered by Undici. In Node 18–23 it behaves like browser fetch: no { agent } option and no automatic HTTP_PROXY / HTTPS_PROXY handling. On Node 24+, HTTP(S)_PROXY variables are respected when NODE_USE_ENV_PROXY=1 or --use-env-proxy is enabled.
To add proxy support to native fetch, configure an Undici dispatcher. You can set a global dispatcher with setGlobalDispatcher(new ProxyAgent(...)), or pass a dispatcher per request. This is a lower-level path than passing an agent to node-fetch.
TLS, certificates, and common errors
UNABLE_TO_VERIFY_LEAF_SIGNATURE usually means the proxy is intercepting and re-signing HTTPS traffic (corporate MITM), or the proxy or target uses a self-signed or untrusted certificate.
Do not set NODE_TLS_REJECT_UNAUTHORIZED=0. That disables TLS checks for the whole process and is not acceptable in production. Instead, load the correct CA explicitly or use NODE_EXTRA_CA_CERTS. Note that a ca option on HttpsProxyAgent affects TLS between Node and the proxy; for a plain http:// proxy with an HTTPS upstream, target certificate issues need global Node CA handling or a separate HTTPS agent.
Rotate proxies and manage sessions
Rotation spreads traffic across IPs and reduces bans. Sticky sessions keep a flow on one IP so login, cart, or pagination behaves more like a real user. A common pattern is random proxy per request, retries with backoff, and a concurrency cap (for example, 2–4 parallel requests per site).
Pattern A picks a random proxy and builds a fresh HttpsProxyAgent per request, retries with jittered exponential backoff, and limits concurrency to 3 in-flight requests. Pattern B reuses the same agent for a session key until a TTL expires, which keeps cookies and reputation separate per site or workflow. A smaller rotation snippet loops through a proxy list, builds a new agent per request, and retries with backoff plus jitter. In production you would also track success rate and latency per proxy and disable the ones that keep failing.
Proxy types and trade-offs
Datacenter proxies are cheap, fast, and easy for sites to detect. Residential proxies use real home ISP IPs, cost more and run slower, but are much harder to identify. ISP proxies are hosted in data centers but issued by real ISPs: more stable than residential, faster, and still difficult to block. Mobile (4G/5G) proxies are the hardest to block and the most expensive.
Most providers support rotation, whether automatic or per-request. Your ideal setup depends on how strict the target is and how much traffic you send. A heavy pipeline usually combines rotation, retries, concurrency caps, and per-proxy reputation tracking.
Compliance and ethical use
A proxy does not exempt you from site terms or data rules. Respect robots.txt where possible, do not scrape personal or authenticated data without explicit permission, log failures so you can slow down before rate limits hit, and keep concurrency reasonable so small sites are not overwhelmed. Higher-level tools that enforce safer scraping patterns can help.
When fetch is not enough for JS-heavy pages
Plain fetch, even with a proxy, does not execute JavaScript. Sites that build pages with client-side code, wait for user actions, or run SPA flows will return an empty shell. For those pages you need a headless browser or a managed rendering service that can execute JS, wait for elements, click, scroll, and handle multi-step flows. A managed API can also take screenshots and run scripted scenarios while you keep calling it with fetch.
A managed alternative: one fetch call
Once you account for rotation, agents, retries, sticky sessions, and TLS fixes, a managed scraping API can be simpler. You call one endpoint with your API key and target URL, and the service handles proxy rotation, geolocation routing, headless rendering for JS-heavy sites, optional resource blocking, and AI-powered extraction. No agents, proxy lists, or session juggling.
Geo-targeting is also handled through a country or locale parameter, which avoids buying and rotating IPs from multiple countries yourself. For search results, a dedicated Google Search API can return region-accurate SERP data without captchas or browser maintenance.
Troubleshooting checklist
If a node-fetch proxy setup fails, check these in order:
- Proxy URL format: http://user:pass@host:port.
- Credentials: URL-encode special characters in username or password.
- Scheme: an HTTP proxy will not work if you point to an HTTPS URL by mistake.
- Reachability: confirm the proxy port is open from your machine (telnet, nc, curl).
- DNS: confirm resolution for both proxy host and target site.
- TLS: inspect the certificate chain; corporate MITM or self-signed certs can break the agent.
- Sticky sessions: confirm you reuse the same agent instead of creating a new one per call.
- Timeouts: increase them; some proxies add noticeable latency.
- Retries: add exponential backoff for transient failures.
- Blocking: check whether the target rejects requests based on headers, cookies, or missing browser fingerprints.
Proxies are only one layer
Modern anti-bot systems look at many signals: missing or incorrect headers, inconsistent cookies, unrealistic request timing, and browser fingerprints that do not match real traffic. A proxy alone will not prevent blocks if the requests do not resemble user behavior. Set realistic headers (especially User-Agent), handle cookies like a browser, and smooth out timing. For JS-heavy or fingerprint-sensitive pages, a headless environment or managed rendering service is usually the better option.
Frequently asked questions
Does Node's built-in fetch support proxies without extra libraries?
On Node 18–23, no: native fetch ignores HTTP(S)_PROXY and does not accept an agent option. You need an Undici dispatcher such as ProxyAgent or EnvHttpProxyAgent, or a managed scraping API. On Node 24+, you can enable proxy support via NODE_USE_ENV_PROXY=1 and HTTP(S)_PROXY / NO_PROXY.
What is the simplest way to use a proxy with node-fetch?
Create an HttpsProxyAgent and pass it through agent in your fetch() call. That is the cleanest and most reliable setup for node-fetch v2 and v3.
How do I pass proxy authentication to https-proxy-agent?
Put the credentials directly in the URL: http://user:pass@host:port. If the username or password has special characters, URL-encode them.
Why do I get UNABLE_TO_VERIFY_LEAF_SIGNATURE, and should I disable TLS checks?
That error signals a certificate the client cannot chain to a trusted root — either the proxy is re-signing HTTPS traffic or the endpoint presents a self-signed or otherwise untrusted cert. The fix is to supply the issuing CA (for example through NODE_EXTRA_CA_CERTS) rather than switching verification off. Turning TLS checks off is tolerable only for a throwaway local test, never for a deployed service.
How can I rotate proxies or keep sticky sessions with node-fetch?
Rotate by creating a new HttpsProxyAgent for each request, or keep sticky sessions by reusing the same agent for a TTL window. Combine this with retries, exponential backoff, and concurrency caps for stability.
When should I switch from DIY proxies to a scraping API?
When you need geolocation, reliability, large proxy pools, CAPTCHA handling, or JS rendering. A scraping API removes proxy maintenance and usually gives a much higher success rate.
Can I scrape JS-heavy pages with fetch only, or do I need a headless browser?
A fetch call returns markup but runs no scripts, so anything a page assembles in the browser stays missing. Single-page apps, content injected after load, and flows that depend on clicks or waits therefore require a headless browser, or a rendering service that performs that work on your behalf.
How do I test geo-restricted content from Node.js?
Two routes work: route requests through proxy IPs located in the target region, or hand a country parameter to a service that already operates regional exits. The second option spares you from running and rotating several country-specific proxy pools.
Are free proxies safe for development or production?
Free proxies are slow, unstable, and can inspect or manipulate your traffic. They are never safe for production and only barely acceptable for tiny experiments or throwaway scripts.