curl with a proxy: flags, schemes, auth and troubleshooting
A proxy lets curl hand a request to an intermediary server instead of contacting the destination directly. That is useful when a response depends on the apparent location of the caller, when a network requires an egress point, or when you are testing how a site behaves from another region. This guide covers the command-line flags, the proxy schemes curl understands, credential handling, persistent configuration, one-off overrides and the errors that most often appear.
Copy-paste proxy commands
The shortest form uses -x (or its long form --proxy) followed by the proxy address and then the destination URL. The examples below use a local proxy and a test endpoint that echoes the caller's address, so you can confirm which path the request actually took.
What curl does with a proxy
curl is a command-line data transfer tool that has been available since 1996 and supports HTTP/HTTPS, FTP, SCP and other protocols. It is deliberately literal: it does not correct a malformed URL or guess a missing scheme. When you add a proxy, curl connects to that proxy first and lets it handle the destination. For a plain HTTP proxy the target hostname is normally resolved by the proxy, not by your machine, which is why options such as --resolve do not change traffic sent through an HTTP proxy.
A plain request, headers and redirects
A bare request to a non-www host often returns a 301 pointing at the www version. curl does not follow redirects unless you ask it to, so the response body may be a short redirect page rather than the content you expected. Add -L to follow the hop, or request the final URL directly. To inspect only the response headers, use -I.
HTTP, HTTPS and SOCKS proxies
The scheme in the proxy URL describes the client-to-proxy leg. An http:// proxy is the common default and works for most web requests. An https:// proxy encrypts the connection between your machine and the proxy; the destination still runs its own TLS session. For any HTTPS destination through a classic proxy, curl uses the CONNECT method to open a tunnel, and the proxy cannot read the decrypted traffic unless it performs a full man-in-the-middle interception with a certificate your machine trusts.
SOCKS4 and SOCKS5, and where DNS happens
SOCKS proxies forward raw traffic without caring about the destination protocol. SOCKS5 adds authentication, UDP support and better compatibility with picky sites. The h suffix in socks5h:// is the part that changes DNS behaviour: with socks5:// curl resolves the target hostname locally and sends the resulting IP to the proxy, while with socks5h:// the proxy performs the lookup. Use socks5h:// when you want the target lookup to originate from the proxy's address.
FTP behaves differently
Proxy support for FTP is not as consistent as for HTTP and HTTPS. HTTP proxies were not designed for FTP commands, so uploads such as STOR and PUT often fail or behave unpredictably. SOCKS proxies struggle with FTP's dual control and data connections: passive mode may work while active mode often breaks. An HTTPS proxy is a TLS-wrapped HTTP proxy, so the same FTP limitations apply.
Choosing a scheme for scraping
HTTP proxies are the fastest and simplest option but are also the easiest for a site to detect. HTTPS proxies add privacy for the client-to-proxy leg while staying simple to configure. SOCKS5 is the most flexible choice for heavy scraping, filter bypassing or mixed protocols. If the target needs JavaScript rendering, pair the proxy with a headless browser or use a managed service rather than maintaining a rotating pool yourself.
Scheme decision table
Use this table to pick a scheme before writing the command. It summarises the behaviour described above; it is not a benchmark.
Proxy authentication
Credentials can go inside the proxy URL or be supplied with -U. Passwords containing special characters must be quoted. Basic authentication is the default. curl also supports --proxy-digest and --proxy-ntlm, but modern proxy deployments rarely use them; NTLM in particular is known for stalling mid-handshake, and many corporate proxies have disabled both. In practice you will usually meet Basic auth or Kerberos/Negotiate. A 407 or a timeout while using Digest or NTLM is a protocol mismatch, not a typo in your command.
Manual Proxy-Authorization headers
You can set a Proxy-Authorization header yourself with --proxy-header. Doing so bypasses curl's built-in proxy authentication logic, so do not combine a manual Proxy-Authorization header with -U or --proxy-anyauth unless you are certain of the result.
Keeping credentials out of history
Plain-text credentials in shell history or committed scripts are easy to leak. Prefer environment variables or an interactive password prompt, and avoid embedding credentials where they are not needed. {{OWNER_CREDENTIAL_HANDLING_NOTE}}
Environment variables
curl reads http_proxy, https_proxy and NO_PROXY automatically. On Linux and macOS use export; in Windows CMD use set; in PowerShell use $env:. In PowerShell, curl is an alias for Invoke-WebRequest, so call curl.exe explicitly. Casing matters: an HTTP_PROXY variable is ignored for security reasons, while HTTPS_PROXY and NO_PROXY may exist in either case.
NO_PROXY and suffix matching
NO_PROXY tells curl which hosts should bypass the proxy. Values are treated as hostname suffix matches, not glob patterns: *.example.com is not supported, while example.com matches example.com and any host ending in example.com such as api.example.com. A leading dot behaves the same as no dot. Recent curl versions also accept CIDR ranges for IPs.
Session scope versus permanent settings
Exported variables last only for the current shell session. To keep them, add them to ~/.bashrc, ~/.profile or ~/.zshrc, or persist them on Windows through System Properties or [System.Environment]::SetEnvironmentVariable. These variables affect every tool that respects them, including wget, apt and npm, so set them deliberately.
Aliases for repeated use
An alias can wrap curl with a fixed proxy, for example alias curl="command curl -x http://127.0.0.1:5000". Naming it curlp avoids breaking scripts that expect plain curl. A more flexible version reads the proxy from $http_proxy, so switching proxies means changing one variable. Windows CMD has no aliases, but a .bat file placed on PATH can call curl.exe with the proxy flag and pass through arguments with %*.
The curlrc configuration file
A curlrc file stores default options so curl loads them on every run. curl checks for a config file unless -q is passed as the first argument; command-line flags override file settings. On Linux and macOS the file is $HOME/.curlrc, or $CURL_HOME/.curlrc when that variable is set. On Windows curl derives a home directory from CURL_HOME, HOME or %APPDATA%, looks for _curlrc or .curlrc there, and then checks the directory containing curl.exe. The exact search order depends on the build, so consult the official documentation. Only the curl CLI reads this file; tools built on libcurl ignore it.
Python example
proxy = "http://127.0.0.1:5000"Rotating proxies
A single proxy becomes a bottleneck once scraping becomes aggressive. A small Bash script can pick a random proxy per request from an array. This is the simplest form of rotation, but free proxies die often, are slow, get banned quickly and may act as malicious man-in-the-middle servers. Use DIY rotation for testing, not production. {{OWNER_MANAGED_ROTATION_DESCRIPTION}}
Troubleshooting
Most proxy failures fall into a few categories: the connection never reached the proxy, the proxy rejected the credentials, the target rejected the proxy's address, or the proxy could not tunnel HTTPS. Verbose mode with -v shows whether the proxy connection was attempted, any redirects or handshake issues, authentication challenges and errors returned by the proxy. Start there before changing flags.
Failure distinction table
The same symptom can come from different causes. Use this table to separate them before editing your command.
SSL errors and self-signed certificates
-k or --insecure disables certificate validation entirely and is suitable only for local development. --proxy-insecure disables verification of the proxy's certificate only, leaving the destination certificate validated. For production, point curl at the actual CA certificate with --proxy-cacert, and add --cacert if the destination also uses a custom CA. Disabling validation exposes you to man-in-the-middle attacks.
When the proxy setting is ignored
If a request goes out directly, check four things: environment variables overriding your flags, an unexpected .curlrc setting, a NO_PROXY match for the host, and an alias that is not redirecting the call. To force a single direct request regardless of configuration, use --noproxy '*'. To skip the proxy for one domain only, pass that domain to --noproxy. Passing an empty string to -x also overrides the proxy for that call.
When CONNECT is not supported
curl tunnels HTTPS through an HTTP proxy with CONNECT. If the proxy does not support CONNECT, HTTPS requests cannot be proxied and curl will not fall back to a direct connection; the request fails. Verbose output shows the CONNECT line. A 405 Method Not Allowed, a 403 Forbidden, a dropped connection or an SSL handshake error after the CONNECT attempt all point to this failure.
Headers, redirects and compression
A custom user agent with -A helps a request blend in when a server treats the default curl signature differently. -H adds headers such as Accept-Language for locale testing. -L follows redirects so you do not stop at a 301 or 302. --compressed asks the server for gzip or deflate content, which reduces bandwidth over a proxy. {{OWNER_MANAGED_HEADER_HANDLING_NOTE}}
Extracting data from the response
curl output can be piped into other tools. A regex against the <title> tag works for quick checks but is brittle for production, and grep -P may not be available on macOS by default. The same applies to meta description extraction. When the response is JSON, a real parser such as jq is far more maintainable. On Windows CMD the grep and jq examples need GNU tools installed; in PowerShell, Select-String can replace grep. Any of these can be combined with a proxy by adding the usual -x flag.
When to move from DIY proxies to an API
A working curl proxy setup stops being enough when the site bans your addresses, when content depends on region, when the page needs JavaScript to render, or when maintaining rotation takes more time than extracting data. At that point a scraping API can handle rotation, retries, JavaScript rendering and geolocation while you keep sending simple curl requests. {{OWNER_MANAGED_API_DESCRIPTION}} {{OWNER_TRIAL_TERMS}}
Conclusion
The methods above cover one-off flags, environment variables, aliases, curlrc files and rotation scripts. If you use the same proxy constantly, a curlrc entry is usually the least repetitive option. Any of the approaches works as long as it produces the request you intended.
| Proxy scheme | Client-to-proxy leg | Target DNS resolved by | HTTPS destination handling | Best fit |
|---|---|---|---|---|
| http:// | Plain | Proxy | CONNECT tunnel | Fast, simple web requests |
| https:// | TLS | Proxy | CONNECT tunnel; destination TLS separate | Privacy on the client-to-proxy leg |
| socks4:// | Raw | Local curl | Raw forwarding | Mixed protocols, older setups |
| socks5:// | Raw | Local curl | Raw forwarding | Mixed protocols with SOCKS5 features |
| socks5h:// | Raw | Proxy | Raw forwarding | Heavy scraping where target lookup should leave from the proxy |
Behaviour summary drawn from the guide's descriptions; not a benchmark. FTP is excluded because proxy support varies by server and mode.
| Symptom | Likely cause | First check |
|---|---|---|
| 407 Proxy Authentication Required | Credentials wrong, unquoted special characters, or unsupported auth scheme | Verify username and password, quote special characters, try --proxy-anyauth |
| 403 after a successful proxy connection | Target rejected the proxy address or rate limit | Try a different region or rotate the address |
| 405, 403, dropped connection or SSL handshake error after CONNECT | Proxy does not support CONNECT | Run with -v and look for the CONNECT line |
| Request goes out directly | NO_PROXY match, conflicting environment variable, curlrc setting or alias | Force a direct request with --noproxy '*' to confirm, then remove the override |
| SSL warning from a self-signed proxy certificate | Proxy certificate not trusted | Use --proxy-cacert with the real CA in production; -k only for testing |
Diagnostic aid based on the guide's troubleshooting descriptions; not a substitute for verbose output.
Frequently asked questions
How do I send a curl request through an HTTP proxy?
Pass -x or --proxy followed by the proxy address, then the destination URL. The scheme defaults to http:// when omitted, so -x 127.0.0.1:8080 is equivalent to -x http://127.0.0.1:8080.
How do I add a username and password to a curl proxy?
Put the credentials in the proxy URL, or use -U user:pass. Quote passwords that contain special characters. Basic authentication is the default; --proxy-anyauth lets curl negotiate the scheme.
What is the difference between HTTP/HTTPS and SOCKS proxies in curl?
HTTP proxies forward ordinary HTTP requests and use CONNECT tunnelling for HTTPS destinations. An HTTPS proxy is an HTTP proxy wrapped in TLS for the client-to-proxy leg. SOCKS4 and SOCKS5 forward raw TCP data; SOCKS5 adds authentication and remote DNS resolution.
How do I disable proxy settings for one curl request?
Use --noproxy '*' to bypass every proxy source, or pass an empty string to -x. To skip the proxy for one domain only, pass that domain to --noproxy.
Why does my real IP still appear when I use a curl proxy?
Either the request did not go through the proxy, or DNS is still resolved locally. Check for a NO_PROXY match, a conflicting environment variable or a curlrc setting overriding your flags. For SOCKS, use socks5h:// so the proxy performs the lookup. Run with -v to confirm the proxy connection; if an HTTPS proxy does not support CONNECT, the request fails rather than leaking your address.
How do I rotate proxies with curl?
A script can select a different proxy for each request, for example by indexing an array with $RANDOM. For production workloads, a managed rotation service is more reliable than a hand-maintained list.
Can I use a managed proxy API instead of setting up proxies manually?
Yes. A managed scraping API handles rotation, retries, geolocation and JavaScript rendering while you keep sending simple curl requests. {{OWNER_MANAGED_API_DESCRIPTION}}
How do I set curl proxy environment variables on macOS, Linux or Windows?
On macOS and Linux use export http_proxy and export https_proxy. In Windows CMD use set. In PowerShell use $env:http_proxy and call curl.exe explicitly because curl is an alias for Invoke-WebRequest.
When should I use -k or --insecure with a curl proxy?
Only during testing, when a proxy uses a self-signed certificate. These flags disable TLS validation and should not be used in production.
How do I safely use curl with a proxy that has a self-signed certificate?
For development you can disable certificate checks. For production, supply the real CA certificate with --proxy-cacert, and add --cacert if the destination also uses a custom CA.
What is the default proxy protocol for curl?
If you omit the scheme, curl assumes http:// for the proxy.
curl google.com
curl -I google.com
curl -L https://google.com
curl "{{APP_BASE_URL}}/api/v1/?api_key=YOUR_KEY&url=https://httpbingo.org/ip"
curl --help proxy
curl -x http://127.0.0.1:5000 https://httpbingo.org/ip
curl --proxy http://127.0.0.1:5000 https://httpbingo.org/ip
curl -x 127.0.0.1:5000 https://httpbingo.org/ip
curl -x http://user:[email protected]:5000 https://httpbingo.org/ip
curl -x http://user:"p@ssw0rd!"@127.0.0.1:5000 https://httpbingo.org/ip
curl --proxy http://127.0.0.1:5000 --proxy-anyauth -U user:password https://httpbingo.org/ip
curl -L -x http://127.0.0.1:5000 https://example.com
curl -U user:password --proxy http://127.0.0.1:5000 https://httpbingo.org/ip
curl --proxy http://127.0.0.1:5000 --proxy-digest -U user:password https://httpbingo.org/ip
curl --proxy http://127.0.0.1:5000 --proxy-ntlm -U user:password https://httpbingo.org/ip
curl --proxy http://127.0.0.1:5000 \
--proxy-user user:password \
--proxy-header "Proxy-Authorization: Basic dXNlcjpwYXNz" \
https://httpbingo.org/ip
curl https://httpbingo.org/ip
export http_proxy=http://127.0.0.1:5000
export NO_PROXY=localhost,.example.com
curl https://localhost/test # bypasses proxy
curl https://api.example.com/data # bypasses proxy
curl https://another.site # goes through proxy
alias curl="command curl -x http://127.0.0.1:5000"
curl https://httpbingo.org/ip
curl -x http://127.0.0.1:5000 https://httpbingo.org/ip
# Create an alias for cURL with proxy
alias curlproxy='curl -x $http_proxy'
curl -x http://127.0.0.1:5000 https://httpbingo.org/ip
curl -x https://127.0.0.1:5000 https://httpbingo.org/ip
# This only disables verification of the proxy's cert; the site cert is still validated
curl --proxy-insecure -x https://proxy:port https://example.com
curl -x socks4://127.0.0.1:1080 https://httpbingo.org/ip
curl -x socks5h://127.0.0.1:1080 https://httpbingo.org/ip
curl -x socks5h://127.0.0.1:1080 https://example.com
#!/bin/bash
proxies=("http://proxy1:8080" "http://proxy2:8080" "http://proxy3:8080")
proxy=${proxies[$RANDOM % ${#proxies[@]}]}
curl -x "$proxy" https://httpbingo.org/ip
curl "{{APP_BASE_URL}}/api/v1/\
?api_key=YOUR_API_KEY\
&url=https://httpbingo.org/ip"
curl -v -x http://proxy.example.com:8080 https://httpbingo.org/ip
curl -k -x http://proxy.example.com:8080 https://https-only-site.com
curl --noproxy "*" https://httpbingo.org/ip
curl --proxy-anyauth -U user:pass -x http://proxy.example.com:8080 https://httpbingo.org/ip
curl -v -x http://proxy.example.com:8080 https://example.com
curl --noproxy "*" https://httpbingo.org/ip
curl --noproxy "example.com" https://example.com/data
curl -x "" https://httpbingo.org/ip
curl -x http://proxy.example.com:8080 \
-A "Mozilla/5.0 (Windows NT 10.0; Win64; x64)" \
https://example.com
curl -x http://proxy.example.com:8080 \
-H "Accept-Language: en-US" \
https://example.com
curl -x http://proxy.example.com:8080 -L https://example.com
curl -x http://proxy.example.com:8080 --compressed https://example.com
curl --silent https://snapshooter.com/ | grep -oP '(?<=<title>).*(?=</title>)'
curl --silent https://www.boxmode.com/ | grep -oP '(?<=<meta[^>]+name="description"[^>]+content=")[^"]*'
curl -s https://httpbingo.org/ip | jq ".origin"
curl -s -x http://127.0.0.1:5000 https://httpbingo.org/ip | jq ".origin"
curl -x http://127.0.0.1:8080 https://example.com
curl -x http://user:[email protected]:8080 https://example.com
curl -U user:pass -x http://proxy:8080 https://example.com
curl --noproxy "*" https://example.com
curl -x "" https://example.com
proxylist=(p1 p2 p3)
curl -x ${proxylist[$RANDOM % ${#proxylist[@]}]} https://example.com
curl -k -x http://proxy:8080 https://example.com
curl -k -x https://proxy:8443 https://example.com
curl --proxy-insecure -x https://proxy:8443 https://example.com
curl \
--proxy-cacert /path/to/proxy-ca.pem \
-x https://proxy:8443 \
https://example.com
curl \
--proxy-cacert /path/to/proxy-ca.pem \
--cacert /path/to/site-ca.pem \
-x https://proxy:8443 \
https://example.com