Ruby Faraday Proxy Setup: Plain, Authenticated, Env, and Rotating
Faraday is a widely used Ruby HTTP client whose adapter-based design lets you keep the same request code while swapping the underlying transport. This guide shows how to attach a proxy to a Faraday connection, add credentials, fall back to environment variables, rotate proxies across requests, and recognize the certificate error that often appears when a proxy intercepts TLS.
Why Faraday is a practical base for proxied requests
Faraday separates the request-building layer from the HTTP adapter. By default it uses Net::HTTP, but you can substitute adapters such as Excon, Typhoeus, Patron, or EventMachine with only a small configuration change. That flexibility means a proxy setup written against Faraday's connection object usually survives an adapter swap, and you avoid rewriting request logic when the transport changes.
What a proxy changes for scraping workloads
A proxy sits between your Ruby process and the target site, so the site sees the proxy's address rather than yours. That matters when you send many requests: sites commonly throttle or block addresses that exceed their tolerance. Cycling through a pool of proxies spreads the request volume across addresses and keeps your own IP out of the block list. The trade-off is that each proxy adds a hop, and free public proxies are frequently slow, unstable, or already blocked.
Project setup and dependency install
Create a working directory and an entry file, then install the Faraday gem. The captured install output shows Faraday 2.7.1 being installed, which is a useful reference point for the API shape used below.
Baseline request before adding a proxy
Start with a plain GET to an IP-echo endpoint so you can see your own public address. Once that works, any change in the printed address after adding a proxy is direct evidence that the proxy is being used.
Attaching a plain proxy to a Faraday connection
Pass the proxy URL as the :proxy option on Faraday.new. The response body from the IP-echo endpoint should then show the proxy's address instead of your own. Note the scope rule: Faraday lets you change many settings per connection or per request, but proxy information is only read at connection initialization. If you need a different proxy, build a new connection.
Authenticated proxies and the TLS certificate error
For a proxy that requires credentials, embed the username and password in the proxy URL as user:pass@host:port. If the proxy terminates TLS with its own certificate, Faraday may raise a Faraday::SSLError reporting a self-signed certificate. The captured workaround adds :ssl => {verify: false} to the connection options. Treat that as a debugging step, not a default: disabling verification removes protection against man-in-the-middle attacks, so prefer a proxy whose certificate chain validates, or pin the expected certificate, in production.
Using http_proxy and https_proxy environment variables
Faraday also honors the standard http_proxy and https_proxy environment variables. With no proxy option in the code, a connection built against an https URL picks up https_proxy automatically, and conn.proxy returns a Faraday::ProxyOptions struct showing the resolved URI. Credentials can be included in the environment variable value in the same user:pass@host:port form. This is convenient for CI or container environments where the proxy is supplied externally, but it is process-wide, so it is a poor fit when different requests need different proxies.
Rotating proxies across requests
A simple rotation strategy keeps a list of proxy hashes and picks one at random for each new connection. Because proxy settings are fixed at connection creation, rotation means constructing a fresh Faraday connection per request rather than mutating an existing one. The captured example uses proxy_list.sample and interpolates protocol, host, and port into the proxy URL; the same hashes can carry credentials if the proxies are authenticated. Random selection is a starting point, not a scheduling policy: for rate-limited targets, a round-robin or health-checked pool usually behaves better than pure randomness.
When a managed proxy service replaces a hand-built pool
Maintaining a list of working proxies is ongoing work: entries expire, get blocked, or slow down. A managed proxy endpoint handles rotation on the provider side, so your Faraday code points at one host and the provider changes the egress address between requests. The captured example routes through a proxy host with the API key as the username and a parameter string as the password, and it disables JS rendering while requesting premium proxies. Those parameter names and the credential placement are provider-specific; confirm them against the provider's current documentation before relying on them. Any free-credit allowance or billing behavior is a commercial term that must be confirmed with the provider rather than assumed from this guide.
Decision checklist before you ship a proxy setup
Use this checklist to pick the right mechanism and to separate the failure modes you are most likely to hit.
- Scope: does the proxy apply to the whole process (environment variable), one connection (the :proxy option), or a rotating pool (new connection per request)?
- Credentials: are they in the proxy URL, and are they kept out of source control and logs?
- TLS: does the proxy present a certificate your Ruby trust store accepts? Only disable verification as a temporary diagnostic.
- Verification: does the IP-echo endpoint return the proxy address, not your own?
- Rotation: is random selection adequate, or does the target need round-robin or health checks?
- Expiry: are public proxy entries refreshed, or is a managed endpoint handling rotation?
Failure distinction: a connection refused or timeout points at the proxy host or port; a 407 response points at missing or wrong credentials; a Faraday::SSLError with a self-signed certificate points at TLS interception by the proxy, not at the target site.
Frequently asked questions
Can I change the proxy for a single request on an existing Faraday connection?
No. Faraday reads proxy information when the connection is initialized, so a different proxy requires a new Faraday.new call. Other settings can be adjusted per connection or per request, but the proxy is fixed for the life of the connection object.
Why do I get a self-signed certificate error when using a proxy?
The proxy is presenting its own certificate for the TLS connection. Faraday reports this as Faraday::SSLError with a message about a self-signed certificate. Disabling verification with :ssl => {verify: false} makes the request proceed, but it also removes protection against interception. Prefer a proxy with a valid certificate chain, or pin the certificate you expect.
Does Faraday pick up https_proxy automatically?
Yes. If no :proxy option is given, Faraday resolves the proxy from the http_proxy and https_proxy environment variables. You can inspect the result through conn.proxy, which returns a Faraday::ProxyOptions struct. Because the setting is process-wide, it is less suitable when different requests need different proxies.
How do I rotate proxies with Faraday?
Keep a list of proxy entries, select one per request, and build a new Faraday connection with that proxy. The captured example uses proxy_list.sample for random selection. For targets with strict rate limits, a round-robin or health-checked pool is usually more predictable than random choice.
$ ruby app.rb
{"ip"=>"73.162.70.177"}