Skip to content

Python Requests POST: JSON, Forms, Files, Sessions and Safe Retries

A POST call in Python is short, but the choices inside it decide whether the server accepts your payload. The parameter you pass to requests.post() sets the request Content-Type, so json=, data= and files= are not interchangeable. This guide walks through the four body shapes you will meet, then adds the production guards: a timeout, a status check, a session for repeated calls, and retries only where repeating the request is safe.

A working POST in five lines

The minimal pattern sends a dict as JSON, applies a timeout, raises on a bad status, and reads the parsed response. The public httpbingo.org/post endpoint echoes back what it received, which makes it a useful sandbox for checking that your payload and headers arrived as intended. The complete runnable snippet is supplied as protected module code0001; keep its exact contents when publishing.

What each part does:

  • json=payload serializes the dict and sets Content-Type: application/json.
  • timeout=10 stops the call from hanging indefinitely.
  • raise_for_status() turns any 4xx or 5xx into a requests.HTTPError.
  • resp.json()["json"] shows the JSON the server actually received.

If you only need a one-off JSON POST, this is enough. The rest of the page covers the cases where a plain call is not enough.

Python example

import requests


payload = {"name": "alice", "role": "admin"}
resp = requests.post("https://httpbingo.org/post", json=payload, timeout=10)
resp.raise_for_status()
print(resp.json()["json"])  # {'name': 'alice', 'role': 'admin'}

POST or GET: the rule and the redirect trap

GET is for retrieval and should not change server state. POST submits data and usually creates a resource. Reach for POST when an HTML form declares method="post" or when API documentation specifies POST for the endpoint.

When you are reverse-engineering a site, open the browser developer tools, switch to the Network panel, submit the form, and read the method, payload and Content-Type of the resulting request. Reproduce those three details in your script.

One behavior catches people out: if the server answers a POST with 301, 302 or 303, Requests follows the redirect as a GET and drops the body. Only 307 and 308 preserve the method and payload. If a POST behaves unexpectedly, inspect resp.history to see whether a redirect changed the request.

Pick the body parameter before you write the request

The parameter you choose determines the Content-Type, so decide the body shape first. Use this map as a pre-flight check:

  • json= with a dict or list: serializes to JSON and sets application/json. Use for JSON APIs.
  • data= with a dict: sends application/x-www-form-urlencoded, the format an HTML form produces. Use for login forms and classic form endpoints.
  • data= with a list of tuples: repeats a key, e.g. [("tag", "python"), ("tag", "http")] becomes tag=python&tag=http.
  • data= with a str or bytes: sends the body untouched and sets no Content-Type, so you must supply it yourself. Use for pre-encoded or signed payloads, XML or protobuf.
  • files= with a file object: sends multipart/form-data. Open files in rb mode so Content-Length matches the byte size.

A common mistake is data=json.dumps(payload). It sends JSON bytes but no Content-Type header, and strict APIs reject it. Use json=payload instead. The exact snippets for each case are supplied as protected modules code0002 through code0005.

Python example

import json
import requests
payload = {"name": "alice", "role": "admin"}
good = requests.post("https://httpbingo.org/post", json=payload, timeout=10)
print(good.request.headers["Content-Type"]) # application/json
print(good.json()["json"])  # {'name': 'alice', 'role': 'admin'}


bad = requests.post("https://httpbingo.org/post", data=json.dumps(payload), timeout=10)
print(bad.request.headers.get("Content-Type")) # None
print(bad.json()["json"])  # None

Python example

import requests
login = {"username": "alice", "password": "swordfish"}
resp = requests.post("https://httpbingo.org/post", data=login, timeout=10)
print(resp.request.headers["Content-Type"])  # application/x-www-form-urlencoded
tags = [("tag", "python"), ("tag", "http")]
resp = requests.post("https://httpbingo.org/post", data=tags, timeout=10)
print(resp.request.body)  # tag=python&tag=http
print(resp.json()["form"])  # {'tag': ['python', 'http']}

Python example

import requests
resp = requests.post(
   "https://httpbingo.org/post",
   data="grant_type=client_credentials&scope=read",
   headers={"Content-Type": "application/x-www-form-urlencoded"},
   timeout=10,
)
print(resp.json()["form"])  # {'grant_type': ['client_credentials'], 'scope': ['read']}

Headers, cookies and authentication

Pass a dict to headers= to add or override request headers. Header names are matched without regard to letter case, so a lookup using "content-type" and one using "Content-Type" resolve to the same stored value. Do not set Content-Type by hand when you use json= or files=; those parameters populate it, and a wrong manual value breaks the request.

For a single request, cookies= sends cookies directly. For repeated calls, use a session so cookies persist automatically.

Authentication has two common forms:

  • auth=("user", "pass") sends HTTP Basic Auth.
  • A bearer token goes in the Authorization header, e.g. headers={"Authorization": f"Bearer {token}"}.

These are not interchangeable. A (user, pass) tuple always produces Basic, and if you pass both auth= and an Authorization header, auth= overwrites the header. Pick one. The exact snippets are supplied as protected modules code0006 through code0008.

Python example

import requests


headers = {"User-Agent": "post-demo/1.0", "X-Request-ID": "42"}
resp = requests.post(
   "https://httpbingo.org/anything",
   headers=headers,
   json={"hello": "world"},
   timeout=10,
)
print(resp.request.headers["User-Agent"])  # post-demo/1.0

Python example

import requests


resp = requests.post(
   "https://httpbingo.org/anything",
   cookies={"session_id": "abc123"},
   json={"hello": "world"},
   timeout=10,
)
print(resp.json()["headers"]["Cookie"])  # ['session_id=abc123']

Reading the response

Most APIs answer in JSON, and resp.json() converts that body into Python objects. If the body is not valid JSON, it raises requests.exceptions.JSONDecodeError. Use resp.text for HTML or plain text and resp.content for bytes such as images or PDFs.

When a POST misbehaves, inspect resp.request.body and resp.request.headers. They show what was actually sent, not what you intended to send. The exact inspection snippet is supplied as protected module code0009.

Python example

import requests
resp = requests.post("https://httpbingo.org/post", json={"key": "value"}, timeout=10)
print(resp.status_code)              # 200
print(resp.headers["content-type"])  # application/json; charset=utf-8
print(resp.elapsed.total_seconds())  # latency check, e.g. 0.21
print(resp.json()["json"])           # {'key': 'value'}
print(resp.text[:80])                # body decoded as str
print(type(resp.content))            # <class 'bytes'>, for binary bodies
print(resp.request.headers["Content-Type"], resp.request.body) # application/json b'{"key": "value"}'

Three guards that make a POST production-safe

Requests does not set a default timeout, so a stalled server can block your worker indefinitely. Pass a (connect, read) tuple, for example timeout=(3.05, 27). Connect is the time to establish the connection; read is how long to wait for the server to send bytes, not the whole download.

By default, status codes are just numbers. raise_for_status() turns any 4xx or 5xx into a requests.HTTPError, so a bad response cannot slip downstream as if it were data.

Catching requests.exceptions.RequestException covers timeouts, DNS failures, refused connections and SSL errors in one branch. Remember that success is not only 200: a create often returns 201 Created, and a success with no body returns 204 No Content. Calling json() on a 204 fails because the body is empty, so read the status before deciding what to do. The exact try/except snippet is supplied as protected module code0010.

Status-code triage:

  • 400: your payload or format is wrong.
  • 401: missing or bad credentials.
  • 403: the server recognized you but will not allow the request.
  • 429: you are sending too fast; slow down.
  • 5xx: the server's problem.

Compressing large POST bodies

Requests does not compress request bodies. Large JSON batches (events, logs, catalogs) are mostly repeated text and shrink well, but the server must be able to decompress them.

Before implementing compression, check the API documentation. Many servers do not decompress request bodies and reject the request, typically with 400 or 415. If the API accepts it, compress the bytes yourself and set both headers: Content-Encoding says how the bytes are packed, and Content-Type says what is inside. Use json.dumps() rather than json=, because json= sends the body uncompressed. Brotli is a drop-in swap (brotli.compress(raw) with Content-Encoding: br) but is smaller and slower to compress and less commonly accepted. Skip compression for small payloads, where the CPU cost can outweigh the savings. The exact gzip snippet is supplied as protected module code0011.

Sessions and retries: enable POST retries only when safe

Every requests.post() call opens a new connection. If you send more than one request to the same site, use requests.Session. A session gives you connection reuse (no repeated TCP/TLS handshake), persistent cookies (a login cookie is sent on later requests), and shared defaults (headers, params or auth set once). Use with requests.Session() as session so the pool closes when you are done.

A session has no default timeout, so set timeout on each call.

POST is not retried by default because repeating it can create the same resource more than once. To make POST retryable, add "POST" to allowed_methods only after the endpoint is idempotent, for example through idempotency keys or server-side deduplication. If the urllib3 mount is not flexible enough, Tenacity can retry at the method level. The exact session and Tenacity snippets are supplied as protected modules code0012 and code0013.

When another HTTP client fits better

httpx offers a Requests-style API with async and HTTP/2 support, and the same API works for sync and async calls. If you do not need those features, stay with Requests. At the time of writing, httpx was still pre-1.0 (0.28.1), so pin the version.

aiohttp is a pure async HTTP client with a more verbose API. It fits when you already run on the asyncio event loop and need streaming.

curl_cffi is a drop-in Requests replacement that impersonates a real browser's TLS and HTTP/2 fingerprints, which can help when a target blocks plain Requests based on JA3 or JA4 fingerprints.

For synchronous POSTs that are not fingerprint-sensitive, plain Requests remains the default choice.

Body parameter decision map
ParameterValue typeContent-Type sentUse when
json=dict or listapplication/jsonJSON API endpoints
data=dictapplication/x-www-form-urlencodedHTML form submissions, login forms
data=list of tuplesapplication/x-www-form-urlencodedRepeated keys such as tag=python&tag=http
data=str or bytesnone set by RequestsPre-encoded or signed payloads, XML, protobuf; set Content-Type yourself
files=file object opened in rbmultipart/form-dataFile uploads

Content-Type values reflect what Requests sets automatically for each parameter; the raw str/bytes case requires a manual header.

Failure triage checklist
SymptomLikely categoryFirst check
400 Bad RequestPayload or format wrongCompare resp.request.body and Content-Type against API docs
401 UnauthorizedMissing or bad credentialsConfirm auth= or Authorization header is present and correct
403 ForbiddenServer recognized you but refusedCheck permissions, user agent and access rules
429 Too Many RequestsSending too fastSlow down or add backoff
5xxServer-side problemRetry only if the endpoint is idempotent
RequestExceptionNetwork-level failureCheck timeout, DNS, connection and SSL settings

Categories reflect the status-code meanings described for this guide; server-specific causes may differ.

Frequently asked questions

How do I set a timeout for a POST request in Python?

Give timeout= a two-part (connect, read) tuple, such as timeout=(3.05, 27), so the connection phase and the wait for response bytes are bounded independently. Requests applies no timeout unless you supply one, which means an unresponsive server can leave your code waiting without end; production calls should always carry an explicit value.

How do I retry a failed POST request in Python?

Mounting a urllib3.util.Retry adapter on a requests.Session is the most direct route. Because a POST is non-idempotent by design, the retry layer leaves it out unless you opt in; add "POST" to allowed_methods only once you have confirmed the endpoint tolerates a repeated submission.

How do I send custom headers like Authorization with a POST request?

Supply a dict through headers=, for example headers={'Authorization': 'Bearer ' + token}. When json= or files= is in play, leave Content-Type out of that dict: Requests preserves whatever you pass, and an incorrect value will break the request. If the same headers recur, set them once with session.headers.update(...) rather than rebuilding the dict per call.

How do I follow or disable redirects on a POST request?

Requests follows redirects by default, but on 301-303 it converts POST into a GET and drops the body. HTTP 307 or 308 keeps the method and body. Disable redirects with allow_redirects=False.

How do I disable SSL verification with requests.post?

verify=False is appropriate only for self-signed certificates during development. Turning it on in production removes certificate validation, which leaves the connection open to man-in-the-middle interception. A safer alternative is to point verify at a specific bundle, verify='/path/to/ca-bundle.pem', so that one CA is trusted, or to install the certificate on the local machine.

How do I send a POST through a proxy in Python?

Provide a proxies dict that names the proxy for each scheme, for example proxies={'http': 'http://user:pass@host:8080', 'https': 'http://user:pass@host:8080'}. When each request should leave from a different IP, pass that dict on the individual call instead of attaching it to the session, or hand the rotation off to a scraping API that changes IPs for you.

What is the difference between the data and json parameters?

Use json=payload to send a JSON body; it serializes the dict and sets Content-Type: application/json. Use data=payload to send application/x-www-form-urlencoded, the format HTML forms use. Passing data=json.dumps(payload) sends JSON bytes but does not set the header, which breaks strict APIs.