Skip to content

Download files from Python by calling wget

Python wget is not a separate Python package. It is a pattern: Python uses its built-in subprocess module to launch the external wget program, and wget performs the HTTP, HTTPS or FTP transfer. That separation is useful because wget already streams data to disk, resumes partial transfers, retries failures and follows links recursively, so a script does not need to reimplement those behaviours.

Set up Python and wget first

Install wget for your operating system, confirm Python 3.10 or later, and verify wget with wget --version. An exit code of 0 means the check succeeded. The install commands are provided as protected module code0001, with per-platform variants in code0009, code0010, code0011 and code0012. Project setup with uv is shown in code0004, and the basic script skeleton is in code0005. Because subprocess gives a script direct access to the shell, only run code whose commands you have read and trust.

Three patterns cover most jobs

Start with subprocess.run and a list containing wget and the URL, as shown in protected module code0002 and explained in code0017. Check result.returncode: 0 indicates success and any other value indicates failure, with the exact meaning depending on the error. By default wget prints progress and errors to the terminal through stdout and stderr; capture them only when logging or debugging requires it. Because wget writes incoming chunks directly to disk, large files do not need to fit in memory.

Choose between -O and -P without mixing them up

Use -O when the exact filename matters. Use -P when the folder matters but the server filename should be kept. If both are needed, give -O a full path such as a downloads folder plus a filename. Creating the target folder with os.makedirs and exist_ok=True makes the script more portable, since some wget builds may fail when the folder is absent. Remember that -O overwrites an existing file without warning, so use a timestamped name or an existence check for repeat runs. Relevant protected modules are code0020, code0021 and code0023.

Resume instead of restarting

Add -c or --continue to request only the missing bytes when a partial file is already present. This works only when the server supports range requests; otherwise wget starts again. For flaky connections, combine resume with --tries, or wrap the call in a Python retry loop with a delay, as shown in protected module code0027. A 206 Partial Content response is the visible sign that resume is working.

Optional flags for stability and control

Use --limit-rate to throttle bandwidth, --tries to retry failed transfers, -q to suppress progress output, --timeout to stop a download hanging indefinitely, and --no-check-certificate only for servers with misconfigured HTTPS when no safer option exists. The -b flag detaches wget, but subprocess.run still waits for the handoff; use subprocess.Popen when Python must continue immediately, as shown in code0032. Flags can be combined, for example retries plus timeout plus rate limiting, to balance stability against bandwidth use. See protected modules code0028 through code0035.

Recursive downloads for offline mirrors

Recursive retrieval follows links from a starting page. The key controls are -r to enable recursion, -l to set depth, -p to fetch page requisites such as images and CSS, and -k to rewrite links for offline viewing. The default depth is 5, while 0 or inf means unlimited, so use a small depth on large sites. The mirroring command is provided as protected module code0036. Recursion is suitable for offline snapshots, but it does not execute JavaScript or handle anti-bot defences.

Know when wget is the wrong tool

wget is a downloader, not a browser or extraction engine. If a task needs clicks, scrolling, JavaScript rendering, structured extraction or protection against rate limiting and blocking, consider a browser-automation tool, a parsing library, or a hosted scraping service instead. For broader protocol support, curl is the closer alternative. Use this decision check: static file or mirror means wget; partial-page extraction means a parser; interactive or dynamic pages mean browser automation.

Flag decision map
GoalUseWatch out for
Keep server filename in a chosen folder-PCreate the folder first on some systems
Force an exact filename-OOverwrites without warning
Force filename and folder together-O with a full pathDo not also rely on -P
Resume a partial file-c / --continueNeeds server range-request support
Retry failures--triesCombine with --continue for dropped connections
Throttle bandwidth--limit-rateValues such as 500k or 2m
Mirror linked pages-r with -l, -p, -kLimit depth on large sites

Flags are passed as separate list items to subprocess.run; -O overrides -P.

Frequently asked questions

What is Python wget and why is it useful for file downloads?

It means launching the wget command-line program from Python through subprocess, so Python controls the job while wget handles transfer, resume, retries and writing chunks to disk.

How do I install wget for use with Python scripts?

Install the operating-system binary for your platform, then confirm it is on PATH with wget --version. Python does not install wget itself; it only calls it. See protected module code0039.

Can I customize the filename when downloading with Python wget?

Use -O followed by the desired name. An existing file with that name is overwritten, so add a timestamp or suffix, or check existence first, when overwriting is a risk. See protected modules code0040 and code0041.

How can I download files to a specific folder using Python wget?

Use -P to choose a folder while keeping the server filename, or pass a full path to -O to set both folder and name. Do not combine -O and -P expecting both to apply, because -O takes precedence and -P is ignored. See protected modules code0042 and code0043.

Is it possible to resume interrupted downloads with Python wget?

Use -c or --continue when a partial file exists. Resume depends on server support for range requests; without that support wget downloads the file again. Combining --continue with --tries adds automatic retries. See protected modules code0044 and code0045.

Can I limit download speed with Python wget?

Use --limit-rate with a value such as 500k or 2m to throttle bandwidth. See protected module code0046.

Can I download multiple files at once?

Use -i with a text file containing one URL per line, or loop over a Python list and call wget for each URL. See protected modules code0047 and code0048.