Skip to content

Faster Playwright Scraping: Waits, Blocking and Parallel Contexts

Playwright slows down mainly when a script waits for more than it needs and downloads material it never uses. This guide explains how to choose a lean setup, time navigation around the data you actually need, cut unnecessary traffic, and scale with isolated contexts.

Start with the right setup

Two habits explain most sluggish runs: waiting for a whole page when only one element matters, and fetching images, fonts, media, ads and trackers that contribute nothing to extraction. The quickest improvements are to navigate on domcontentloaded, wait for a specific selector, avoid fixed sleeps, and stop heavy asset types. A complete starter pattern combining those ideas is provided in protected module code0001.

Use Playwright when JavaScript rendering, interaction or browser-like behaviour is required. If a site serves the needed data over plain HTTP, a browser adds overhead without benefit. Node.js often feels quicker because Playwright is JavaScript-first, gets features earlier and has lower startup overhead; Python remains workable, but keep async flows free of blocking sync calls. Install only the browser you need, normally Chromium, using protected modules code0002 to code0007. Keep headless mode for production and reserve headful mode, optionally with slow motion, for selector and timing diagnosis, as shown in code0008 and code0009.

Time navigation around proof, not pixels

Chromium is the sensible default for scraping because it starts quickly, handles current JavaScript-heavy pages smoothly and matches how most sites are tested. Firefox can differ on newer language or style features, while WebKit is mainly valuable when Safari behaviour itself is under test and can be slower or more timing-sensitive.

For navigation, domcontentloaded means the initial HTML has been parsed and the DOM can be queried. The load event waits longer for images, stylesheets and fonts, which are often irrelevant. Pair domcontentloaded with a targeted selector wait on JavaScript-heavy pages, and avoid networkidle when background requests keep the network busy. Examples are in code0010 and code0011. Prefer a selector tied to the target content over a generic container, and use attached rather than visible only when visibility is not required.

Animations can delay clicks, scrolling and state changes. Inject CSS that removes transitions and animations, ideally before page load with addInitScript so every navigation inherits it; code0012 and code0013 show the single-page and multi-navigation forms. Set explicit timeouts so a broken page fails quickly instead of hanging a batch, following the pattern in code0014. A practical checklist is: headless on, Chromium only, domcontentloaded, one proof selector, animations off, tight timeouts, then measure again.

Cut traffic and round trips

Resource blocking usually gives the largest gain. Intercept requests, inspect their type, and abort images, fonts and media while allowing HTML, scripts and data calls through. Do not block JavaScript by default, because that commonly breaks rendering. The basic pattern is in code0015.

Trackers need a different filter: check script, xhr, fetch, ping and beacon requests, compare the host with the page host, and abort only known third-party domains. Keep the list short and remove the last addition if data disappears, as illustrated in code0016. Before choosing blocks, inspect the Network tab sorted by size, time and domain to separate heavy first-party assets from repeatable third-party noise. Observe first, then block deliberately.

For lists, repeated individual calls such as textContent or getAttribute create many browser round trips. A single evaluate call can collect related fields and return plain serialisable data, shown in code0019. For infinite scroll, scrollIntoViewIfNeeded moves only when an element is off-screen; combine it with a wait for new items, as in code0017 and code0018. If loading depends on global scroll events rather than element visibility, a controlled wheel or window scroll loop may still be needed. To skip a predictable non-critical request entirely, route.fulfill can return a minimal local response, with the safeguards in code0020: never fake authentication, pricing or other business-critical data.

Scale with contexts, then fix access

Parallel work should use several browser contexts inside one browser process rather than several browsers. Contexts isolate cookies and storage like separate incognito windows while sharing startup and memory costs. Start with three to five contexts, watch CPU, memory and error rates, and increase gradually; too much parallelism can itself slow the run or trigger throttling. The cleanup-safe pattern is in code0021.

Access problems also cost time through retries. Rotate a proxy per context or batch rather than per request, prefer residential or mobile proxies where protection is aggressive, and test proxy latency because a slow proxy can erase code gains; see code0022. Vary user agent and viewport per context using realistic, consistent values, as in code0023. Stealth patches such as the setup in code0024 may reduce obvious automation signals, but they do not fix poor waits, excessive rates or serious bot protection. Small randomised pauses between major actions, illustrated in code0025, are preferable to perfectly timed loops.

When to stop optimising locally: if proxy management, fingerprinting, retries and infrastructure dominate the work, the bottleneck has moved from script speed to access infrastructure. At that point, compare a managed scraping service against continuing with Playwright, and keep Playwright where direct control matters most.

Speed decision map
DecisionFaster defaultUse the alternative when
BrowserChromiumFirefox or WebKit is needed for engine-specific testing
Display modeHeadlessHeadful is needed to diagnose selectors or timing
Navigation waitdomcontentloaded plus a proof selectorA later event is required for the target data
TrafficBlock images, fonts, media and known trackersA blocked resource proves necessary for rendering data
ScalingMultiple contexts in one browserIsolation or resource limits require separate processes

Defaults are starting points; measure on the target site before changing them.

Frequently asked questions

Is Playwright good for web scraping?

Yes. It suits modern, JavaScript-heavy pages and sites that reject simple HTTP clients, because it drives a real browser for dynamic content, sign-in flows and interactive elements.

How do you make Playwright faster?

Wait less and download less: use domcontentloaded, block heavy assets, replace fixed sleeps with selector waits, extract data in bulk inside the page, and parallelise with browser contexts. Most slowdowns come from defaults rather than Playwright itself.

Should I use Chromium, Firefox, or WebKit for scraping?

Chromium is usually the best scraping choice: faster startup, predictable behaviour on current sites and fewer quirks. Firefox and WebKit remain useful for cross-engine testing, but are rarely faster or more stable for scraping.

How do I debug slow Playwright scripts?

Run headful with slow motion, then inspect navigation waits, selector waits and network traffic in DevTools. Slow runs usually wait for the wrong event or fetch resources they never use.