Running Puppeteer and Playwright on AWS Lambda
A browser script that runs fine on a laptop can deploy to Lambda without errors and still be slow and expensive. Each fresh execution environment starts with an empty /tmp, so a ZIP-packaged function spends seconds unpacking Chromium before it loads anything, and AWS bills that INIT phase. Bot-protected sites add a second problem: the browser reports headless signals and the request comes from an AWS address.
This guide walks through what the packaged Chromium actually contains, where cold-start time goes, which defaults to change, how to read the launch errors you are most likely to hit, and how to send only the pages that block your function to a managed scraping API. The measurements below come from Node.js 24 functions on x86_64 using @sparticuz/chromium 153.0.0, puppeteer-core 25.13.0, and playwright-core 1.64.0; timing tests ran in eu-north-1 and protected-site tests in us-east-1. Treat the numbers as a dated reference and re-measure on your own targets.
What the Lambda Chromium package actually ships
For Node.js, @sparticuz/chromium bundles a compressed Chromium build plus the launch flags it needs. It pairs with puppeteer-core and playwright-core, the variants of each library that do not download their own browser. The package's own source and build files show several details that shape behavior later in this guide.
The binary is chrome-headless-shell, a separate headless browser rather than full Chrome, and it identifies itself as HeadlessChrome. The archive chromium.br is 67 MB compressed and expands to a 209 MB executable in /tmp the first time executablePath() is called. Fonts are limited to three Open Sans files, so every generic family renders the same way. WebGL goes through SwiftShader, a CPU renderer, unless graphics are disabled. The default flag list includes --single-process and --disable-web-security, which can break browser contexts and switch off the same-origin policy respectively.
Expect a version gap between the packaged browser and the one your library was built for. Here the package carried Chromium 153, whereas puppeteer-core 25.13.0 is built for Chrome 155 and playwright-core 1.64.0 for Chromium 156. Neither library raised a version error while driving the older binary, yet pages can still behave differently, so test your own targets against it.
Where cold-start time and money go
For a ZIP-packaged function, most of the cold start is Chromium unpacking itself into /tmp. The INIT phase, where Lambda loads your code before the first request, is billed for ZIP functions alongside handler time, so every second of a browser cold start is billed. A REPORT line makes this visible: a 3,276.70 ms handler duration with a 1,052.19 ms INIT produced a 4,329 ms billed duration.
Across four cold starts at 2,048 MB, the phases broke down as follows: INIT took 0.89 to 1.09 s, executablePath() unpacking took 2.43 to 3.19 s, puppeteer.launch() took 0.12 to 0.18 s, the first page load took 0.24 to 0.28 s, and browser.close() took 0.15 to 0.37 s, for a billed total of 4.1 to 5.0 s.
Unpacking is CPU-bound, and Lambda allocates CPU in proportion to memory, with roughly one vCPU at 1,769 MB. On a large documentation page, a 1,024 MB function unpacked in 5.13 s and billed 11.8 s (11.84 GB-seconds); 1,769 MB unpacked in 3.70 s and billed 7.0 s (12.07 GB-seconds); 3,008 MB unpacked in 2.21 s and billed 5.2 s (15.20 GB-seconds). Moving from 1,024 MB to 1,769 MB made the cold start 1.7x faster for about 2% more GB-seconds; 3,008 MB was 2.3x faster than 1,024 MB but cost 28% more. Start at 1,769 MB and raise memory when latency matters more than cost. Where you spend effort on cold starts depends on traffic shape: fan-out bursts and on-demand requests feel it most, while a steady queue-driven batch keeps warm environments alive for most of its messages.
Cutting startup with a container image or SnapStart
Two changes attack the same bottleneck. Unpacking Chromium during the image build roughly halves the cold start and adds no per-invocation charge. SnapStart instead swaps INIT and the browser launch for a 0.49 to 0.97 s restore, which is billed as a restore charge. The reason both help is that /tmp begins empty in every new environment, so either approach takes the unpacking out of the request path.
At 2,048 MB, a ZIP package unpacked at runtime billed 4.1 to 5.0 s across four cold starts (INIT plus unpacking). A container image with Chromium unpacked at build time billed 2.1 to 2.2 s across four cold starts (INIT plus launch). A container image with SnapStart, launching the browser during INIT, restored in 0.49 to 0.97 s across eight restores, then billed 1.3 to 1.8 s for the first books.toscrape.com page, with a restore charge per GB and a snapshot cache charge.
Unpacking at build time adds no per-invocation charge: the image runs the package's unpacking once during docker build and copies the result to /opt, so a cold start goes straight to launch. SnapStart supports container images and snapshots memory and disk after INIT, so a browser started during INIT is restored already running. In all eight test restores the DevTools connection stayed open, but AWS does not guarantee that, so the handler below exits if the browser disconnects.
At the example rates on the Lambda pricing page, one 2 GB restore costs about $0.00028, more than a full 4.5 s ZIP cold start at about $0.00015. Caching one 2 GB version adds about $7.80 for a 30-day month, and Lambda also bills INIT at publish time and whenever it re-runs INIT to patch the snapshot. Add SnapStart when a faster startup is worth the restore charge.
Container image details that prevent crashes
The container image uses a package.json with @sparticuz/chromium 153.0.0 and puppeteer-core 25.13.0, and a Dockerfile that unpacks Chromium at build time and adds the label that allows SnapStart. Each detail below prevents a crash the Dockerfile hits without it:
- chmod -R a+rX. The binary lands with mode 0700, but container functions execute as a non-root user, so the permission change is what lets the process start; skip it and launch fails with spawn /opt/chrome/chromium EACCES.
- HOME=/tmp. The base image leaves HOME at /, which the non-root user cannot write to, so Chromium exits and Puppeteer reports Target closed.
- /tmp/*.json. Copying only the .so files leaves vk_swiftshader_icd.json missing, and Chromium fails with the same Target closed error.
- AWS_EXECUTION_ENV. The package only unpacks its Amazon Linux 2023 libraries when this variable names a supported Node.js runtime, and a plain docker build leaves it unset.
The Node.js 22 base image is a deliberate choice. When the Node.js 24 image was tested, it restored and then exited with Missing required environment variables: AWS_LAMBDA_LOG_GROUP_NAME, AWS_LAMBDA_LOG_STREAM_NAME. If you want a newer base image, first confirm that a single restore from it returns a page.
The handler launches the browser at module level. Every restored environment starts from the same snapshot, so anything the tab loads during INIT, such as cookies, appears in every environment; keep the tab blank during INIT. The tab keeps cookies between invocations, so this handler suits pages that may share state. Warm invocations reused the tab and billed 0.57 to 1.69 s for the books.toscrape.com page.
SnapStart applies only to published versions, so invoke the version number or an alias, not $LATEST. The --provenance=false flag keeps Docker from adding an attestation manifest, which Lambda rejects.
Using Google's official headless shell
The Chrome for Testing channel from Google ships chrome-headless-shell for Linux arm64 and x64, which lets you pick a browser that lines up with the Chrome version your puppeteer-core release targets. That official build depends on a handful of system libraries absent from the Lambda base image, and ldd reveals which ones. The Dockerfile below installs those libraries plus the arm64 build, keeps only puppeteer-core in package.json, and the handler starts the official binary with four flags: --no-sandbox, --no-zygote, --disable-dev-shm-usage, and --disable-gpu.
To deploy it, run the image commands above with a new repository and function name, linux/arm64 in --platform, arm64 in --architectures, and no --snap-start. For the x64 build, add at-spi2-atk to the dnf install list. On Graviton at 2,048 MB, cold starts billed 1.5 to 2.8 s, in the same range as the container image above. Choose the official build for a Chrome version that matches your library and for native arm64 support.
Graviton costs about 13% less per page. Arm64's GB-second price is 20% lower, and on the same documentation page its median warm time was only slightly slower, 1.07 s compared with 0.98 s on x86_64. For arm64, use @sparticuz/chromium-min with the arm64 layer or remote pack the README describes, or the official arm64 build above, because the tested npm package includes only x64 binaries.
When Puppeteer cannot launch Chromium on Lambda
These launch errors appear often in the package's issue tracker. The table shows the messages the tested versions printed, their causes, and the fixes:
- The input directory "/var/bin" does not exist: a bundler such as esbuild or webpack moved the code away from the package's bin folder. Mark @sparticuz/chromium as external, for example esbuild --external:@sparticuz/chromium.
- /tmp/chromium: cannot execute binary file, exit code 126: the npm package's x64 binary on an arm64 function; older Node.js versions report spawn Unknown system error -8. Deploy as x86_64, use @sparticuz/chromium-min with the arm64 pack, or use Google's official arm64 build.
- error while loading shared libraries: libnspr4.so or libnss3.so: the Amazon Linux 2023 libraries were not unpacked, as in a docker build without AWS_EXECUTION_ENV. Set AWS_EXECUTION_ENV and LD_LIBRARY_PATH as in the Dockerfile above.
- browser.close() hangs: close waits for work still running in the browser. Kill the browser process instead of closing it.
Inside a warm function: tabs, contexts, flags, and close
A warm invocation skips the unpacking, but coding patterns from desktop scripts still decide how many seconds each page costs. In a GitHub code search run for this guide, thousands of JavaScript and TypeScript files using @sparticuz/chromium called browser.close() and passed the default flags. Fewer than 1 in 30 mentioned SIGKILL, and about 1 in 8 mentioned setGraphicsMode. These timings come from three pages at 2,048 MB:
- Puppeteer newPage() after any tab in the browser was closed: 1.5 to 2.5 s.
- Playwright newContext() plus newPage() after a context was closed: 1.1 to 1.8 s.
- Launch a new browser and open its first tab: 0.17 to 0.21 s.
- page.goto() in a reused tab: 0.08 to 0.42 s, with one 4.5 s outlier.
- browser.close(): 0.14 to 2.1 s, and some calls hung until the timeout.
- browser.process().kill("SIGKILL"): 0 to 22 ms.
Reuse one tab and avoid opening new tabs in a long-lived browser; the slow newPage() happened even with another tab still open. Reach for isolation only when tasks genuinely must not share state. Relaunching the browser is quick because Chromium already sits unpacked in /tmp, yet the first navigation still took 1.1 to 1.2 s, roughly matching a fresh Playwright context. Either route adds about a second to each isolated task.
Remove --single-process. With the package's default flags, Puppeteer's first browser context failed with Target closed, and Playwright's second context could not open a page. Without the flag, Puppeteer opened three contexts and Playwright opened six, with no clear peak-memory difference over two runs each.
Disable the graphics stack unless a page needs WebGL. With the default SwiftShader flags, a warm browser waiting for each load event took 6.2 to 7.8 s for six pages, including an image-heavy listing; with chromium.setGraphicsMode = false it took 2.7 to 3.2 s. Waiting only for domcontentloaded narrowed the gap to 3.0 to 3.2 s (one 6.3 s outlier) versus 2.5 to 2.8 s, and both settings extracted the same elements.
Kill the process after extracting data. browser.close() waits for work still in progress; on a large news page it hung in 4 of 7 warm invocations, and each hang was billed for the full function timeout. With SIGKILL, 12 of 12 invocations on the same page finished. One hung invocation's log stops at the close call and its REPORT line bills the whole 120 s timeout.
Clear core dumps from /tmp. The unpacked browser leaves about 300 MB free in the default 512 MB /tmp. Issue #512 reported teardown crashes on nodejs24.x that wrote core dumps of hundreds of megabytes, which can fill the remaining space. Deleting core.* files takes one directory listing, and killing the process skips the teardown where the issue reported the crash.
Keep the same-origin policy on. With the default flags, a script on example.com read 51,274 characters from books.toscrape.com through fetch(). Without --disable-web-security, --allow-running-insecure-content, and --disable-site-isolation-trials, the call failed and pages still loaded normally. Scraped pages run their own JavaScript, so with the policy disabled a malicious page could read internal services when the function runs in a VPC.
What a protected site sees from Lambda
A protected site sees a headless browser on an AWS address, and both are common reasons scrapers get blocked. The default Lambda browser reports these signals to a fingerprint test page:
- User agent: HeadlessChrome/153.0.8010.0, and client hints also contain HeadlessChrome.
- navigator.webdriver: true, even when --enable-automation is absent.
- Fonts: monospace, serif, and sans-serif all 849 px wide, one font for every family.
- WebGL renderer: SwiftShader through ANGLE and Vulkan, a CPU renderer.
- Screen versus viewport: screen 800 x 600, viewport 1920 x 1080, unusual on a real display.
- window.chrome: undefined, while real Chrome defines it.
- Plugins and media devices: 0 and 0.
- Notifications: denied in one API, prompt in the other, while real browsers report the same state in both.
- CPU and time zone: 2 cores, UTC.
- Network origin: Amazon autonomous system numbers (AS16509, AS14618), reverse DNS in amazonaws.com.
A public detection page, bot.sannysoft.com, flagged the same values when the Lambda browser loaded it. Indeed's Cloudflare block page for the Lambda browser in us-east-1 showed the AWS IP address. The TLS fingerprint was identical from Lambda and from a residential connection, with the same JA4 value t13d1516h2_8daaf6152771_806a8c22fdea; the TCP handshake was different. Zardaxt, an open-source classifier, identified Lambda's handshake as Linux and flagged the connection as a likely relay such as a proxy, with a 0.97 score in 3 of 3 runs, even with a matching Linux user agent; the same user agents from the residential connection were not flagged in any run.
At the protocol level, one well-known automation check no longer triggers. A DevTools Protocol client used to trigger a getter on Error.stack when it serialized console output. With the Runtime domain enabled explicitly, the check triggered on Chrome 124 and did not trigger on Chrome 155 or on Lambda's Chromium 153. That removes one check, not every protocol-level signal.
Matching user-agent and client-hint values without the Headless token fixed only the user agent row; navigator.webdriver stayed true. For pages that block the Lambda browser on these signals, a managed scraping API fetches them with its own browsers and proxies.
Fingerprint or IP? A test on 14 protected sites
Both the fingerprint and the IP decided the outcome, and 5 of the 14 sites blocked every do-it-yourself setup. To separate the IP from the fingerprint, one list of public pages went through four browser setups and through a managed scraping API:
- Targets: 14 listing, search, and profile pages behind Cloudflare, DataDome, HUMAN, Akamai, Kasada, or Amazon's own bot protection, plus 2 unprotected pages as a baseline.
- Browser setups: the same Chromium 153 binary ran from an AWS address in us-east-1 and from a residential connection in India. The residential machine had 8 CPU cores, compared with 2 on Lambda. On each network the browser ran once with default settings and once with a consistent user agent, client hints, and Accept-Language header. Neither setup used a stealth plugin or a proxy service.
- API: Auto-Mode, which tries configurations from cheapest to most expensive within one request. Requests used mode=auto, plus country_code=us for the US-only Realtor.com page.
- Success: a page counted as loaded when it returned HTTP 200, or timed out after the content arrived, and contained the expected text in more than 5K characters with no challenge title. The same rule applied to every setup, including the API, and each site was tested three times.
The per-site results follow in the protected-site results table. Two results need context. Best Buy served a country selector to the residential connection, so its residential cells measure geography, not bot defense, and Realtor.com returned HTTP 429 to every browser setup. Both baseline pages loaded in every run, so each setup worked on unprotected pages.
The default fingerprint failed from both networks. Apart from Nike and the US-only Best Buy page, the default browser loaded nothing from either network. The IP mattered once the headers were consistent: the Indeed, Zillow, and StockX pages loaded in every residential run with consistent headers and in no AWS run. Each blocked attempt still bills Lambda time. With consistent headers, Lambda needed about three attempts for each loaded page, and with the defaults about seven. The residential column with consistent headers shows what one clean residential IP added to the same browser; that setup still loaded fewer than half of the attempts. Some sites blocked every do-it-yourself setup: Etsy, Home Depot, Crunchbase, Glassdoor, and Realtor.com did not load in any browser setup. The results show outcomes, but they do not show which signal each bot-protection vendor used.
The API's Auto-Mode loaded every site in the test, including those five. Sometimes the cheapest successful configuration did not use a browser: Auto-Mode tries configurations from cheapest to most expensive, stops at the first success, and charges only for that configuration. For Zillow, its cheapest configuration, a plain HTTP request, loaded a page that the Lambda browser did not load in any run. Blocks depended on both the network and what the browser reported about itself; handler changes help with the browser side, and the next section adds a route for the network side.
A fallback design that sends protected pages to a scraping API
Keep unprotected pages, internal tools, and screenshots or PDFs on your own Lambda browser, and send pages that block that browser to a web scraping API that runs the browsers and rotates proxies for you. The handler routes each URL like this: try the Lambda browser first; if the response status is 403, 429, or 503, or the page title matches a challenge pattern, remember the host and send it to the API; if navigation times out, send that URL to the API once without remembering the host.
The handler applies the browser changes from the sections above and routes hosts that block the browser to Auto-Mode. It uses CHROME_PATH when set, so the same file also runs in the container image from the start-faster section and skips the unpacking. With the two unprotected test pages, its cold starts billed 2.7 to 3.0 s in the image and about 4.5 s from the ZIP package. Block detection uses status codes and page titles, so add the challenge titles of your own targets to BLOCK_TITLE.
These details make sure one failed URL does not fail the whole batch:
- Timeouts: a navigation timeout sends that URL to the API once, without marking the host as blocked.
- API_CONCURRENCY sets the number of concurrent API calls. Match it to your plan's concurrency, which the pricing page lists for each plan.
- One deadline: all API calls share the invocation's remaining time minus 5 seconds, so the batch returns every result before the function times out.
- Retries: temporary responses get up to two more attempts, 2 and then 4 seconds apart, while enough time remains before the deadline.
- looksBlocked marks any result with a non-200 status or a challenge title.
The API key is stored in Parameter Store as a SecureString, because with the default encryption key anyone who can view the function's configuration can read an environment variable. The handler reads the key once per environment with the AWS SDK that the Node.js runtime includes. MAX_COST is optional and gives Auto-Mode a per-page credit budget; add it to the --environment variables with a value from the Auto-Mode documentation.
Deploying and running the fallback handler
Save the handler as index.mjs in the same folder as the package.json the container image uses. The ZIP package is about 78 MB, over the 50 MB limit for a direct upload, so the commands upload it through Amazon S3. Skip the first two commands, which create page-fetcher-role, if you already created it in the SnapStart section. Replace YOUR_API_KEY with the key from your account, and set YOUR_BUCKET and the account ID.
A new IAM role can take a few seconds to become available to Lambda, so if create-function reports that the role cannot be assumed, run that command again.
Save four URLs as event.json: two unprotected pages and two protected ones. The function's timeout is 300 seconds, so --cli-read-timeout 0 in the invoke command prevents the AWS CLI from stopping at its own default timeout of 60 seconds. The browser handled both unprotected pages, and the API returned both pages that blocked the browser. A separate request to the screenshot API for the same Indeed URL, with premium proxies and JavaScript rendering enabled, produced the screenshot shown in the source.
The same handler with Playwright
Playwright needs three changes, and the function deploys with the same commands. If the Puppeteer version is already deployed, skip the two role commands and put-parameter, and replace create-function with aws lambda update-function-code. Install playwright-core in the same project, replace the Puppeteer import with the playwright-core chromium import, and replace the handler export to use launchServer() instead of launch(). launchServer exposes the browser process, so the handler can kill it instead of waiting for close(). With the same event, Playwright sent two pages through the browser and two through the API.
Running it at scale
Keep the 2,048 MB function for Chromium and process blocked URLs from an SQS queue with a 256 MB function that only calls the API. One such arm64 function sent all 16 pages from the protected-site test to Auto-Mode in parallel, so the API step runs on one-eighth of the browser function's memory. Enable ReportBatchItemFailures and return the message IDs of failed results, so only those messages go back to the queue. Set the event source mapping's maximum concurrency so that the number of running functions multiplied by API_CONCURRENCY matches your plan's concurrency.
Set the queue's visibility timeout to at least six times the function timeout, as AWS recommends, with a dead-letter queue after five failed receives. Write each page to S3 under a hash of its URL, because a synchronous response is limited to 6 MB. Skip URLs that are already stored, because standard queues can deliver a message twice. Keep apiHosts in a shared table such as DynamoDB, with a time-to-live on each entry so a host gets another browser attempt later.
The JSON line the handler logs for each URL shows which hosts need the API. For spend, track cost per successful page, which is total Lambda and API spend divided by the pages you keep. That number shows when sending a host to the API costs less than retrying it in the browser.
Running it on Lambda Managed Instances
Lambda Managed Instances run functions on EC2 capacity in your account, and one environment serves many requests at once. For Node.js, the documented default is 64 concurrent requests per vCPU, all sharing /tmp. In the test, six simultaneous requests each loaded two unprotected pages in the fallback handler's browser, on a 2,048 MB function with one vCPU per environment. A first burst lands on new environments, where /tmp is empty:
- First burst, without chromiumPath: 3 of 6 Chromium launches failed with spawn ETXTBSY; 1.2 to 4.6 s per request, including the failed requests.
- Later bursts, without chromiumPath: 12 of 12 succeeded; 2.5 to 11.9 s.
- First burst, with chromiumPath: 6 of 6 succeeded; 4.4 to 14.6 s.
- Later burst, with chromiumPath: 6 of 6 succeeded; 3.1 to 10.4 s.
Unpack Chromium only once. The failures came from requests that launched /tmp/chromium while another request was still writing it. The chromiumPath variable makes concurrent requests in one Node.js worker wait for the same unpacking step. Node.js runs one worker per vCPU, and each worker has its own chromiumPath, so keep each environment at one vCPU, as in this test.
Limit concurrency to what the environment can render. Set PerExecutionEnvironmentMaxConcurrency to the number of browsers an environment can run. Pages routed to the API leave that browser capacity free, because the API renders them with its own browsers. Managed Instances also need a VPC with outbound access and a published version.
What to return when an LLM reads the page
Many scraped pages are sent to a model's context window, and raw HTML is usually the most expensive format to send there. Counted with the o200k_base tokenizer, four protected pages fetched through the API show the difference in the token comparison table: Indeed search went from 631,443 HTML tokens to 13,202 Markdown tokens and 152 AI-extracted field tokens; Zillow city page from 240,898 to 3,698 and 160; Walmart search from 443,512 to 9,214 and 324; Amazon search from 309,194 to 53,965 and 317. Totals were 1,625,047 HTML tokens, 80,079 Markdown tokens, and 953 AI-extracted field tokens.
For Indeed, the Markdown output is 48x smaller than the HTML, and the HTML alone would not fit in a 200K-token context window. The return_page_markdown=true parameter returns Markdown, and the ai_extract_rules parameter returns only the fields you describe.
The fallback handler takes a markdown flag and passes it only to the API path. With the markdown event, the API returned Zillow as 9,761 characters of Markdown, compared with the 663,855 characters of HTML it returned in the deploy step. The handler reads title from the HTML title tag, so that field stays empty for Markdown results.
The browser path still returns HTML, so convert it with a library such as Turndown before sending it to a model. When an AI agent uses the output, a remote MCP server exposes the same API as Model Context Protocol tools the agent calls directly.
Lightpanda: a lightweight browser for pages that need no rendering
Lightpanda is an open-source browser written in Zig that supports the DevTools Protocol, so Puppeteer connects to it with puppeteer.connect(). These tests used version 1.0.0. Lightpanda skips graphical rendering, which makes it suitable for data extraction, while Chromium remains the better choice for pixel-accurate screenshots and PDFs.
Lightpanda's 188 MB binary fits in a ZIP-packaged function, and Lambda extracts the package before INIT, so nothing unpacks at runtime. Both browsers loaded the same six pages, including one page that JavaScript renders, in one function at 2,048 MB:
- Cold start with one page, billed: Chromium 3.1 to 3.9 s; Lightpanda 0.92 to 0.98 s.
- Six pages in a warm function, billed: Chromium 7.3 to 7.7 s; Lightpanda 2.2 to 2.4 s.
- Max memory used, six pages: Chromium 635 to 641 MB; Lightpanda 183 to 193 MB.
Both browsers extracted the same elements from all six pages. In a separate check, Indeed returned 403 to both browsers, so protected pages should still go to the API path. At 512 MB, Lightpanda ran the six pages in 3.4 to 4.2 s, for about 8x fewer GB-seconds than Chromium at 2,048 MB.
The tested binary runs on Lambda with one extra step. It needs glibc 2.35, while Amazon Linux 2023, which Lambda's Node.js runtimes use, includes 2.34, so the downloaded binary exits with version `GLIBC_2.35' not found. If a later release starts without that error, skip the bundling. The commands download it and bundle a newer loader and libraries from Debian, and the handler starts Lightpanda through that loader and connects Puppeteer.
The ZIP package is about 60 MB, so it deploys through S3 with the fallback handler's commands and a new function name. Check each target's output before switching. When a page's data is already in its HTML or available from one API call, skip the browser entirely.
A decision checklist before you ship
Use this checklist to map each symptom to one concrete change:
- Cold start over 4 s on a ZIP package: unpack Chromium at build time in a container image, or add SnapStart if a faster restore is worth the charge.
- Slow unpacking at low memory: raise memory to at least 1,769 MB, where Lambda gives roughly one vCPU.
- browser.close() hangs or bills the full timeout: kill the process with SIGKILL after extracting data.
- New tabs take seconds: reuse one tab, and isolate tasks only when they must not share state.
- Page loads are slow with default flags: set chromium.setGraphicsMode = false unless the page needs WebGL, and remove --single-process.
- A page returns 403, 429, or 503, or a challenge title: remember the host and route it to the API.
- A navigation times out: retry that URL through the API once, without marking the host as blocked.
- HTML is too large for a model context: request Markdown or AI-extracted fields from the API path.
- Concurrent requests fail with spawn ETXTBSY on Managed Instances: share one chromiumPath promise per worker and keep each environment at one vCPU.
A worked example using the captured four-URL batch: books.toscrape.com and its page-2 URL returned HTTP 200 through the Lambda browser with titles and lengths around 51 KB. The Indeed jobs page and the Zillow Seattle page were routed to the API and returned HTTP 200 with lengths of 1,723,540 and 663,855 characters. With markdown=true, the Zillow result came back as 9,761 characters of Markdown and an empty title field, because the handler reads title from the HTML title tag.
Final thoughts
Puppeteer and Playwright on Lambda drive @sparticuz/chromium, a specially packaged headless shell whose unpacking, tab lifecycle, and default flags decide most of its cost and behavior. On protected sites, even the best browser setup on Lambda's AWS address loaded 13 of 42 attempts in the 14-site test.
Unpack Chromium at build time, choose memory for CPU speed, reuse one tab, kill the process, and disable the default flags that slow pages, break contexts, or weaken security. For pages that still block the Lambda browser, Auto-Mode loaded all 42 test attempts, trying configurations from cheapest to most expensive and charging only for the one that succeeded. Route the pages that block your function through the fallback handler, and request Markdown when an LLM reads the result.
| Part | What the repository shows | Why it matters |
|---|---|---|
| Browser build | The build imports Chromium's headless.gn config, so the binary is chrome-headless-shell, a separate headless browser, not full Chrome | It reports itself as HeadlessChrome |
| Packaging | chromium.br is a 67 MB Brotli archive that unpacks to a 209 MB executable in /tmp on the first call to executablePath() | That unpacking was the largest part of each measured cold start |
| Fonts | The font archive holds 3 Open Sans files | Every generic font family renders in Open Sans |
| Graphics | WebGL runs through SwiftShader, a CPU renderer, unless you disable graphics | Sites can read SwiftShader in the WebGL renderer string |
| Default flags | The list includes --single-process and --disable-web-security | The first can break browser contexts, and the second disables the same-origin policy |
Version 153.0.0 of @sparticuz/chromium; re-check the package source for later versions.
| Phase | Time |
|---|---|
| INIT (Node.js loads Puppeteer, Playwright, and the package) | 0.89 to 1.09 s |
| executablePath() unpacks Chromium into /tmp | 2.43 to 3.19 s |
| puppeteer.launch() | 0.12 to 0.18 s |
| First page (example.com) | 0.24 to 0.28 s |
| browser.close() | 0.15 to 0.37 s |
| Billed total | 4.1 to 5.0 s |
Four cold starts at 2,048 MB on Node.js 24 x86_64 in eu-north-1.
| Packaging | Billed cold start at 2,048 MB | What you pay for |
|---|---|---|
| ZIP package, unpacked at runtime | 4.1 to 5.0 s (4 cold starts) | INIT plus the unpacking |
| Container image, Chromium unpacked at build time | 2.1 to 2.2 s (4 cold starts) | INIT plus launch |
| Container image with SnapStart, browser launched during INIT | Restore 0.49 to 0.97 s (8 restores), then 1.3 to 1.8 s billed for the first books.toscrape.com page | A restore charge per GB and a snapshot cache charge |
The first two rows loaded the same example.com page.
| Error | Cause | Fix |
|---|---|---|
| The input directory "/var/bin" does not exist | A bundler such as esbuild or webpack moved the code away from the package's bin folder | Mark @sparticuz/chromium as external, as the package README shows, for example esbuild --external:@sparticuz/chromium |
| /tmp/chromium: cannot execute binary file, exit code 126 | The npm package's x64 binary on an arm64 function. Older Node.js versions report spawn Unknown system error -8 | Deploy as x86_64, use @sparticuz/chromium-min with the arm64 pack, or use Google's official arm64 build above |
| error while loading shared libraries: libnspr4.so or libnss3.so | The Amazon Linux 2023 libraries weren't unpacked, as in a docker build without AWS_EXECUTION_ENV | Set AWS_EXECUTION_ENV and LD_LIBRARY_PATH as in the Dockerfile above |
| browser.close() hangs | Close waits for work still running in the browser | Kill the browser process instead of closing it |
Messages printed by the tested versions.
| Operation | Measured time |
|---|---|
| Puppeteer newPage() after any tab in the browser was closed | 1.5 to 2.5 s |
| Playwright newContext() plus newPage() after a context was closed | 1.1 to 1.8 s |
| Launch a new browser and open its first tab | 0.17 to 0.21 s |
| page.goto() in a reused tab | 0.08 to 0.42 s, with one 4.5 s outlier |
| browser.close() | 0.14 to 2.1 s, and some calls hung until the timeout |
| browser.process().kill("SIGKILL") | 0 to 22 ms |
Three pages at 2,048 MB.
| Signal | Lambda default | Note |
|---|---|---|
| User agent | HeadlessChrome/153.0.8010.0 | Client hints also contain HeadlessChrome |
| navigator.webdriver | true | Still true when --enable-automation is absent |
| Fonts | Monospace, serif, and sans-serif all 849 px wide | One font for every family |
| WebGL renderer | SwiftShader through ANGLE and Vulkan | A CPU renderer |
| Screen versus viewport | Screen 800 x 600, viewport 1920 x 1080 | Unusual on a real display |
| window.chrome | undefined | Real Chrome defines it |
| Plugins and media devices | 0 and 0 | |
| Notifications | denied in one API, prompt in the other | Real browsers report the same state in both |
| CPU and time zone | 2 cores, UTC | |
| Network origin | Amazon autonomous system numbers (AS16509, AS14618), reverse DNS in amazonaws.com |
Reported to a fingerprint test page by the default Lambda browser.
| Burst | Result | Time per request |
|---|---|---|
| First burst, without chromiumPath | 3 of 6 Chromium launches failed with spawn ETXTBSY | 1.2 to 4.6 s, including the failed requests |
| Later bursts, without chromiumPath | 12 of 12 succeeded | 2.5 to 11.9 s |
| First burst, with chromiumPath | 6 of 6 succeeded | 4.4 to 14.6 s |
| Later burst, with chromiumPath | 6 of 6 succeeded | 3.1 to 10.4 s |
Six simultaneous requests each loading two unprotected pages on a 2,048 MB function with one vCPU per environment.
| Measurement | Chromium | Lightpanda |
|---|---|---|
| Cold start with one page, billed | 3.1 to 3.9 s | 0.92 to 0.98 s |
| 6 pages in a warm function, billed | 7.3 to 7.7 s | 2.2 to 2.4 s |
| Max memory used, 6 pages | 635 to 641 MB | 183 to 193 MB |
Lightpanda 1.0.0; both browsers loaded the same six pages, including one JavaScript-rendered page, in one function at 2,048 MB.
Frequently asked questions
Can a website detect a headless browser running on AWS Lambda?
Yes. In testing, the default Lambda Chromium from @sparticuz/chromium reported a HeadlessChrome user agent, navigator.webdriver set to true, one font family, a SwiftShader WebGL renderer, and an Amazon IP address. With those defaults it loaded 6 of 42 page attempts on 14 sites with bot protection, while Auto-Mode loaded all 42.
What are the disadvantages of using AWS Lambda for web scraping?
Each new execution environment starts with an empty /tmp, so a browser in a ZIP package unpacks Chromium on every cold start. AWS bills the INIT phase as well, so every cold start is billed time. With a consistent user agent and headers, a browser loaded 13 of 42 protected page attempts from AWS, while Auto-Mode loaded all 42.
Can you run headless Chrome on AWS Lambda?
Yes. Use a Chromium build that runs on Lambda, such as the @sparticuz/chromium npm package or Google's official headless shell in a container image, and drive it with puppeteer-core or playwright-core. Start at 1,769 MB or more, the equivalent of one vCPU. At 2,048 MB, a cold start with the ZIP package billed 4.1 to 5.0 seconds.
Does Playwright work on AWS Lambda?
Yes. In testing, playwright-core launched the @sparticuz/chromium binary on Lambda once the --single-process flag was removed from the package's defaults. With that flag, Playwright's second browser context could not open a page. Without it, six consecutive contexts worked on the Node.js 24 runtime.
Is chrome-aws-lambda still maintained?
No. The original chrome-aws-lambda package has not published a new version since 10.1.0. Its successor fork, @sparticuz/chrome-aws-lambda, is deprecated on npm, and its deprecation message directs users to @sparticuz/chromium. The @sparticuz/chromium package provides Chromium builds made for Lambda, with Lambda layers for x64 and arm64.
How much memory does Puppeteer need on Lambda?
Start at 1,769 MB, where Lambda gives the equivalent of one vCPU, because CPU sets the unpacking speed. Unpacking Chromium took 5.1 seconds at 1,024 MB and 3.7 seconds at 1,769 MB, for 2% more GB-seconds per cold start. At 3,008 MB, the cold start was faster again but cost 28% more.
How do you make Puppeteer start faster on Lambda?
Unpack Chromium at build time in a container image. At 2,048 MB, the billed cold start was 4.1 to 5.0 seconds with the ZIP package and 2.1 to 2.2 seconds with Chromium unpacked at build time. SnapStart restored a browser launched during INIT in 0.49 to 0.97 seconds, and each restore adds a charge.
{
"type": "module",
"dependencies": {
"@sparticuz/chromium": "153.0.0",
"puppeteer-core": "25.13.0"
}
}
{
"type": "module",
"dependencies": {
"@sparticuz/chromium": "153.0.0",
"puppeteer-core": "25.13.0"
}
}
{"urls": ["https://books.toscrape.com/", "https://books.toscrape.com/catalogue/page-2.html", "https://www.indeed.com/q-python-developer-jobs.html", "https://www.zillow.com/seattle-wa/"]}
[{"url":"https://books.toscrape.com/","via":"browser","status":200,"title":"All products | Books to Scrape - Sandbox","length":51275,"looksBlocked":false},{"url":"https://books.toscrape.com/catalogue/page-2.html","via":"browser","status":200,"title":"All products | Books to Scrape - Sandbox","length":50847,"looksBlocked":false},{"url":"https://www.indeed.com/q-python-developer-jobs.html","via":"scrapingbee","status":200,"title":"Now Hiring: 7,000 Python Developer Jobs | Indeed","length":1723540,"looksBlocked":false},{"url":"https://www.zillow.com/seattle-wa/","via":"scrapingbee","status":200,"title":"Seattle WA Real Estate - Seattle WA Homes For Sale | Zillow","length":663855,"looksBlocked":false}]
{"urls": ["https://books.toscrape.com/", "https://www.zillow.com/seattle-wa/"], "markdown": true}
[{"url":"https://books.toscrape.com/","via":"browser","status":200,"title":"All products | Books to Scrape - Sandbox","length":51275,"looksBlocked":false},{"url":"https://www.zillow.com/seattle-wa/","via":"scrapingbee","status":200,"title":"","length":9761,"looksBlocked":false}]
curl -sL -o lightpanda https://github.com/lightpanda-io/browser/releases/download/1.0.0/lightpanda-x86_64-linux
chmod 755 lightpanda
mkdir -p glibc
docker run --rm --platform linux/amd64 -v "$PWD/glibc":/out debian:bookworm-slim \
cp -L /lib64/ld-linux-x86-64.so.2 /lib/x86_64-linux-gnu/libc.so.6 /lib/x86_64-linux-gnu/libm.so.6 /out/
npm install --save-exact [email protected]
zip -qr function.zip index.mjs package.json lightpanda glibc node_modules
| Memory | Unpack time | Billed cold start | GB-seconds per cold start |
|---|---|---|---|
| 1,024 MB | 5.13 s | 11.8 s | 11.84 |
| 1,769 MB | 3.70 s | 7.0 s | 12.07 |
| 3,008 MB | 2.21 s | 5.2 s | 15.20 |
A large documentation page; re-measure on your own targets.
| Site | Auto-Mode | AWS IP, default settings | AWS IP, consistent headers | Residential IP, default settings | Residential IP, consistent headers |
|---|---|---|---|---|---|
| Nike | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 |
| Best Buy | 3/3 | 3/3 | 3/3 | 0/3 | 0/3 |
| Walmart | 3/3 | 0/3 | 3/3 | 0/3 | 3/3 |
| Amazon | 3/3 | 0/3 | 3/3 | 0/3 | 3/3 |
| Indeed | 3/3 | 0/3 | 0/3 | 0/3 | 3/3 |
| Zillow | 3/3 | 0/3 | 0/3 | 0/3 | 3/3 |
| StockX | 3/3 | 0/3 | 0/3 | 0/3 | 3/3 |
| Tripadvisor | 3/3 | 0/3 | 1/3 | 0/3 | 1/3 |
| Leboncoin | 3/3 | 0/3 | 0/3 | 0/3 | 1/3 |
| Etsy | 3/3 | 0/3 | 0/3 | 0/3 | 0/3 |
| Home Depot | 3/3 | 0/3 | 0/3 | 0/3 | 0/3 |
| Crunchbase | 3/3 | 0/3 | 0/3 | 0/3 | 0/3 |
| Glassdoor | 3/3 | 0/3 | 0/3 | 0/3 | 0/3 |
| Realtor.com | 3/3 | 0/3 | 0/3 | 0/3 | 0/3 |
| Total | 42/42 | 6/42 | 13/42 | 3/42 | 20/42 |
Each site tested three times; success required HTTP 200 or a timeout after content arrived, expected text in more than 5K characters, and no challenge title. Best Buy's residential cells measure geography, not bot defense, and Realtor.com returned HTTP 429 to every browser setup.
| Page | HTML tokens | Markdown tokens | AI-extracted fields, tokens |
|---|---|---|---|
| Indeed search | 631,443 | 13,202 | 152 |
| Zillow city page | 240,898 | 3,698 | 160 |
| Walmart search | 443,512 | 9,214 | 324 |
| Amazon search | 309,194 | 53,965 | 317 |
| Total | 1,625,047 | 80,079 | 953 |
Counted with the o200k_base tokenizer on four protected pages fetched through the API.