Skip to content

How to scrape Home Depot product data with JavaScript rendering

Home Depot product pages load much of their content dynamically and may apply anti-bot checks. This guide shows a practical workflow: obtain an API key, enable JavaScript rendering, target a search or category URL, extract structured product fields with CSS selectors, and handle pagination when results span multiple pages.

Set up the request: API key and JavaScript rendering

Create an account, open the dashboard and copy the API key shown in the upper-right area. Keep the key out of public repositories and out of shared code. A simple HTTP request is not enough for Home Depot because product details are injected after JavaScript runs. Enable JavaScript rendering in the request so the page is rendered before parsing. The protected request module code0009 shows the parameter set used for the example, including render_js, extract_rules, js_scenario, premium_proxy and country_code.

Target the right URL and extract structured fields

Browse the site manually first so you understand how search, category and product URLs differ. The example search URL is preserved in protected module code0001. For a search such as drilling machines, spaces are converted to plus signs before the query is appended to the search path, as shown in protected module code0008. Define extract_rules so the parser targets one product card at a time and then reads fields inside it. In the example, the captured fields are brand_name, product_name, current_price, model, rating, reviews, link and product_id, plus a total_results label. The full selector map is preserved in protected module code0006; treat those selectors as brittle and re-check them when the page layout changes. Add a js_scenario when results load on scroll: the example in protected module code0007 combines infinite_scroll with a short wait, with a one-second delay between scrolls and a two-second pause after scrolling ends.

Validate the response and know what you can capture

Check the response before parsing: an invalid-key message should trigger a clear error pointing to the dashboard API-key page, and an empty product list should be reported as a failed retrieval rather than silently accepted. That handling is preserved in protected module code0010. A curl alternative for a basic rendered GET request is preserved in protected module code0011, using the configured {{OWNER_API_BASE_URL}} origin; it does not include the additional extract rules. Useful data beyond the example fields includes titles, descriptions, discounts, availability states, images, videos, SKU and model numbers, specifications, related products and store availability.

Choose DIY or API, then paginate with the Nao offset

A DIY setup means operating proxies, headless browsers, CAPTCHA handling and IP rotation yourself, which gives control but adds ongoing maintenance. An API-based setup moves that infrastructure work to the provider so the script focuses on URLs, selectors and output. Choose DIY only if you already have reliable infrastructure and need custom control; choose the API route if you need rendered pages and proxy handling quickly. For multi-page searches, use the Nao offset: the second page example is preserved in protected module code0013, the offset calculation in code0014, and the paginated request construction in code0015. In the example, each page advances by 24 results, so page zero starts at 0, page one at 24, and so on. Stop when a page returns no products, and record the offset with each batch so results can be deduplicated later.

Fields captured in the example extract rules
FieldWhat it representsExample use
brand_nameBrand shown on the product cardBrand-level price comparison
product_nameProduct titleCatalogue matching
current_priceDisplayed current pricePrice tracking
modelModel identifier textVariant matching
ratingRating label from the rating elementQuality filtering
reviewsReview count textPopularity ranking
linkProduct page URLFollow-up detail scrape
product_idProduct identifier attributeDeduplication key

Field names follow the protected extract_rules module code0006; selectors themselves are not reproduced here.

Frequently asked questions

Is scraping Home Depot allowed?

Personal use is described as generally permitted, but Home Depot's Terms of Service should be reviewed. For commercial use, check robots.txt and rate limits, and keep request frequency respectful.

Can I scrape product prices and images?

Yes. Prices, images and other product details can be extracted when JavaScript rendering is enabled, so content that loads after the initial render is available for parsing.

How often can I scrape Home Depot?

Frequency depends on the plan in use. Request spacing and proxy rotation are handled automatically to reduce detection risk; larger runs may require a higher-limit plan, to be confirmed by the owner.

Does Home Depot use JavaScript to load content?

Yes. Product information, prices and availability rely heavily on JavaScript. Enabling render_js=true is presented as essential so dynamically loaded content is captured before extraction.

curl -X GET "{{APP_BASE_URL}}/api/v1/?api_key=YOUR_API_KEY&url=https://www.homedepot.com/s/power%20tools&render_js=true"