Scrape Amazon product data with Python
Amazon product pages are a moving target: selectors change, anti-bot rules shift, and the same price can appear in three different places. A structured product data API replaces that fragility with a single request that returns titles, prices, ratings, images, specifications, stock, and delivery windows as JSON. This guide walks through a Python workflow that starts from an ASIN (or a keyword search) and ends with a clean export you can hand to analysts.
Quick walkthrough
The whole pipeline fits into six steps:
- Create a small Python project with
requestsandpython-dotenv, and load your API key from a.envfile. - Get your ASINs one of two ways: send keywords to the Amazon Search endpoint and collect the ASINs it returns, or go straight ahead if you already have the identifiers on hand.
- For each ASIN, call the Amazon Product endpoint to fetch structured JSON instead of scraping HTML.
- Extract the fields you need and shape them into a clean dictionary or row.
- Export the results to JSON, CSV, Google Sheets, or a database.
- When scaling, add retries, rate limiting, and logging so the pipeline stays reliable.
Choose your approach: raw HTML vs product data API
There are two roads. Raw HTML scraping means sending requests, spoofing headers, rotating proxies, and parsing the page with BeautifulSoup or Parsel. It can work, but you will spend time on 503s, CAPTCHAs, throttling, and markup that mutates faster than your selectors. A product data API takes care of proxies, headers, rendering, and retries. You pass an ASIN and get structured JSON back. No parsing, no selector hunting, and the workflow stays stable when Amazon reshuffles its HTML.
Decision checklist:
- Do you need a stable field contract across thousands of ASINs? → API.
- Do you need full review text at volume? → dedicated review scraper, not the product endpoint alone.
- Do you need 3D/360 viewers reconstructed? → neither approach gives a clean field; detect and link only.
- Do you need a one-off page snapshot? → raw HTML or a screenshot API is fine.
Prerequisites and environment setup
You need Python 3.10+, a virtual environment (the walkthrough uses uv), the requests library, python-dotenv for loading secrets, basic comfort with JSON, and an API key. The free tier gives you {{OWNER_FREE_TIER_CREDITS}}, which is enough to experiment.
Project setup commands and the .env loading snippet are provided as exact code modules (code0001, code0002, code0003). Keep the key in .env, never in source.
First test call to Amazon
A good test target is the LEGO Rivendell set, ASIN B0BNWCWG7L. The minimal request uses the Amazon Product endpoint with query set to the ASIN, light_request set to True for a faster response, and domain set to com. The exact request code is in code0004 (private source) and the run command is code0005.
A successful call returns a large JSON blob with product name, price, rating, images, product details, stock, delivery, and sometimes review snippets. A trimmed preview is in code0006. No headless browser, no CAPTCHA, no proxy gymnastics.
Get the product ASIN
ASIN stands for Amazon Standard Identification Number. It is the anchor that stays stable while prices shift, titles get edited, or sellers swap images. There are two common ways to get it.
From a product URL: most Amazon URLs contain /dp/<ASIN> or /gp/product/<ASIN>. A small regex extracts it (code0007). This works for the vast majority of product URLs.
From a keyword search: if you start from keywords like "lego castle" or "noise cancelling headphones", use the Amazon Search endpoint to get a product list, then pull ASINs from the results (code0008 and code0009). light_request=true keeps the search fast and cheap because it skips a full browser.
The product endpoint repeats both asin and parent_asin in the response, which is handy for grouping variations (code0010).
Extract product name
The product endpoint returns a clean product_name value. On the raw page the title sits under #productTitle, but that selector is a classic HTML scraping target that breaks when layouts change. The API route gives you the same field every time. The request and extraction snippet are in code0011 (private source).
Retrieve product description
On the page, the description lives under #productDescription, sometimes in nested spans and sometimes in unpredictable UL/LI structures. The API avoids that. Before fetching the description, upgrade the request params so the API returns data in the language, country, and currency you want: country, language, and currency (code0012, private source).
The product data API gives you three description-related fields:
description— the main long-form text block.product_overview— short structured details when available.bullet_points— the feature bullets near the top of the page.
A helper snippet merges all three into one clean text blob (code0013).
Collect product specifications
Specs are usually the swamp of Amazon scraping. On the page they are scattered across tables, sidebars, and product-details blocks. With the API you read structured fields: product_details, sometimes product_overview, and top-level fields like brand, manufacturer, and product_dimensions (code0014).
For analytics or warehouse loading, flatten everything into one spec dictionary so you can store it, turn it into rows, or dump it to CSV without juggling nested structures (code0015).
Get product rating
Ratings on the raw page use star SVGs, aria-label text, nested spans, and layouts that shift by category. The API gives you clean numeric fields: rating, rating_stars_distribution (a list of rating/percentage pairs), and reviews (code0016).
Extraction code is in code0017. For long-term tracking, flatten into a compact metrics dictionary with asin, rating, five_star_pct, one_star_pct, and review_count (code0018).
Collect product reviews
Reviews come in two flavors, and mixing them up creates messy pipelines.
Summary data: average rating, star distribution, review count. Use this for rankings, snapshots, and dashboards.
Full content: review text, titles, timestamps, verified purchase flags, helpful votes, author info. Use this for sentiment analysis, topic extraction, duplicate detection, and text processing.
Option 1 — product endpoint with light mode off. The product endpoint can return review data, but light_request=True sometimes trims deeper fields. If reviews are missing or skinny, set light_request to False so a real browser is used (code0019).
Option 2 — dedicated reviews scraper. For full text, metadata, timestamps, verified status, and helpful votes at volume, use a dedicated review scraper built for pagination and structured records. This is where projects drift into NLP, long-term monitoring, fraud detection, or approximating sales data from review velocity and rating swings.
A practical rule: start with summary metrics (stable and cheap) and fetch full review text only when you need deeper analysis.
Product price retrieval
Price is one of the messiest parts of HTML scraping. Amazon switches price formats depending on seller, Prime status, coupons, variations, stock, and your browsing location. In markup that becomes scattered spans, duplicated numbers, and the eternal question of which one is the real price.
The API normalizes this. Key fields:
priceandcurrency— the basic current price signal.price_buybox— the Buy Box price, usually the one you care about.price_shipping— shipping cost if it exists.pricing_url— the offer listing page, useful for digging into sellers.
A snippet converts price-ish values to Decimal, handles FREE and comma/dot formats, calculates a final price, and flags free shipping (code0020).
Product image extraction
On the raw page, images involve #landingImage, carousel logic, hidden JSON, hover swaps, and variant-specific images. The API includes an images array, and that is the whole story (code0021). Most workflows keep the URLs rather than downloading local copies because it is faster, cheaper, and scales better in bulk. If you need a full-page screenshot for a catalog, QA check, or visual monitoring, a screenshot API is a simple add-on.
Video URLs and 360-degree views
Titles, prices, ratings, images, and specs come through clean. Videos and 360-degree or 3D product views are trickier because Amazon loads them via JavaScript, separate media endpoints, and dynamic players. They are not guaranteed in every response, even with full rendering.
First move: turn off light mode so a real browser session runs (code0022). If the product response includes direct media fields, store the URLs and use them for playback, catalog previews, or QA checks.
Reality check: Amazon almost never exposes 3D or 360-degree viewers as a clean JSON field. Those views are loaded through JavaScript, stitched from multiple files, and often depend on internal viewers. Even with raw HTML you would hit a mess of masks, tiles, and segment files. To try, set add_html to True (code0023) and search for the viewer entry point, commonly data-src="/view-3d?asin=ASIN_HERE" (code0024). A simple regex scan is in code0025. If you find a viewer URL, save it as metadata or open it in a browser and inspect the Network tab. Rebuilding the 3D experience yourself is a project on its own. The practical rule: use the API for structured data and treat 3D/360 assets as optional extras you detect and link to when they exist.
Extract product brand and category
Brand and category become important once you build datasets. They let you slice search data into segments like "all LEGO sets" or "everything in Electronics → Headphones". With the API, brand is usually a top-level field like brand, and many listings also include manufacturer (code0026).
Category can show up as an empty list, a list of strings, or a ladder of objects representing the full path through Amazon's taxonomy. When the ladder is available, save all levels or just the leaf category (code0027).
A subset of listings also carries ranking signals such as sales_rank or best_sellers_rank; presence is not guaranteed. When they do show up, you can plot them over time within the same category ladder, which helps with competitive monitoring and offers a lightweight stand-in for Amazon product sales data when real sales numbers are out of reach (code0028).
Export product availability and stock status
Availability punches above its weight. Even without real sales numbers, watching stock flips and price swings gives a good feel for what is moving. In the API response, the stock story shows up in a few spots:
buybox[0]["stock"]— the human-readable stock label ("In Stock", "Only 2 left", "Currently unavailable").max_quantity— the highest quantity Amazon will let you add to cart, useful as a soft low-stock indicator.- Buy Box info — the whole offer context: seller name, condition, price, returns policy.
A snippet turns these into simple status labels (code0029). Once you log labels over time, patterns appear: products bouncing between in stock and out of stock, max quantity suddenly dropping, Buy Box switching sellers overnight, variations running out one by one.
Get product delivery details
On the raw page, delivery dates can appear in multiple spots, shift when you change ZIP codes, or hide behind dynamic widgets. The API returns a clean delivery array (code0030). You can normalize it into a format that plays nicely with dashboards or databases (code0031).
Keeping delivery records normalized pays off once you operate at scale: you can follow how promised delivery windows shift over time, compare fulfillment speed across sellers, and watch whether Prime delivery stays consistent between regions and product types.
Extract product details from Amazon search
If you start from keywords, the Search endpoint is the fastest way to get moving. It returns multiple products in one call with ASINs, titles, prices, ratings, review counts, and flags like Prime or sponsorship (code0032 and code0033). That becomes your discovery layer.
The standard flow:
- Search by keyword → get a list of ASINs.
- Loop through those ASINs → call the Product endpoint for full details.
A minimal Python example that does exactly that is in code0034 (private source). This two-step pipeline is the most practical way to combine search data with full product details while staying API-first, stable, and scalable.
Export to JSON
Since the API already hands you clean JSON, exporting is the easy part. Grab the fields you care about, shape them into a tidy dictionary, and save. A tiny example fetches a product by ASIN and writes a trimmed version into a .json file (code0035, private source). The result is a clean, predictable JSON export you can drop into Pandas, push into Sheets, or feed into your next pipeline step.
Export to Google Sheets
For a team, Google Sheets is usually the friendliest landing zone. Everyone can filter, comment, and build tiny dashboards without touching code. Since the API already returns clean JSON, the only job is turning JSON into rows.
Option 1 — Python + Google Sheets API. Install a Google Sheets client library, authenticate with a service account (the sane way) or OAuth, then call the append-values endpoint. A typical row contains ASIN, product name, brand, price, currency, rating, stock status, and first image URL (code0036).
Option 2 — no-code automation. Tools like Zapier, Make, or n8n follow the same logic: fetch product data → map JSON fields to columns → append rows. This is useful when stakeholders want a live sheet and you do not want to redeploy a backend just to update a spreadsheet.
Sheets works well here because the data is already structured. You can build price-tracking dashboards, compare products by rating or brand, detect stock changes, log ASINs from search results, and share everything with non-technical teammates instantly.
Export to Excel
If your analysts spend their day in Excel, handing them a CSV file is the smoothest route. The API already returns structured product records, so exporting comes down to choosing your fields and writing one row per product. A minimal example writes a single product into a CSV row using Python's built-in csv module (code0037); looping over a list of records extends it to a full dataset. Excel users can then filter, pivot, chart trends, or blend the dataset with other sources.
Scraping Amazon products without code
If nobody on your team writes Python, no-code platforms (Make, Zapier, n8n, Airtable automations) can hit the Amazon API just like a script would, wrapped in a click-and-drag UI. The usual flow:
- Trigger — a schedule, a button press, or a new keyword dropped into a sheet.
- API call — fire the Amazon Product or Search endpoint.
- Iterator — loop over each product in the results.
- Storage — push rows into Google Sheets, Airtable, Notion, a database, or wherever the data should live.
This setup suits analysts, PMs, and marketing folks who want structured data without hassling developers. When you need real data cleaning, rate control, or custom logic, switch to Python.
Scraping Amazon products with AI
If you want to skip selectors entirely, the AI Web Scraping API lets you describe the fields in plain language and handles the HTML, layout quirks, and dynamic widgets. A conceptual payload for grabbing Amazon search results is in code0038. The API fetches the page, interprets shifting layouts, and returns structured data without fixed selectors. It is useful when you want extraction that survives redesigns or when you are building something quick and do not want to babysit a scraper.
Scaling, compliance, and advanced use cases
Pulling one ASIN is nothing. Pulling thousands a day or a million a month is a pipeline. When you scale, you will want:
- Rate limiting so you do not hammer endpoints or trigger avoidable blocks.
- Retries with backoff to smooth over temporary errors or timeouts.
- Logging so you know exactly what ran, when, and what it returned.
- Monitoring to detect anomalies or shifts in Amazon's behavior early.
- A storage strategy to track historical changes in price, stock, rank, and delivery.
Once you are collecting at scale, you are building a dataset, not a one-off export. Many teams blend Amazon product data with other e-commerce sources to analyze pricing, identify gaps, predict trends, detect arbitrage opportunities, or monitor competitors across marketplaces.
On the compliance side, scrape responsibly: respect Amazon's infrastructure, stay at product-level data, review Amazon's Terms of Service, get legal guidance for customer-facing use at scale, and use battle-tested tools instead of rolling your own proxy farms. Long-term scraping is about resilience: a quick script is fragile, but a proper pipeline survives layout changes, API hiccups, and marketplace shifts.
| Field or control | Type | What it carries | Notes |
|---|---|---|---|
| api_key | request param | API key | Load from .env, never hard-code |
| query | request param | ASIN or keyword | ASIN for product endpoint, keyword for search |
| light_request | request param | Boolean | True = faster, no full browser; False = richer reviews and media |
| domain | request param | Marketplace domain | e.g. com |
| country | request param | Country code | e.g. us |
| language | request param | Language code | e.g. en_US |
| currency | request param | Currency code | e.g. USD |
| add_html | request param | Boolean | Include raw HTML in response for 3D viewer detection |
| asin | response field | Product identifier | Stable anchor |
| parent_asin | response field | Parent identifier | Groups variations |
| product_name | response field | Title | Clean string |
| description | response field | Long-form text | Main description block |
| product_overview | response field | Structured details | May be missing |
| bullet_points | response field | Feature bullets | Near top of page |
| product_details | response field | Spec map | Weight, dimensions, release date |
| rating | response field | Numeric rating | e.g. 4.8 |
| rating_stars_distribution | response field | List of rating/percentage pairs | 5 rows typically |
| reviews | response field | List of review objects | Author, rating, title, timestamp |
| price | response field | Current price | Normalized |
| price_buybox | response field | Buy Box price | Usually the real one |
| price_shipping | response field | Shipping cost | May be FREE or missing |
| pricing_url | response field | Offer listing URL | For seller research |
| images | response field | List of image URLs | Store URLs, download later if needed |
| brand | response field | Brand name | Top-level |
| manufacturer | response field | Manufacturer | May be present |
| category | response field | List or ladder | Empty, strings, or objects |
| sales_rank | response field | Ranking signal | Not always present |
| best_sellers_rank | response field | Ranking signal | Not always present |
| buybox[0].stock | response field | Stock label | Human-readable |
| max_quantity | response field | Max cart quantity | Soft low-stock indicator |
| delivery | response field | List of delivery entries | type and date.by |
Field presence varies by listing and by light_request setting. Treat missing fields as normal, not as errors.
Frequently asked questions
Is it legal to scrape Amazon product data with an API?
It depends on how you use it. Product-level data is generally less sensitive, but you should review Amazon's Terms of Service and get legal advice if you are scraping at scale or commercially. The API handles infrastructure and rate limiting, but compliance is ultimately your responsibility. {{OWNER_LEGAL_DISCLAIMER}}
What is the difference between scraping Amazon HTML and using a product data API?
HTML scraping is brittle: selectors break, layouts shift, widgets move, and CAPTCHAs pop up. A product data API returns clean JSON with titles, prices, ratings, images, specs, stock, and delivery info in one shot, without proxies or headless browsers.
How do I get an ASIN if I only have a product name or keyword?
Use the Amazon Search endpoint. Send a keyword and get a list of products with ASINs, titles, prices, ratings, and images. Then fetch full product details from the Product endpoint.
How can I avoid getting blocked while scraping Amazon?
Use rate limiting, space out requests, and rely on tools that handle proxies, headers, retries, and browser simulation for you. A managed API handles the heavy lifting so you do not have to fight Amazon's anti-bot systems manually.
How do I get historical Amazon product prices?
Log API responses on a schedule and store price, Buy Box info, stock labels, and delivery windows. Over time this becomes a historical dataset you can use for trend analysis and forecasting.
Can I scrape Amazon product reviews at scale?
Yes, but do not rely on the product page alone. Use a dedicated reviews scraper or the Amazon Review API, which is built for volume and returns full review text and metadata reliably.
What is the best way to share scraped Amazon data with non-technical teams?
Push structured rows into Google Sheets, Excel, or a no-code dashboard. A simple pipeline is: scrape, map fields, append rows, so non-technical teammates can explore the data instantly.
When should I turn off light_request?
Turn it off when you need richer review data or dynamic media that the fast path trims. Keep it on for ASIN discovery and quick field lookups where speed and cost matter more than depth.
{
"asin": "B0BNWCWG7L",
"product_name": "LEGO Icons The Lord of The Rings: Rivendell Building Set for Adults...",
"price": 499.99,
"rating": 4.8,
"images": [
"https://m.media-amazon.com/images/I/91Un4M87n2L._AC_SL1500_.jpg",
"https://m.media-amazon.com/images/I/51E5mAhLkTL._AC_.jpg"
],
"product_details": {
"asin": "B0BNWCWG7L",
"item_weight": "15.2 pounds",
"product_dimensions": "20.59 x 18.9 x 8.58 inches",
"release_date": "June 1, 2023"
},
"stock": "In Stock"
}
{
"page": 1,
"products": [
{
"asin": "B009CGCQPU",
"title": "Pride & Prejudice",
"price": 3.99,
"rating": 4.8,
"reviews_count": 37500
}
]
}
asin = data["asin"]
parent_asin = data.get("parent_asin")
{
"rating": 4.8,
"rating_stars_distribution": [
{ "rating": 5, "percentage": 92 },
{ "rating": 4, "percentage": 4 },
{ "rating": 3, "percentage": 1 },
{ "rating": 2, "percentage": 1 },
{ "rating": 1, "percentage": 2 }
],
"reviews": [
{
"author": "Danil",
"rating": 5,
"title": "5.0 out of 5 stars Stunning set...",
"timestamp": "Reviewed in the United States July 15, 2025"
}
]
}
"delivery": [
{
"type": "FREE delivery",
"date": { "by": "Tuesday, January 20" }
},
{
"type": "get FREE delivery Prime members",
"date": { "by": "Saturday, January 17" }
}
]
curl "{{APP_BASE_URL}}/api/v1/amazon/search?api_key=$SCRAPINGBEE_API_KEY&query=Pride+and+prejudice&light_request=true&country=us&domain=com&language=en_US¤cy=USD&sort_by=bestsellers&start_page=1&pages=1"
{
"page": 1,
"products": [
{
"asin": "B009CGCQPU",
"title": "Pride & Prejudice",
"price": 3.99,
"currency": "USD",
"rating": 4.8,
"reviews_count": 37500,
"is_prime": false,
"organic_position": 1
}
]
}
{
"ai_query": "Return a list of products, their prices, ratings, and product links for DSLR cameras.",
"ai_extract_rules": {
"products[]": {
"title": "text of product title",
"price": "price value",
"rating": "star rating",
"url": "link to product page"
}
}
}