Web scraping tutorials and integrations by language
Pick a language, then work through the same practical jobs in order: connect the SDK, make requests reliable, run them in parallel, extract structured data, deal with pages that load as you scroll, and capture screenshots.
A separate general section covers browser and request tasks that do not depend on your programming language.
How to choose a guide
Use this checklist to choose the right guide without reading every title:
- Need browser internals or a copied request? Start with Shadow DOM, curl extraction, element removal, JavaScript scrolling, or CSS selector guides.
- Need to connect a project? Start with the SDK guide for Python, NodeJS, Ruby, Go, PHP or C#.
- Requests failing intermittently? Use the retry guide for your language before adding more requests.
- Need speed? Use the concurrent-requests guide, then check how infinite-scroll pages are handled.
- Need structured output? Use data extraction, and the table-extraction guide when the target is a table.
- Need visual evidence? Use the screenshots guide for your language.
General: browser and request tasks
These guides are language-agnostic and useful when the problem sits in the browser or in the request itself. They cover extracting content hidden in a Shadow DOM, copying requests as curl from Chrome, Firefox and Safari, removing an element from an HTML response, scrolling to load more content through a JavaScript scenario, and extracting CSS selectors using Chrome. If you can reproduce the request or identify the selector in the browser first, the language guides that follow become much shorter.
Language guides: same jobs, different runtimes
Python, NodeJS and Ruby each follow a consistent sequence: SDK setup, retrying failed requests, concurrent requests, data extraction, table extraction where listed, infinite scroll, and screenshots. Python also includes adding items to an eCommerce shopping cart. Go, PHP and C# follow the same core sequence without the table-extraction step in the listing. That repetition is intentional: once you understand one language's flow, you can transfer the task order to another runtime and focus only on the syntax and SDK differences.
Worked example: infinite-scroll extraction
A practical example using only the listed tasks: suppose you need product data from pages that load more items as you scroll. Start with the data-extraction guide for your language to define the fields, add the infinite-scroll guide to load the full list, use concurrent requests only after a single page works reliably, and keep the retry guide ready for failed requests. If the page hides content in a Shadow DOM or you need to inspect the underlying request, switch to the general guides for that step, then return to your language guide for the extraction code.
Ready to get started?
Once you have worked through a few guides, the natural next step is to apply them to a real project. Before you commit to a plan, check the registration page for the current credit allowance, how long it lasts, and whether a payment card is needed — those terms can change, so treat any figure you see elsewhere as provisional until the signup page confirms it.
| Language | Setup | Retries | Concurrent | Extraction | Infinite scroll | Screenshots |
|---|---|---|---|---|---|---|
| Python | Yes | Yes | Yes | Yes | Yes | Yes |
| NodeJS | Yes | Not listed | Yes | Yes | Yes | Yes |
| Ruby | Yes | Yes | Yes | Yes | Yes | Yes |
| Go | Yes | Yes | Yes | Yes | Yes | Yes |
| PHP | Yes | Yes | Yes | Yes | Yes | Yes |
| C# | Yes | Yes | Yes | Yes | Yes | Yes |
Based solely on the tutorial titles listed for each language section; table extraction is listed for Python, NodeJS and Ruby, and the shopping-cart guide is listed only for Python.
Frequently asked questions
Which tutorial should I start with?
Start with the language you already use, then follow the same task order in each section: set up the SDK, retry failures, run requests concurrently, extract data, handle infinite scroll, and capture screenshots. Python, NodeJS, Ruby, Go, PHP and C# each have a matching set, so you can compare the approach in your stack with another language without changing the goal.
When are the general guides more useful than a language guide?
Use the general guides when the task is browser- or request-focused rather than language-specific: Shadow DOM content, copying a request as curl from Chrome, Firefox or Safari, removing an element from HTML, loading more content through a JavaScript scenario, and finding CSS selectors in Chrome. Use a language guide when you need the SDK setup, retry pattern or concurrency model for that runtime.
Why do the same tasks appear for several languages?
The language sections repeat the same jobs in different runtimes. That makes it easier to spot what is truly language-specific — SDK setup and concurrency — versus what is the same scraping task, such as extracting a table, handling infinite scroll, or making a screenshot.