Skip to content

Universal Scraping API for Any Public Page

Send one request for a public URL and receive rendered HTML or structured data, with JavaScript execution, proxy handling and anti-bot measures managed for you.

OUTPUT FORMATSHTMLJSONMarkdown

This guide explains when to use the general endpoint, which controls matter, and how to choose an output format before you build a pipeline around it.

What the universal endpoint is for

A general-purpose endpoint is useful when the target set is broad or changes often. Instead of maintaining separate parsers for profiles, product pages, event listings and articles, you submit a URL and extraction specification and receive a consistent response shape. That consistency simplifies retries, logging and downstream storage.

Use it for cross-site research, catalog comparison, event aggregation and contact-discovery workflows where public pages are the source of truth. If a single site needs a highly specialised, stable schema, a dedicated scraper may still be simpler to maintain; the universal route is strongest when variety is the problem.

A practical setup checklist

Create an account and retrieve an API key. Install the client for your stack or call the endpoint directly. Send the target URL with your key, then add only the controls you need.

Checklist before your first production run:

  • Confirm the page is public and the fields you need are visible after rendering
  • Decide between rendered HTML, selector-based JSON, AI-based JSON, a screenshot or Markdown
  • Enable JavaScript rendering only when dynamic content is required, because it changes credit use
  • Add geo-targeting when content or pricing varies by country
  • Add a stealth proxy when ordinary routing is blocked
  • Test a small batch, inspect missing fields, then scale concurrency gradually

Control map: choosing the right option

render_js: run the page in a real browser so dynamic content appears in the response. stealth_proxy: use a stealth route for sites with stronger bot defences or CAPTCHA challenges. country_code: route through a residential address in the specified country. extract_rules: define CSS or XPath selectors for precise, repeatable field extraction. ai_extract_rules: describe the fields you want in natural language and receive structured JSON without per-site selectors.

The key distinction is between selector rules and AI rules. Selectors give deterministic control when the layout is known. AI rules are faster to prototype on unfamiliar sites, but you should still validate field names, types and missing values before relying on them in production.

Outputs and interactions worth knowing

Beyond raw HTML, the same workflow supports full-page screenshots for visual checks, JavaScript scenarios that click, scroll or wait for content, Markdown output for LLM pipelines, Google result extraction, and no-code automation through workflow tools. Treat these as output and interaction choices around the core request, not as separate scraping systems.

For example, a product-specification job can start with rendered HTML to verify availability, move to selector rules once the layout is stable, and add a screenshot only for pages where visual layout affects interpretation. That staged approach keeps costs and debugging effort proportional to the problem.

Reliability, privacy and plan selection

Proxy rotation and headless-browser rendering are the two mechanisms that most reduce IP-based blocking, and the service is designed to keep throughput steady when request volume spikes, with typical page results arriving within a few seconds. On personal data, the stated position is narrow: personal data from scraped sites is not collected or stored unless a user explicitly asks for it. Treat that as a policy to verify against your own compliance requirements rather than an automatic guarantee for every jurisdiction or use case. If your workflow touches personal data, document your lawful basis, retention period and deletion process before scaling up, and confirm the provider's current data-processing terms in writing. For regulated data categories, a legal review is advisable before any production run. When comparing plans, evaluate them on the dimensions that actually drive cost and throughput: monthly credit allowance, how many requests can run simultaneously, the credit multipliers attached to JavaScript rendering and premium proxies, geo-targeting coverage, screenshot and extraction-rule availability, and whether priority support, a dedicated account manager and team management are part of the tier. A useful worksheet is to list each candidate plan in a row and each of those dimensions in a column, then mark included, not included or metered. Two credit rules deserve particular attention because they change effective capacity: credits are described as valid for one month without rollover, and base requests start at one credit while JavaScript rendering and premium proxies consume more. Model your expected monthly request mix against those multipliers before choosing a tier, since a plan that looks generous at base cost can be consumed quickly by rendering-heavy workloads. Also confirm how failed or blocked requests are billed, because retry-heavy pipelines can distort a simple credits-per-request estimate. Pricing, plan names, credit allowances, concurrency limits, included features, VAT treatment and cancellation terms are commercial terms that must be confirmed by the operator before they are published; until then, treat any figures you encounter as provisional and request a current written quote. The same applies to any free-credit offer: the quantity, expiry window and whether a payment card is required should be confirmed directly rather than assumed from marketing copy.

Support and documentation

The documentation set includes worked examples in Python, Node.js and cURL, plus a request builder that assembles a call from parameters you supply, with further samples in other common languages. A knowledge base covers frequent use cases with accompanying code, and support is offered through live chat or email. For evaluation purposes, a sensible sequence is to read the language example closest to your stack, reproduce it against a single public URL, then adapt the request builder output to your own parameters and compare the two responses for consistency. That gives you a baseline before you invest in integration work. When assessing support quality for your own needs, note the channels available, the typical response expectations you are given, and whether answers come from engineers who can discuss request-level detail; these are questions worth asking directly rather than inferring from a product page. Customer counts, average ratings and testimonials are claims that require verified evidence from the operator before publication, so they are not asserted here. If you need social proof to make a procurement decision, request current, attributable references and check review platforms independently. The same caution applies to any statement about success rates or reliability: ask for the measurement method, the sample and the date before treating a figure as a service-level expectation, and confirm any contractual uptime or support commitment in the agreement itself.

Frequently asked questions

What data can I extract from any website?

Any content visible on a public page can be returned. Selector-based rules target specific elements, while a natural-language specification can produce structured JSON without writing selectors for each site.

How do I get started scraping any website?

Create an account, obtain an API key, and send a request containing the target URL. Rendering and proxy rotation are handled server-side, and a free credit allowance is available for testing.

Will I get blocked when scraping any website?

Requests use rotating residential proxies and a real headless browser to resemble ordinary traffic. For more aggressive defences, a stealth proxy option can be enabled on the request.

Can I scrape any website without writing code?

Yes. A request builder accepts a URL and parameters and produces a ready-to-copy snippet in several common languages, so you can test without first writing integration code.

How do I handle JavaScript when scraping any website?

Enable JavaScript rendering on the request. The page runs in a headless browser and the returned HTML reflects the rendered state rather than only the initial markup.

Can I scrape any website from a specific country?

Yes. Supply a country code to route the request through a residential address in that country, which helps when page content differs by location.