Skip to content

Data Extraction API: CSS and XPath Rules That Return JSON

When the output you need is a small, stable JSON object rather than a full HTML dump, a selector-driven extraction API can replace a fragile custom parser.

OUTPUT FORMATSHTMLJSONMarkdown

You describe what to keep — a heading, a list of cards, a link attribute — and the service returns those values. This guide explains the rule shapes, the request controls, and the cases where extraction is not the right tool.

What the extraction layer does

The API fetches a target page and applies your extraction rules to the resulting document. Rules are expressed with CSS or XPath selectors. A rule can pull a single text value, a list of repeated elements, or an attribute such as a link target. Because the same request can also enable rendering and proxies, the rules run against the page as the service retrieved it, including JavaScript-rendered content when rendering is switched on.

The practical benefit is separation of concerns: your application asks for named fields, and the parsing logic lives in the rule set rather than in hand-written DOM traversal code. When the page structure changes, you update the rule, not the application.

Three rule shapes to know

Most extraction jobs reduce to three shapes.

  1. Single field. A selector such as an h1 or an id selector returns the matched element's text. Use this for titles, subtitles, prices shown once, or a canonical label.
  1. Nested list. A selector picks a repeated container (for example, a card), the rule declares a list type, and an output map names the fields inside each card. Each field has its own selector relative to the container. This is how a page of articles, products or search results becomes an array of objects.
  1. Attribute. Appending an attribute reference to a selector returns the attribute value instead of the text. The supplied example uses a link selector with an @href suffix to capture the destination URL of each card title.

A fourth, smaller capability is output cleaning: returned values can be trimmed so leading and trailing whitespace does not leak into your data.

A worked example: blog index to JSON

The supplied example extracts a blog index. The rule set names three top-level fields: a title from an h1, a subtitle from an id selector, and an articles list. The articles list selects a card container and maps three fields inside each card: the post title text, the post link attribute, and the post description text.

The exact request contract is preserved in the public contract module code0003, so you can copy the real parameter names and selector suffixes rather than a paraphrase. In prose, the request says: fetch the blog page, then return an object with a title string, a subtitle string, and an array where each item has title, link and description. The response example shows exactly that shape — a top-level object plus an array of article objects, with the link values resolved to full URLs.

This is the pattern to copy for any repeated-card page: identify the container, identify the fields inside it, and decide which fields are text and which are attributes.

Request controls and what they change

The extraction rules are one parameter among several that shape the response. The table below maps the controls that matter for extraction jobs to their effect. Values are the supplied contract values; treat them as the request contract rather than as marketing claims.

When selector extraction is the wrong tool

Selector rules are a poor fit when the page has no stable structure, when the data you need is spread across many pages that each require a different navigation path, or when you need a full-fidelity copy of the page rather than named fields. In those cases a full-page render, a screenshot, or a dedicated scraper for a known site is a better starting point. Extraction rules also do not decide whether a request is allowed; that is a separate question about the target site and your use case.

Failure checklist before you change the rule

When a field comes back empty or wrong, separate the causes before editing selectors.

  • Did the page load at all? Check the HTTP status and whether the body contains the expected container.
  • Was the content rendered? If the target text only exists after JavaScript runs, enable rendering and retry before changing the selector.
  • Did the selector match more than one element? A single-field rule returns the first match; if you expected a list, switch to a list rule.
  • Is the value in an attribute rather than text? Add the attribute reference.
  • Is whitespace the only problem? Apply output cleaning rather than rewriting the selector.

This order avoids the common mistake of rewriting a correct selector when the real issue was rendering or blocking.

Supporting resources

The documentation for the extraction feature is the place to confirm the current parameter names and selector syntax before you build a rule set. When you adapt a sample, check three things: which selector language it uses, whether it declares a list type, and whether any field ends in an attribute suffix. Those three checks catch most copy-paste errors. The knowledge base is organised around common extraction scenarios, so it is worth scanning for a scenario close to yours before writing rules from scratch. Support channels and any trial or cancellation terms are set by the operator of this service and should be confirmed on the current pricing and support pages rather than assumed from this guide.

Extraction request controls
ControlWhat it doesNotes
extract_rulesDefines named output fields and selectorsAccepts single selectors, list rules with output maps, and attribute references
List typeMarks a rule as returning an arrayUsed with a container selector and an output map
Attribute referenceReturns an attribute value instead of textExample uses @href on a link selector
JavaScript renderingRuns the page's scripts before extractionEach successful request with JS rendering enabled costs 5 API credits
Output cleaningTrims returned valuesRemoves leading and trailing whitespace
Screenshots & Extraction RulesPlan feature covering screenshots and structured extractionIncluded on all listed plans

Feature availability and credit costs are the supplied contract values. Check the current pricing page before relying on them for a purchase decision.

Frequently asked questions

Which selector languages can I use in extraction rules?

Rules can be written with CSS selectors or XPath. The supplied examples use CSS selectors, including id selectors and attribute references. XPath is supported for cases where CSS cannot express the match cleanly.

How do I return a list of items rather than a single value?

Use a container selector for the repeated element, declare the rule as a list, and provide an output map for the fields inside each item. Each field selector is evaluated relative to the container, so the same rule works for every card on the page.

Can I extract a link URL instead of the link text?

Yes. Append the attribute reference to the selector — the supplied example uses an @href suffix on the title selector — and the response contains the attribute value rather than the element text.

Why is a field empty even though the selector looks correct?

The most common causes are that the content is rendered by JavaScript and rendering was not enabled, that the page was blocked before extraction, or that the selector matched a different element than expected. Check the response status and the rendered body before changing the rule.

How do extraction requests affect API credit usage?

Requests start at one credit each. Enabling JavaScript rendering makes each successful request cost 5 API credits, and premium proxies cost 10 or 25 credits depending on whether rendering is enabled. Credits are valid for one month and do not roll over. Confirm current credit rules with the service operator before budgeting.