Data extraction in Node.js with selector rules
When a page already contains the values you need, the slow part is often turning raw HTML into a small, predictable object.
This guide shows the selector-rule approach: name each field, give it a CSS selector, and let the API return a structured result instead of parsing the whole document yourself. The example uses a Node.js client and a documentation page with a heading and subtitle.
What selector rules do
A selector rule is a label paired with a CSS selector. The label becomes the key in the returned object, and the selector tells the service which element supplies the value. This removes the need to post-process the request content with an external HTML parser for straightforward field extraction.
For a page whose main heading is an h1 and whose subtitle is a span with the class text-[20px], the rule set is:
- title → h1
- subtitle → span.text-[20px]
The response is a JSON object with those two keys, for example a title value of "Documentation - Data Extraction" and a subtitle value of "Extract data with CSS selector". The exact values depend on the target page.
Verify selectors before sending the request
Open the target page in a browser, open the developer tools console, and run document.querySelector("CSS_SELECTOR") with your candidate selector. If the call returns the element you expect, the selector is a reasonable candidate for the rule. If it returns null, the selector does not match the current document; adjust it before running the extraction.
This check is especially useful for class-based selectors such as span.text-[20px], which can be sensitive to markup changes. A selector that worked during development may stop matching after a site update.
Node.js client workflow
The supplied example imports the SDK package shown in the protected code module, creates a client with an API key, and calls the client's get method with a URL and a params object. Inside params, the extract_rules object maps each label to its selector. The response data is decoded with TextDecoder and printed as text.
The important inputs and outputs are:
- Input: target URL.
- Input: extract_rules object with label-to-selector pairs.
- Input: API key supplied to the client constructor.
- Output: a response whose data is decoded into a JSON string containing the named fields.
The exact code is provided in the accompanying protected code module and should be used as the source of truth for method names, parameter names, and response handling.
Decision checklist: selector rules or manual parsing?
Use selector rules when:
- You need a small number of named fields from a page.
- The fields can be identified by stable CSS selectors.
- You want a structured object without writing an HTML parser.
Consider manual parsing or a different approach when:
- The values require computation across multiple elements.
- The page markup changes frequently and selectors are unstable.
- You need to follow links or paginate beyond a single response.
A selector mismatch usually shows up as a missing key or an empty value in the returned object. A request-level failure, such as an authentication or network problem, appears as an error from the client call. Keeping those two failure types separate makes debugging faster.
Field and control map
The extraction request has three layers:
- Client configuration: API key and SDK import.
- Request parameters: target URL and extract_rules.
- Rule entries: each label and its CSS selector.
The response layer contains the decoded body. In the example, decoding produces a JSON string with title and subtitle keys. If you add more rules, add more label-to-selector pairs; the response object grows accordingly.
Where to go next
The data extraction documentation covers the feature in more detail. For CSS selector syntax, the W3Schools CSS selectors reference is a useful companion. Both are linked from the source page and are appropriate next reads after this workflow.
Frequently asked questions
What should I do if document.querySelector returns null?
The selector did not match an element in the current page. Check the element's tag, class, and surrounding structure, then try a more specific or more stable selector before adding it to extract_rules.
What does the extraction response look like?
The example decodes the response data into a JSON string containing the labels you defined. With title and subtitle rules, the result is an object with those two keys and the matched text as values.
Can I extract more than two fields?
Yes. Add another label-to-selector pair inside extract_rules. Each pair becomes a key in the returned object.