Skip to content

Perplexity Shared-Answer Scraping for Public Threads

Perplexity's public shared-answer pages are useful for three jobs: studying how the assistant synthesizes an answer, watching how it describes a brand, and mapping which domains it cites.

OUTPUT FORMATSHTMLJSONMarkdown

Doing that by hand does not scale, and building your own headless-browser fleet with rotating proxies is a separate project. A managed scraping API collapses that into one HTTP call: you send a Perplexity URL plus an API key, and the service handles proxy rotation, fingerprint spoofing, and JavaScript rendering before returning a parseable response. This page explains what you can pull, which request controls change the result, how credits are consumed, and how the plan tiers differ so you can decide before signing up.

What you can pull from Perplexity

Public shared-answer pages are the unit of work. Each one carries the query text, the AI-synthesized answer, the source citations with their URLs, and the follow-up question thread. That combination supports three distinct pipelines:

  • Shared-answer research. Collect public threads to study how the assistant frames answers, which sources it leans on, and which questions attract the most engagement.
  • Brand mention monitoring. Run a set of queries and check whether your brand appears in the answer, how the product is described, and where the description drifts from reality.
  • Citation mapping. Aggregate the domains cited across the topics you cover to see which sources the assistant treats as authoritative.

A single call returns all four fields for one page. You do not parse HTML on your side; the response is already structured.

Request controls and what each one changes

The base call needs only an API key and a Perplexity URL. The controls below are what you reach for when the base call is not enough. Each one changes either the rendering path, the network path, or the shape of the response.

  • render_js=true — runs the page in a real headless browser, executes the page's JavaScript server-side, and returns the post-render HTML. Use it when the answer or citations only appear after client-side rendering.
  • stealth_proxy — routes the request through a stealth proxy path intended to get past CAPTCHA-style challenges.
  • country_code — routes the request through a residential IP in the named country. Use it when the visible content varies by geography.
  • extract_rules — CSS selectors mapped to field names. The response comes back as parsed JSON keyed by your field names.
  • ai_extract_rules — a natural-language specification of the fields you want. Also returns parsed JSON.

A useful rule of thumb: reach for render_js first when fields are missing, then country_code when the missing fields look geo-dependent, then the extraction rules when you want named fields rather than raw HTML.

Worked example: one public shared-answer URL

Suppose you have a public Perplexity shared-answer URL for a question about a product category. Your first call sends the URL with render_js=true. The response contains the query text, the synthesized answer, the citation list with URLs, and the follow-up thread. From that single response you can:

  1. Store the query text as the topic label.
  2. Store the answer as the snapshot for that date.
  3. Extract each citation URL and count domains to feed the citation-mapping pipeline.
  4. Store the follow-up thread to see which adjacent questions the assistant surfaced.

If a field you need is missing, add extract_rules with a CSS selector mapped to that field name, or ai_extract_rules with a plain-language description. The response then arrives as parsed JSON with your field names, so downstream storage does not need a second parsing step.

Three different failure modes and the control that addresses each

Not every failed request means the same thing. The right response depends on which of these three you are seeing:

  • Concurrency limit reached. The API enforces concurrency limits per plan, not per-target rate limits. If you are hitting your plan's ceiling, the fix is to reduce in-flight requests or move to a plan with higher concurrency. The plan table below lists the concurrent-request figure for each tier.
  • Target throttling. If Perplexity itself throttles aggressive patterns, spacing out requests or using multiple sessions is the fix. This is a target-side behavior, not an API-side limit.
  • Geo-restricted content. If the answer or citations differ from what you see in a browser in another country, set country_code to route through a residential IP in that country.

Keeping these three separate prevents the common mistake of raising plan concurrency when the real problem is geo-dependent content.

How credits are consumed

Credits are the unit of consumption, and the cost per request depends on which controls you enable. The base request starts at 1 credit. Enabling JavaScript rendering raises the cost to 5 credits per successful request. Enabling premium proxies costs 10 credits with JS rendering disabled, or 25 credits with JS rendering enabled. Credits are valid for one month and do not roll over. Geotargeting routes through any of 200+ countries and territories (40+ on classic proxies). Screenshots and extraction rules are available on every plan tier shown below.

The practical implication: a monitoring pipeline that runs render_js=true on every call consumes five times the credits of a plain request. If only a subset of pages need rendering, splitting the job into a plain-request pass and a render pass keeps the monthly credit total predictable.

Plan tiers at a glance

The table below lists every plan tier with its monthly price, API credit allowance, concurrent-request ceiling, and feature inclusion. Prices are exclusive of VAT. The feature rows are identical across tiers for JavaScript rendering, rotating and premium proxies, geotargeting, and screenshots and extraction rules; the differences are concentrated in credit volume, concurrency, and the support and team-management rows.

Read the table by matching your expected monthly credit consumption to the credit column, then confirm the concurrency column covers your peak in-flight requests. The support and team-management rows matter only if you need priority email support, a dedicated account manager, or team management with per-user credit and concurrency quotas.

Setup path

The workflow is four steps:

  1. Create an account to get an API key and the free starting credits.
  2. Install the SDK for your stack (Python, Node.js, Go, PHP, Ruby, or Java), or use raw cURL.
  3. Send one HTTP call with your API key and a Perplexity URL. The service rotates proxies, spoofs fingerprints, and renders JavaScript behind the scenes.
  4. Add advanced parameters only when the base call is not enough: render_js for JavaScript, stealth_proxy for CAPTCHA challenges, country_code for geo-targeting, and ai_extract_rules for natural-language extraction.

The dashboard request builder lets you paste a URL, set parameters, and copy a generated snippet in any of the supported languages, so you can get a first request working without writing the call by hand.

Plan tiers, credits, concurrency, and feature inclusion
PlanPriceAPI CreditsConcurrent requestsJavaScript renderingRotating & Premium ProxiesGeotargetingScreenshots & Extraction RulesPriority Email SupportDedicated Account ManagerTeam Management
Hobby$19 /mo75,00025IncludedIncludedIncludedIncludedNot includedNot includedNot included
Freelance$49 /mo250,00050IncludedIncludedIncludedIncludedNot includedNot includedNot included
Startup (Recommended)$99 /mo1,000,000100IncludedIncludedIncludedIncludedNot includedNot includedNot included
Business$249 /mo3,000,000200IncludedIncludedIncludedIncludedIncludedIncludedIncluded
Business +$599 /mo8,000,000400IncludedIncludedIncludedIncludedIncludedIncludedIncluded
Enterprise 14$999 /mo14,000,000500IncludedIncludedIncludedIncludedIncludedIncludedIncluded
Enterprise 24$1,599 /mo24,000,000600IncludedIncludedIncludedIncludedIncludedIncludedIncluded
Enterprise 41 (Recommended)$2,399 /mo41,000,000700IncludedIncludedIncludedIncludedIncludedIncludedIncluded
Enterprise 69$3,599 /mo69,000,000800IncludedIncludedIncludedIncludedIncludedIncludedIncluded
Enterprise 120$5,799 /mo120,000,000900IncludedIncludedIncludedIncludedIncludedIncludedIncluded

All prices are exclusive of VAT. API credits are valid for one month and do not roll over. Requests start at 1 credit each. Each successful request that enables JS rendering costs 5 API credits. Each successful request that enables premium proxies costs 10 or 25 API credits, depending on whether JS rendering is disabled or not. Geotargeting routes through any of 200+ countries and territories (40+ on classic proxies). Team management allows inviting multiple users and assigning custom credit and concurrency quotas. These figures are a dated reference; confirm current pricing and allowances before purchase.

Frequently asked questions

Can I scrape Perplexity without writing code?

Yes. The dashboard request builder lets you paste a URL, set parameters, and copy the generated snippet in Python, Node, Go, PHP, Ruby, or cURL. You get a working call without hand-writing the request.

How do I handle JavaScript when scraping Perplexity?

Pass render_js=true on the request. The page runs in a real headless browser, the JavaScript executes server-side, and you get back the post-render HTML. This is the control to reach for when the answer or citations only appear after client-side rendering.

Can I scrape Perplexity from a specific country?

Yes. Pass country_code on the request to route through a residential IP in that country. This is useful when the visible content varies by geography.

What's the rate limit for scraping Perplexity?

The API enforces concurrency limits per plan, not per-target rate limits. If the target throttles aggressive patterns, space out requests or use multiple sessions. If you are hitting your plan's concurrency ceiling, reduce in-flight requests or move to a plan with higher concurrency.

How do I get structured JSON from Perplexity?

Pass extract_rules with CSS selectors mapped to field names, or use ai_extract_rules with a natural-language specification. The response comes back as parsed JSON keyed by your field names.

Which programming languages can I use to scrape Perplexity?

Any. SDKs are published for Python, Node.js, Go, PHP, Ruby, and Java. Plain cURL also works, so you can pick whatever your stack already uses.

What does a Perplexity shared-answer page return?

Each public shared-answer page returns the query text, the AI-synthesized answer, the source citations with their URLs, and the follow-up question thread. That is the full set of fields a single call produces.

How do I keep credit consumption predictable?

Split the job into a plain-request pass and a render pass. A base request starts at 1 credit; enabling JavaScript rendering raises it to 5 credits per successful request; enabling premium proxies costs 10 credits with JS rendering disabled or 25 credits with it enabled. Credits are valid for one month and do not roll over, so matching the render pass to only the pages that need it keeps the monthly total predictable.