Pull Brave Search News Headlines as Structured JSON
Brave Search News aggregates headlines from many outlets.
Collecting that feed yourself usually means writing a separate parser for each publisher and keeping it alive as markup changes. An HTTP scraping API moves that work off your plate: you send a Brave Search News article or results URL, the service renders the page, handles proxy rotation and bot mitigation, and returns HTML or parsed JSON. This page explains the request inputs, the fields you get back, how credit costs stack, and where the limits actually sit.
What the API does with a Brave Search News URL
The workflow is four steps. You create an account and receive an API key with free starting credits. You install a client SDK or call the endpoint directly. You pass a Brave Search News URL plus your key. The service renders the page, rotates proxies, and returns either the rendered HTML or structured output you defined. Optional parameters extend the base call: JavaScript rendering, stealth proxying for CAPTCHA-heavy pages, country targeting, and extraction rules written either as CSS selectors or as a natural-language specification. The result is a single request per page instead of a bespoke scraper per outlet.
Fields returned for each news result
Each Brave Search News result carries the headline, the source publication, the publish timestamp, a snippet, a thumbnail, and the article URL. Brave's index is privacy-focused and sometimes surfaces outlets that the major engines do not, which is one reason teams add it as a second feed alongside other news sources. The field set is stable enough to map directly into a news table or a monitoring dashboard.
Request inputs and what each one changes
The base call needs only the target URL and your API key. Everything else is optional and each option changes both behavior and credit cost:
- render_js: executes page JavaScript before extraction. Costs 5 credits per successful request.
- stealth_proxy: routes through stealth infrastructure for CAPTCHA-heavy targets. Costs 75 credits per request.
- premium_proxy: premium proxy pool. Costs 10 credits without JS rendering, 25 with it.
- country_code: routes through a residential IP in the chosen country. Useful when news content differs by geography. Geotargeting covers 200+ countries and territories on premium proxies, 40+ on classic proxies.
- extract_rules: CSS or XPath selectors mapped to field names; the response comes back as parsed JSON.
- ai_extract_rules: a natural-language description of the fields you want, for cases where writing selectors is awkward.
Credits are valid for one month and do not roll over. Requests start at 1 credit each, and the option costs above stack on top of that base.
Choosing between raw HTML, extract_rules and ai_extract_rules
Use raw HTML when you already have a parser and want full control. Use extract_rules when the page structure is stable and you know the selectors, because it is deterministic and cheap to reason about. Use ai_extract_rules when the layout shifts often or when the fields you want are described more easily in words than in selectors. A practical pattern is to start with ai_extract_rules to discover the shape of the data, then lock in extract_rules once the selectors settle. That keeps early iteration fast without paying AI extraction costs forever.
Worked example: monitoring one keyword across a country
Suppose you track a product name and want only results as seen from one market. You send the Brave Search News results URL for that keyword, set country_code to the target market, enable render_js because the results page is JavaScript-driven, and attach extract_rules mapping headline, source, timestamp and link to your own field names. The response is parsed JSON you can insert straight into a monitoring table. If the same query later returns a layout you did not anticipate, switch that one request to ai_extract_rules with a plain description of the four fields and compare the output before changing your selectors.
Rate limits versus target-side throttling
These are two different constraints and they need different fixes. The API enforces concurrency limits per plan: the number of requests that can be in flight at the same time. That is a ceiling on your own parallelism, and the fix is a higher plan or a queue that respects the limit. Separately, the target site may throttle aggressive request patterns regardless of your plan. That is a target-side behavior, and the fix is spacing requests out or spreading them across multiple sessions. If requests start failing in bursts while your concurrency is under the plan limit, look at the target side first.
Setup path and language support
Client SDKs are published for Python, Node.js, Go, PHP, Ruby and Java, and plain cURL works for any stack. The documentation includes code samples in Java, Python, Go, PHP, cURL and JavaScript. A first request typically takes a few minutes from signup to response. The API key and free starting credits are issued at account creation.
Three workflows this feed supports
Brand and keyword monitoring: pull results for every term you track and diff new headlines against the previous run. Competitor analysis: collect article summaries to spot shifts in how competitors are covered. Trending topic discovery: use the feed to find emerging subjects before they saturate other channels. All three use the same request shape and differ only in the query and the extraction schema.
Plan tiers and what changes between them
Plans differ on three axes: monthly API credits, concurrent requests, and support features. All tiers include JavaScript rendering, rotating and premium proxies, geotargeting, and screenshots with extraction rules. Priority email support, a dedicated account manager, and team management are included on the Business and Business + plans only; the Hobby, Freelance and Startup plans exclude them, and every Enterprise plan includes them. Team management lets you invite multiple users and assign each a custom quota of credits and concurrency. Prices are exclusive of VAT. The full tier table with every plan and its exact credit and concurrency figures is provided in the pricing module that accompanies this page; check current pricing before committing to a tier.
Data handling and compliance posture
The service does not collect or store personal data from scraped sites unless the user explicitly requests it. Proxy rotation and headless browser rendering are used to reduce IP-based blocking and mimic normal browsing behavior. The platform is described as scaling across thousands of headless browsers and rotating proxies, with typical response times in the 1-5 second range on most sites including JavaScript-heavy ones. These are the provider's own operational claims; verify them against your own workload before relying on them for service-level planning.
| Field | What it contains | Typical use |
|---|---|---|
| headline | Article title as shown in Brave Search News | Deduplication and alerting |
| source publication | Outlet name | Source weighting and filtering |
| publish timestamp | When the article was published | Recency sorting and windowing |
| snippet | Short excerpt of the article | Preview and relevance scoring |
| thumbnail | Image URL for the result | Dashboard display |
| article URL | Link to the full article | Fetching full text downstream |
Field presence reflects the captured page description; verify against a live response before building a strict schema.
| Option | Credits per successful request | Notes |
|---|---|---|
| Base request | 1 | Starting cost per request |
| render_js | 5 | JavaScript rendering enabled |
| premium_proxy without JS | 10 | Premium proxy pool |
| premium_proxy with JS | 25 | Premium proxy plus rendering |
| stealth_proxy | 75 | CAPTCHA-heavy targets |
Option credits are additive to the 1-credit base, not mutually exclusive tiers: a request with JS rendering and a premium proxy costs 1 + 5 + 25. Figures are the captured published rates; credits are valid for one month and do not roll over. Confirm current rates before budgeting.
| Tier group | Credits range | Concurrency range | Support features |
|---|---|---|---|
| Standard (Hobby to Business +) | 75,000 to 8,000,000 | 25 to 400 | Priority support, account manager and team management included on Business and Business + only; excluded on Hobby, Freelance and Startup |
| Enterprise (14 to 120) | 14,000,000 to 120,000,000 | 500 to 900 | All support features included |
Summary only. The complete per-plan table with every original row and figure is delivered in the pricing module; prices exclude VAT and are subject to change.
Frequently asked questions
Do I need a Brave account to use this?
No. You need an account with the API provider, which issues an API key and free starting credits without requiring a card. You then send the Brave Search News URL and the service handles the rest.
Can I get results as seen from a specific country?
Yes. Set country_code on the request and traffic routes through a residential IP in that country. This matters when news coverage differs by geography. Geotargeting covers 200+ countries and territories on premium proxies and 40+ on classic proxies.
Is there a per-target rate limit?
The API enforces concurrency limits per plan rather than per-target rate limits. If the target site throttles aggressive patterns, space requests out or use multiple sessions. Those are separate constraints with separate fixes.
How do I get parsed JSON instead of HTML?
Pass extract_rules with CSS selectors mapped to your field names, or use ai_extract_rules with a natural-language description of the fields. Either way the response comes back as parsed JSON.
Which languages can I call the API from?
Any language that can make an HTTP request. Published SDKs cover Python, Node.js, Go, PHP, Ruby and Java, and plain cURL works for everything else.
What does a request cost?
Free starting credits are issued at signup with no card required, and paid plans begin at the entry tier. The base cost is 1 credit per request, with add-ons stacking on top: 5 credits for JavaScript rendering, 25 for premium proxy with JS rendering, and 75 for stealth proxy. Credits are valid for one month and do not roll over.