OpenNews Scraper API
News moves fast, and so does the markup around it.
A hosted scraping API lets your team fetch OpenNews and other news pages without standing up a fleet of headless browsers, rotating proxies, or retry logic. You send a URL and your key; the service handles rendering and returns HTML you can parse or extract with CSS/XPath rules. This page explains what the OpenNews scraper API is for, how the plan attributes map to real news workflows, and where the limits are.
What the OpenNews scraper API does
The OpenNews scraper API is a hosted endpoint that retrieves a target page and returns rendered HTML. For news work, that means you can collect article pages, section fronts, and author pages without maintaining your own browser infrastructure. The service handles JavaScript rendering, proxy rotation, and geotargeting, so pages that load content dynamically or vary by region are more likely to return usable markup. You then parse the HTML yourself or use extraction rules to pull specific fields. The API is not a pre-built OpenNews database — it is a fetching and rendering layer that returns page content for your own pipeline.
Where teams use it
Three news workflows come up most often. First, trending-story collection: pulling section fronts and topic pages to see what is being covered right now, which supports market-sentiment analysis and editorial planning. Second, article metadata: capturing titles, summaries, publication details, and timestamps for competitor or industry tracking. Third, author and contact details: building a list of bylines and public contact information for lead generation or collaboration outreach. Each workflow starts with a URL list and a parser; the API supplies the rendered page, and your code decides which fields matter.
Reading the plan attributes
The plan table is best read as a set of levers rather than a single price. API credits are the monthly request allowance: they are consumed per request, do not roll over from one month to the next, and requests begin at one credit each. Turning on JavaScript rendering raises a successful request to five credits. Premium proxies raise it further, to ten or twenty-five credits depending on whether JS rendering is also enabled. Concurrent requests set how many calls can be in flight at once, which is the practical throughput ceiling. Geotargeting routes traffic through 200+ countries and territories (40+ on classic proxies). Screenshots and extraction rules cover full-page captures and structured pulls via CSS or XPath. Priority email support, a dedicated account manager, and team management are positioned on the higher tiers. Prices, credit allowances, concurrency limits, and VAT treatment are commercial terms that the operating provider sets and may change; verify them against the current pricing page before committing to a plan (EXACTREADERLITERAL0END). Listed prices are exclusive of VAT.
A worked example: one article page
Suppose you want to capture the headline, byline, publish date, and body text from a single OpenNews article. You send the article URL with JavaScript rendering enabled and an extraction rule set that targets the headline element, the byline element, the date element, and the article body container. The API renders the page, applies the rules, and returns the matched values. Because JS rendering is on, that request costs five credits rather than one. If the page also requires a premium proxy for regional access, the cost rises to twenty-five credits. If the selector for the byline misses because the markup changed, you get an empty field rather than an error — which is why a validation step in your pipeline matters.
Telling failures apart
Three failure modes look alike but call for different responses. A blocked or empty response usually means the target refused the request; rotating proxies or a different geotarget can help. A page that returns but lacks dynamic content usually means JavaScript rendering was off or the wait was too short. A page that returns full HTML yet yields empty extracted fields usually means the CSS or XPath selector no longer matches the current markup. Inspecting the raw HTML before debugging selectors saves time. Proxy rotation and headless-browser rendering are designed to reduce IP-based blocking risk, but no provider can guarantee success against every target (EXACTREADERLITERAL1END).
Getting started
A request needs two inputs: your API key and the URL of the page you want to scrape. The official Python library wraps the request so you do not have to manage proxies or browsers yourself. Trial terms — including any free call allowance and whether a card is required — are set by the operating provider and should be confirmed before you rely on them (EXACTREADERLITERAL2END). Documentation and code samples cover Python, Java, Go, PHP, Curl, and JavaScript.
Compliance and legal considerations
Whether scraping a given news site is permissible depends on that site's terms of service and the laws that apply to you. A scraping API handles the technical fetching, but it does not grant permission to collect or reuse content. Before running a collection job, confirm that your use case — research, monitoring, republication — is allowed, and check whether personal data such as author contact details is subject to additional rules. Data-handling commitments for scraped personal data are set by the operating provider and should be verified against current policy before you rely on them (EXACTREADERLITERAL3END).
| Attribute | What it controls | Notes |
|---|---|---|
| API credits | Monthly request allowance | Valid for one month; do not roll over; requests start at 1 credit each |
| Concurrent requests | Simultaneous in-flight calls | Throughput ceiling |
| JavaScript rendering | Executes page scripts before returning HTML | 5 credits per successful request when enabled |
| Rotating & premium proxies | Proxy rotation to reduce IP-based blocking risk; premium tier for harder targets | 10 or 25 credits depending on JS rendering |
| Geotargeting | Routes requests through a chosen country/region | 200+ countries and territories (40+ on classic proxies) |
| Screenshots & extraction rules | Full-page screenshots and CSS/XPath structured extraction | Included on all listed tiers |
| Priority email support | Faster support channel | Business tier and above |
| Dedicated account manager | Named contact | Business tier and above |
| Team management | Invite users and assign credit/concurrency quotas | Business tier and above |
Attribute descriptions reflect the listed plan table; confirm current terms, prices, support commitments and proxy capability with the provider before purchase (EXACTREADERLITERAL4END, EXACTREADERLITERAL5END, EXACTREADERLITERAL6END).
Frequently asked questions
What is OpenNews?
OpenNews is a collaborative platform for journalists and storytellers. It offers content sharing, community engagement, and training resources aimed at supporting news reporting and innovation in journalism.
What is OpenNews scraping?
It is the process of programmatically extracting data from OpenNews and similar news sites. A hosted scraping API handles JavaScript rendering and proxy management so your team can focus on the data rather than the browser infrastructure.
Is scraping OpenNews legal?
It depends on the site's terms of service and the laws that apply to you. A scraping API streamlines the technical process but does not grant permission. Confirm that your intended use complies with the relevant terms and regulations before collecting data.
What data can I extract from OpenNews?
Common fields include titles, summaries, article metadata, comments, and author details. With extraction rules you can target specific elements — headline, byline, date, body container, comment thread — and retrieve only the fields your pipeline needs.
How do I scrape OpenNews data quickly?
Send the page URL and your API key to the endpoint. Enable JavaScript rendering if the page loads content dynamically, and use proxy rotation if the target blocks repeated requests. Extraction rules let you return structured fields in the same call rather than parsing raw HTML downstream.
Do I need an API to scrape OpenNews?
No — you can scrape manually or with your own browser automation. An API becomes useful when you need to scale across many pages, handle JavaScript-heavy layouts, or avoid maintaining proxy and browser infrastructure yourself.
How many credits does a typical news-page request use?
A plain request starts at one credit. Enabling JavaScript rendering raises it to five credits per successful request. Adding premium proxies raises it to ten or twenty-five credits depending on whether JS rendering is also enabled. Check the plan tooltips for the exact multipliers before sizing a job.