Ahrefs Free Tools Scraper API
Ahrefs' free tools expose a useful slice of SEO data without a paid seat: Domain Rating and URL Rating on the Website Checker, keyword ideas on the Keyword Generator, referring-domain counts on the Backlink Checker.
The catch is that those pages are JavaScript-heavy and sit behind bot detection, so a plain HTTP fetch usually returns an empty shell. This API renders the page in a real headless browser, rotates proxies, and can hand back either the post-render HTML or JSON shaped by your own selectors — one request per Ahrefs URL.
What the public surface actually returns
The free tools are a preview, not the full Ahrefs index. Treat every number as a snapshot of what the public page shows at the moment of the request, and design your pipeline around that.
- Website Checker — Domain Rating (DR) and URL Rating (UR) for a domain or URL.
- Keyword Generator — a preview list of organic keyword ideas for a seed term.
- Backlink Checker — a backlink count snapshot and referring-domain information.
- Across public pages — traffic estimates and competitor lists where the page renders them.
If a metric only exists behind an Ahrefs login or a paid plan, it is out of scope here. The honest boundary is: whatever renders on the public page can be extracted; whatever does not, cannot.
Field and control map
Each row below pairs a job you might want to do with the request parameter that unlocks it. The parameter names are exact API contract strings — copy them as written.
- Get the rendered page at all →
render_js=true. Ahrefs frontends build their tables client-side; without this you get the skeleton. - Parse specific numbers into JSON →
extract_ruleswith CSS selectors mapped to the fields you want (DR, UR, backlink counts, top keywords). The API returns parsed JSON instead of raw HTML. - Let a model read the page →
ai_extract_rulesfor natural-language extraction when the markup shifts and selector maintenance is the bottleneck. - Get past aggressive blocking →
stealth_proxyon the request. - See the page as another market sees it →
country_codeto route through a residential IP in that country. - Drive interactions before capture → JavaScript scenario actions (click, scroll, type) so lazy-loaded tables finish populating.
- Keep a visual record → full-page screenshots alongside the extracted data.
Worked example: a weekly DR/UR leaderboard
Say you track twelve competitors in one niche and want a weekly authority ranking.
- Collect the twelve domains from your CRM or a spreadsheet.
- For each domain, send a GET request to the Ahrefs Website Checker URL for that domain, with
render_js=trueand anextract_rulesmap that pulls the DR and UR values into named fields. - Store the JSON response with a timestamp — the value is the pair (domain, week), not just the number.
- Diff week over week to spot movement, and keep the raw HTML or screenshot for any week where a number looks anomalous.
The same loop works for the Keyword Generator (one seed term per row of your editorial calendar) and the Backlink Checker (one tracked domain per row). The pattern is identical: URL in, rendered page out, selectors applied, JSON stored.
Three failure modes and their first fix
Not every empty result means the same thing. Diagnose before you retry.
- Blocked response (CAPTCHA page, 403, challenge HTML). First fix: add
stealth_proxy. If that still fails, check whether the target is rate-limiting your source IP range and slow the schedule down. - Empty selector match (page rendered, but your
extract_rulesreturned nulls). First fix: inspect the post-render HTML and confirm the selector still matches. Ahrefs markup changes; selectors rot. - JS not executed (page returned but tables are empty shells). First fix: confirm
render_js=trueis actually on the request. This is the most common cause of "the API returned nothing useful."
A fourth, quieter failure: the page rendered and your selector matched, but the number is stale because the public tool itself caches. In that case the fix is scheduling, not scraping.
When the public surface is not enough
The free tools are a preview. If your workflow needs full backlink exports, historical index data, per-keyword volume with confidence, or bulk API access to Ahrefs' own index, the honest answer is a paid Ahrefs plan — this scraper does not replace it and does not claim to.
Use this API when the question is "what does the public page say right now, for these N domains or seeds, on a schedule?" Use a paid plan when the question is "what does the full index say, historically, at volume?" Mixing the two is fine: many teams use the public surface for broad weekly monitoring and a paid seat for deep dives.
Setup in four steps
- Create an account — you get an API key and free credits to test with.
- Pick a client — Python, Node, Go, PHP or Ruby SDKs are available; each Ahrefs scrape becomes a single function call, with no proxy rotation or headless Chrome to manage yourself.
- Send a GET request with your API key and an Ahrefs public-tool URL (for example, the Website Checker page for a domain). The API renders the JavaScript and returns rendered HTML or structured JSON.
- Add parameters as needed —
render_jsfor JavaScript,stealth_proxyfor CAPTCHAs,country_codefor geo-targeting,ai_extract_rulesfor natural-language extraction.
No-code option: the dashboard request builder lets you paste a URL, set parameters, and copy a generated snippet in Python, Node, Go, PHP, Ruby or cURL.
| Ahrefs tool | Metric you can pull | Typical use |
|---|---|---|
| Website Checker | Domain Rating (DR), URL Rating (UR) | Competitor authority leaderboard |
| Keyword Generator | Top organic keyword preview | Keyword-gap dataset for editorial calendar |
| Backlink Checker | Backlink count snapshot, referring domains | Weekly backlink monitoring |
| Public pages generally | Traffic estimate, competitor list | Market snapshot |
These are the metrics the public free tools render. Anything behind an Ahrefs login or paid plan is out of scope.
| Parameter | What it does | When to reach for it |
|---|---|---|
| render_js | Runs the page in a real headless browser and returns post-render HTML | Ahrefs tables are empty in the raw response |
| extract_rules | CSS selectors mapped to named fields; returns parsed JSON | You want DR, UR, backlink counts or keywords as structured data |
| ai_extract_rules | Natural-language extraction | Markup shifts often and selector maintenance is costly |
| stealth_proxy | Routes through stealth proxies | CAPTCHA or challenge page appears |
| country_code | Routes through a residential IP in the given country | Content varies by geo |
| JavaScript scenario | Click, scroll, type before capture | Lazy-loaded tables need interaction to populate |
Parameter names are exact API contract strings. Behaviour described is the documented intent; verify against current API documentation before production use.
Frequently asked questions
What data can I extract from Ahrefs?
Any value that appears on the public page is fair game. Map extract_rules to CSS selectors for the specific fields you need, or hand the page to ai_extract_rules and let a model return structured JSON. Typical targets are DR and UR on the Website Checker, keyword previews on the Keyword Generator, and backlink counts on the Backlink Checker.
How do I get started scraping Ahrefs?
Sign up, grab your API key, and send a GET request with the target URL. The API renders the page, rotates proxies, and returns the data. Free credits are available to test with.
Will I get blocked when scraping Ahrefs?
Each call leaves through a rotating residential proxy inside a genuine headless browser, so the traffic looks like a normal visitor. When a target tightens its defenses, turn on stealth_proxy for that request.
Can I scrape Ahrefs without writing code?
Yes. The dashboard request builder lets you paste a URL, set parameters, and copy the generated snippet in Python, Node, Go, PHP, Ruby or cURL.
How do I handle JavaScript when scraping Ahrefs?
Pass render_js=true on the request. The page runs in a real headless browser, the JavaScript executes server-side, and you get the post-render HTML back.
Can I scrape Ahrefs from a specific country?
Yes. Add country_code to the request and the call leaves through a residential IP in that country. Reach for it whenever the page serves different content based on the visitor's location.
What is the difference between a blocked response and an empty selector match?
A blocked response means the request never reached the page content — you will see a CAPTCHA, a 403, or challenge HTML. An empty selector match means the page rendered but your extract_rules selectors no longer match the markup. The first fix is stealth_proxy; the second is updating the selectors against the current post-render HTML.