Find all links with DOM Crawler and PHP
Pulling every anchor URL out of a page is a common first step before filtering, deduplicating, or following links.
With DOM Crawler you can do it two equivalent ways: a CSS-style filter or an XPath filter. Both examples fetch HTML with Guzzle, load it into a Crawler, select anchors, and print each href value.
Setup and workflow
Install Guzzle with Composer if you do not already have it, since the examples use it to retrieve the page body as a string. Pass that HTML to a new Crawler instance. The two selection methods then differ only in how anchors are matched.
Checklist before you run either example:
- Guzzle and the Symfony DomCrawler component are installed via Composer
- You have a target URL to fetch and convert its response body to a string
- You expect relative, absolute, fragment-only, and placeholder href values in the output
- You plan to filter or resolve URLs after collection, because raw href values are printed as-is
Option 1: filter method
This variant selects anchors with a CSS-style selector. It is the shorter option when you only need a tag name.
Protected module code0001 contains the complete filter example, including the Guzzle request, Crawler construction, anchor selection, href loop, and sample output.
Option 2: filterXPath method
This variant selects the same anchors with an XPath expression. Choose it when you are already thinking in XPath or need a more expressive query than a simple tag selector.
Protected module code0002 contains the complete filterXPath example, including the Guzzle request, Crawler construction, anchor selection, href loop, and sample output.
Reading the output
Both variants produce the same kind of result: one href per line, in document order. In the captured example output you can see a root path, login and register paths, in-page fragments such as pricing and FAQ anchors, a blog path, a bare fragment placeholder, and feature paths. That mix is normal for a real homepage.
Use the output as a diagnostic:
- Fragment-only values like # are placeholders, not crawlable destinations
- Relative paths need a base URL before they become absolute URLs
- Duplicates are possible if a page repeats the same href in navigation, body, and footer
- Empty or missing href attributes will print as empty lines, so filter them if you build a sitemap or crawler queue
Which method to choose
Pick filter for simple tag, class, or attribute selectors. Pick filterXPath when the selection depends on document position, ancestry, or text content that CSS cannot express. In both cases the loop pattern is identical: iterate the selected nodes and read the href attribute from each one.
If you need only a subset of links, narrow the selector before looping rather than filtering a full list afterward. That keeps the intent clear and avoids collecting anchors you never wanted.
Frequently asked questions
Can I use XPath selectors in DOM Crawler?
Yes. The filterXPath variant accepts an XPath expression, and the example uses //a to match every anchor element in the document.
How do I narrow the selection beyond all anchors?
Related questions cover selecting elements without specific attributes, by attribute, by class, and by multiple tags. Those are separate selection tasks from the plain anchor search shown here.
What about sibling nodes, tables, or values between nodes?
Related questions cover sibling nodes, tables, and values between two nodes. Each needs its own selector or traversal logic rather than the simple href loop.