Extract Every Anchor Link With BeautifulSoup in Python
When you need a complete inventory of a page's links, BeautifulSoup offers a direct route: parse the fetched HTML, search for the anchor tag, and read each match's destination and label.
This guide explains that workflow, what to check in the results, and how it fits alongside other common BeautifulSoup searches.
The core idea: search by anchor tag
Links in HTML are represented by anchor elements. BeautifulSoup's find_all method accepts a tag name, so supplying the anchor tag returns the full set of those elements from the parsed document in one call. That makes it the natural starting point when the goal is a page-wide link inventory rather than a single match.
Walking through the example
The supplied example, preserved as protected module code0001, follows a short sequence: fetch the page with requests, build a BeautifulSoup object from the response content using the html.parser parser, call find_all for the anchor tag, and loop through the results. For each element it prints the href attribute as the link destination and the string value as the link text. Replace the example page URL with the address you want to inspect, keeping the same fetch, parse, search, and iterate order.
A quick results checklist
Before using the output, run through this checklist: confirm the fetched response actually contains the page you intended; verify the parser choice matches the document format; check that find_all was given the anchor tag and not a class or attribute filter; inspect a few results to confirm href values look like real destinations; and note where link text is missing, since string content can be empty or nested differently on some anchors. Keeping those checks separate helps you tell a parsing problem from a page that genuinely has few links.
Related BeautifulSoup questions
Finding every anchor is only one of the common BeautifulSoup tasks. Related questions cover choosing between Scrapy and BeautifulSoup, using XPath selectors, locating elements by class or by attribute, handling elements without specific attributes, searching across multiple tags, finding sibling nodes, scraping tables, and selecting values between two nodes. Each of those narrows or reshapes the search described here, so they are useful next steps once a plain link inventory works.
Frequently asked questions
What does find_all return when searching for links?
Pass the anchor tag name to find_all on the parsed page. That returns every matching anchor element, after which each item's href attribute holds its destination and its string content holds the visible text where present.
Where is the URL stored on each link element?
Use the href attribute for the destination. The example prints that attribute alongside the element's string value, so the two pieces of information stay paired per link.