Finding HTML elements by attribute with BeautifulSoup
When a page marks elements with a custom attribute rather than a class or id, BeautifulSoup can still isolate them.
Both find and find_all accept an attrs dictionary that pairs an attribute name with the value you want, so you can target markup like a data-* flag directly. This guide shows the syntax, a worked Python example, and how to tell a failed match apart from a failed read.
Match on an attribute value with attrs
Pass the attribute name and its expected value as a dictionary to the attrs argument of find or find_all. The dictionary key is the attribute name exactly as it appears in the HTML, and the value is the string you want to match. This works for standard attributes and for custom data-* attributes alike.
Use find when you expect one element and find_all when several may share the value. Supplying a tag name as the first argument narrows the search further; leaving it out searches every tag.
Worked example: tooltips marked with a size attribute
Consider a pricing table whose tooltip buttons carry the attribute data-microtip-size set to medium. The filter is written as attrs={"data-microtip-size": "medium"}, and the tag argument is button, so only buttons with that exact attribute value are returned.
After collecting the matches, the example prints the aria-label attribute of each one with tooltip.get("aria-label"). That second step matters: the attribute you filter on is not necessarily the text you want to keep. The captured output shows a credit-validity sentence, illustrating that the visible label text can live in a different attribute from the one used for matching.
The request targets https://finescrape.com; substitute the page you are allowed to fetch.
Decision checklist before you run the search
- Is the attribute name spelled exactly as in the HTML, including hyphens and case?
- Does the value need to match exactly, or are you after a partial match?
- Do you expect one element (find) or many (find_all)?
- Is a tag name available to narrow the search and speed it up?
- Which attribute holds the data you actually want to output?
- What should happen when the list of matches is empty?
Field and control map
- Tag argument (for example "button"): limits which element types are considered.
- attrs dictionary: key is the attribute name, value is the expected attribute value.
- Return value of find_all: a list-like collection of matching elements.
- .get("aria-label"): reads a named attribute from a single matched element, returning None if it is absent.
- Loop over matches: lets you process each element individually rather than only the first.
Telling failures apart
An empty result and a missing output value are different problems. If find_all returns nothing, the attribute name, the value, or the tag filter does not match the parsed markup; check the raw HTML first. If matches exist but the printed value is None, the filter worked and the attribute you tried to read is not present on those elements. Separating these two cases saves time before you adjust the query.
Related BeautifulSoup tasks
Adjacent questions cover finding elements by class, locating elements that lack a given attribute, collecting all links, selecting between two nodes, and scraping tables. Those pages are useful once attribute matching is working and you need a different selection strategy.
| Goal | Call shape | Result |
|---|---|---|
| First matching element | find(tag, attrs={name: value}) | Single element or None |
| All matching elements | find_all(tag, attrs={name: value}) | List of elements |
| Read a stored attribute | .get(attribute_name) | Value string or None |
| Any tag type | omit the tag argument | Searches all tags |
Call shapes reflect the attrs-based approach shown in the example; exact behavior depends on the BeautifulSoup version installed.
Frequently asked questions
Can I match more than one attribute at once?
The attrs dictionary accepts multiple key-value pairs, and an element must satisfy all of them to match. The example uses a single pair, but the same structure extends to several attributes.
Why does my find_all return an empty list?
The attribute name, its value, or the tag filter does not match the parsed HTML. Verify the exact spelling and value in the page source, including hyphens and case, before changing the query.
How do I get text rather than an attribute?
Use .get() for an attribute value and the element's text accessor for visible text. The example reads aria-label because the wanted string is stored there, not in the element body.
Does this work for standard attributes like id or href?
Yes. The attrs dictionary is not limited to data-* attributes; any attribute name and value pair can be supplied.