Skip to content

Finding elements by XPath in Selenium

XPath locators in Selenium come down to two calls: find_element returns the first match and raises an error when nothing matches, while find_elements returns a list that is simply empty when nothing matches.

OUTPUT FORMATSHTMLJSONMarkdown

The XPath string you pass decides which nodes qualify. This guide covers the two calls, the syntax tokens that do most of the work, and how to read the three failure signals you are most likely to hit.

find_element vs find_elements: choosing the right call

Both methods take a By.XPATH locator strategy and an XPath string. The difference is what happens on a miss and what you get back.

  • find_element(By.XPATH, ...) returns the first element in the page that matches the XPath selector. If nothing matches, it raises NoSuchElementException.
  • find_elements(By.XPATH, ...) hands back a list containing each matching element. When the selector matches nothing, you get an empty list back rather than an exception.

A quick decision rule: if you need exactly one node and its absence is a real failure, use find_element. If you are counting, iterating, or absence is a normal outcome, use find_elements and check the list length before indexing.

On the API history: the older find_element_by_xpath() call was the conventional approach for years. Selenium 4.0 deprecated it, and Selenium 4.3.0 dropped it entirely together with the rest of the find_element_by_* family. In their place, Selenium exposes one find_element() / find_elements() pair that takes a By locator strategy, so the same call shape works across the language bindings.

XPath syntax basics that cover most locators

Four tokens handle the majority of everyday selectors:

  • // matches an element anywhere in the document, regardless of its position in the DOM tree.
  • / matches a direct child of the current node.
  • [@attribute='value'] filters elements by an exact attribute value, for example //input[@id='email'].
  • [contains(@attribute, 'value')] filters elements where an attribute contains a substring, which is useful for matching one of several space-separated CSS classes, for example //a[contains(@class, 'btn')].

A short field map helps when reading selectors:

Prefer the most specific filter that still matches reliably. An exact ID is usually more stable than a substring class match, but a substring match survives when an element carries several classes at once.

Worked example: one page, three selectors

The example below opens a page and uses three XPath selectors against it: //img to collect every image, //a[contains(@class, 'btn')] to collect links carrying a btn class, and //h1 to grab the first heading. It prints the heading text and the counts of images and button links.

The structure is the lesson: find_elements is used where a collection is expected, and find_element is used where exactly one node is wanted. The try/finally block ensures the browser is closed even if a selector fails.

The exact source code is preserved in the protected code module (code0001) and accompanies this page. The site URL in the example is a configuration placeholder (https://finescrape.com), not a fixed destination.

Reading the three common failure signals

The same XPath call can fail in three distinct ways, and each points to a different fix.

The key distinction is between a selector that is wrong and a selector that is right but early. NoSuchElementException can mean either; InvalidSelectorException means the string itself cannot be parsed; an empty list means the query ran but matched nothing. If the element appears later, the fix is usually a wait, not a new selector.

Related Selenium tasks

If you are working through Selenium element handling, these companion questions cover nearby tasks: finding elements by CSS selector, blocking image loading, getting page source, scrolling to an element, taking a screenshot, waiting for page load, resolving chromedriver and geckodriver PATH errors, and saving and loading cookies. A broader Selenium web scraping tutorial is also available for end-to-end workflows.

XPath token field map
TokenWhat it matchesTypical use
//Any descendant at any depthBroad search for a tag
/Direct child onlyPrecise structural paths
[@attr='value']Exact attribute matchIDs, names, exact classes
[contains(@attr, 'value')]Substring attribute matchMulti-class elements, partial URLs

Token descriptions are general XPath semantics; the example selectors are illustrative, not a claim about any specific live page.

Failure signal comparison
SignalWhat it meansFirst check
NoSuchElementExceptionfind_element found no matchXPath syntax, and whether the element is in the DOM at query time (it may load asynchronously)
InvalidSelectorExceptionThe XPath string is malformed, e.g. unbalanced brackets or quotesRe-read the selector for bracket and quote balance
Empty list from find_elementsNo elements matched; no exception is raisedCheck the list length before indexing

Exception names and behaviors reflect Selenium 4.x as described in the captured reference; verify against your installed Selenium version.

Frequently asked questions

Why does find_element raise NoSuchElementException instead of returning None?

find_element is designed to return exactly one element, so a miss is treated as an error. If absence is a normal outcome in your flow, use find_elements and check whether the returned list is empty.

Is find_element_by_xpath still available?

No. Selenium 4.0 marked it deprecated, and Selenium 4.3.0 took it out entirely, together with the whole find_element_by_* family. The replacement is the unified pair find_element(By.XPATH, ...) or find_elements(By.XPATH, ...), which takes a By locator strategy.

What is the difference between InvalidSelectorException and NoSuchElementException?

InvalidSelectorException means the XPath string cannot be parsed, for example because of unbalanced brackets or quotes. NoSuchElementException means the string parsed but find_element found no matching node. Fix the string in the first case; check the DOM or timing in the second.

How do I match an element that has several CSS classes?

Use a substring match on the class attribute, such as //a[contains(@class, 'btn')]. This matches when 'btn' appears among the element's space-separated classes.