Scrape Booking.com Search Listings with Python and Selenium
This guide shows how to collect structured property data from Booking.com search results using Python, Selenium, and Selenium Wire. It covers setting up the browser driver, reading each property card, and moving through paginated results without scraping a page before its new listings have loaded.
Set up the project and browser driver
Use Python 3.10 as the reference version, although the approach should work with most Python versions. Create a project folder and an application file, then install selenium, selenium-wire, and webdriver-manager. Selenium controls the browser, webdriver-manager supplies the matching Chrome driver without a manual download, and Selenium Wire adds request inspection for the pagination step. The setup commands are provided in protected modules code0001 and code0002.
Load a search-results page
Open a Booking.com search-results URL with Selenium and confirm that Chrome loads the expected listings. If availability matters, build the URL from a search that already includes the desired check-in and check-out dates. The initial browser setup is in protected module code0003. Using a manually specified driver path is possible, as shown in protected module code0004, but it requires updating the binary whenever Chrome changes; webdriver-manager avoids that maintenance.
Plan the fields and scope each card
Decide the output fields before writing selectors. This workflow collects the property name, locality, rating, review count, thumbnail, and price. First collect every property card on the page, then process one card at a time so each selector stays scoped to the correct listing. The card collection is in protected module code0005.
Extract each property field
Field map: name comes from the title element; link comes from the href attribute of the title link; address comes from the address element; price comes from the final-price element and excludes taxes; rating and review count come from the review-score block; thumbnail comes from the src attribute of the image element. Name and link extraction are in protected module code0006, address in code0007, price in code0008, review data in code0009 and code0010, and thumbnail in code0011. For the review block, split the visible text at newline characters: the first part is the score and the last part is the review count.
Wait for the next page before scraping it
Click the next-page control using the XPath selector in protected module code0012. Because the results update through a background request, do not scrape immediately. Switch the import to Selenium Wire as shown in protected module code0013, identify the request fired by the page change in the browser Network tab, and wait for that request with protected module code0014. Before each new page transition, clear stored requests with protected module code0015 so a completed request from an earlier page cannot make the wait return too early.
Run the complete workflow and scale it
Protected module code0016 combines card extraction, pagination, request waiting, and collection into a properties list. Before running a larger job, use this checklist: confirm the URL contains the intended destination and dates; verify that every card yields all required fields; check that prices are understood as tax-exclusive; confirm the review-score split produces a score and a count; and verify that the request wait completes before each page is read. For a larger project, consider parallel drivers, database storage, or combining Scrapy with Selenium for pause and resume, filtering, proxy rotation, and multiple output formats.
| Output field | Where to find it | Value to store |
|---|---|---|
| Name | Title element inside the property card | Visible property name |
| Link | Href attribute of the title link | Property URL |
| Address | Address element | Locality text |
| Price | Final-price element | Displayed price, excluding taxes |
| Rating | First line of the review-score block | Review score |
| Review count | Last line of the review-score block | Review count |
| Thumbnail | Src attribute of the image element | Image URL |
Price excludes taxes. Review-score text is split at newline characters.
Frequently asked questions
Why do I need to include dates in the search URL?
Use a search URL that already reflects the dates you care about. The example URL in the guide has no check-in or check-out dates, so availability for specific dates will not be represented unless those dates are selected in the search form and the resulting URL is copied.
Does the extracted price include taxes?
The price element gives the final price shown for the listing and does not include taxes. If tax-inclusive pricing is required, inspect the surrounding DOM structure for additional tax information.
How do I know the next page has loaded before scraping it?
Selenium alone does not tell you when a background request has finished. Selenium Wire can inspect requests, so you can wait for the request triggered by the next-page action before reading the updated listings. Clearing stored requests before each page change prevents an earlier request to the same endpoint from satisfying the wait for a later page.
How are rating and review count separated?
The review-score block contains the score, a descriptive label, and the review count on separate lines. Splitting its visible text on the newline character separates the score from the count, with the middle label discarded.