Build a Python Flight Scraper for Google Flights, Kayak and Skyscanner
Choose the collection method that matches your goal, render JavaScript-heavy fare pages, extract predictable fields, and turn repeated checks into a price tracker. This guide compares four approaches, explains why plain requests fail, and shows how to make a multi-source scraper more resilient.
Four ways to get flight data
There is no official Google Flights API because Google discontinued QPX Express in 2018. That gap explains why developers collect fares from flight sites or use aggregator services. Four practical routes are available: reconstruct a site's own API, automate a browser, use an official or aggregator API, or use a managed scraping API. Speed is only one decision factor. Maintenance and permission matter just as much: reconstructed and browser-based methods break often, official APIs are sanctioned but may be gated, and scraping any site requires respecting its terms and robots.txt.
Choosing by speed, reliability and upkeep
A reconstructed API is usually fastest and free, but it is brittle and may still need anti-bot help. Browser automation with Playwright or Selenium lets you interact like a person, yet it is slow, resource-heavy and easy to fingerprint. Official and aggregator APIs such as Amadeus, Duffel and Kiwi are clean when access is available, but they can be paid, approval-gated or limited in routes and fields. A managed scraping API renders pages and rotates stealth routing in one call, making it the strongest fit for multi-source production work without operating a browser and proxy fleet.
Why plain requests return empty fare pages
Flight sites are JavaScript applications. The initial HTML may be only a shell, with fares inserted after scripts run. Google Flights sits behind Google WAF, while Kayak and Skyscanner use Akamai protections that fingerprint clients and block datacenter addresses. A direct request can therefore return HTTP 200 with hundreds of kilobytes of HTML and still contain no prices. In the captured JFK to LAX example, a plain request produced 658,116 characters with zero dollar signs and no 'Nonstop' text. After rendering, the same search produced more than four million characters, 279 dollar signs and visible nonstop results. JavaScript rendering means executing page scripts in a headless browser so dynamic fares enter the HTML. Residential routing uses a home connection rather than a datacenter, and stealth routing adds further anti-bot evasion.
Fetch rendered results
Before coding, prepare Python 3.10 or newer, the requests package and an API key stored in an environment variable rather than hard-coded. Protected modules code0001 and code0002 cover installation and key export. For Google Flights, combine JavaScript rendering, the Google-specific option and a wait period so fares can stream in. For Kayak and Skyscanner, use stealth routing instead; Skyscanner may need a longer wait because results populate progressively. The protected fetch module is code0005, with the rendered result shown in code0006. Check each target's terms and robots.txt and keep request rates polite.
Command example
pip install requestsExtract structured fields
Raw HTML is rarely useful downstream. Plain-English extraction asks for a named flights array containing airline, integer price, departure and arrival times, duration, stop count and details. That produces a stable shape even when page markup changes. The protected extraction module is code0007 and its example output is code0008. CSS selectors can work on stable, semantic pages, but Google Flights uses obfuscated, rotating class names. In the captured test, a copied selector returned zero rows, demonstrating the maintenance risk. Use selectors where markup is controlled and plain-English extraction for flight pages whose structure changes frequently.
Use one pattern across three sources
One pattern can cover Google Flights, Kayak and Skyscanner. Only the URL and routing option need to change: Google uses its specific option, while the other two use stealth routing. The extraction request stays the same because it describes fields rather than page structure. Protected module code0011 defines the source map. In captured examples, the same JFK to LAX nonstop appeared at $184 on Google Flights, $181 on Kayak and $175 on Skyscanner, showing why comparing sources can reveal meaningful differences. Kayak examples are in code0012 and Skyscanner examples in code0013.
The fast-flights shortcut
For Google Flights alone, fast-flights reconstructs the protobuf-encoded tfs request and parses the server response without a full render. Install the library with typing_extensions because the latter is imported but not declared as a dependency; protected module code0014 covers that step. A custom fetcher can route the generated URL through a scraping service and retry when Google serves a consent page. Protected module code0015 contains that integration. The captured run parsed 30 itineraries, with prices matching the rendered extraction. Treat this as a fast shortcut, not a durable foundation: it is unofficial, Google-only and vulnerable to private schema changes.
Command example
pip install fast-flights typing_extensionsHandle blocks, retries and rate limits
Transient failures are normal. Retry network errors with increasing delays, retry server errors, return gracefully for other unsuccessful statuses, and skip a source that yields no fares instead of crashing the whole job. Protected module code0017 implements that pattern. In one captured run, Google returned a cheapest JetBlue fare of $168 while Kayak returned no parseable fares and was skipped; a later run or longer wait could recover it. Use the strongest routing tier only where needed, add polite delays, and cache results that do not need to be fresh.
Turn the scraper into a price tracker
A tracker repeats the scraper on a schedule, stores each cheapest fare with a timestamp, compares it with the previous observation and alerts when price falls to or below a threshold. A single SQLite table is enough for a small deployment. Protected module code0019 records history and evaluates the alert condition, while code0020 sends the message. In the captured replay, JFK to LAX moved from $184 to $168 against a $175 threshold, triggering an alert on the second run. Run the job with cron or a scheduled workflow, keep credentials in a protected environment file, and never place the key directly in a crontab.
Official APIs versus scraping
Official APIs are preferable when they cover your routes and access rules. Access conditions change, so verify current terms before building. Captured evidence described Amadeus Self-Service as decommissioned on July 17, 2026, Duffel as bookable and pay-as-you-go, Kiwi Tequila as invitation-only, Skyscanner Travel API as partner-approval-only, and Aviationstack as status and schedule data rather than live bookable fares. When a free tier, approval gate or field limitation blocks the project, managed scraping offers broader source coverage.
Stay unblocked without overspending
Match routing strength to each site instead of paying for the heaviest tier everywhere. Rendering alone suits easier pages, premium routing helps where datacenter addresses are blocked, and stealth routing is reserved for Kayak- and Skyscanner-class protections. Captured July 2026 figures listed Classic at 1 credit, rendering at 5, premium routing with rendering at 25, stealth at 75, plain-English extraction adding 5, and Google URLs at a flat 20; confirm current rates with the owner before publication. Because server errors were not charged in the captured workflow, retrying them was safe. Cache repeatable results and maintain a polite request rate.
| Method | Speed | Reliability | Cost pattern | Best fit |
|---|---|---|---|---|
| Reconstructed API | Fastest | Low to medium | Free | Quick Google Flights lookups and hobby projects |
| Browser automation | Slow | Low | Free infrastructure plus proxies | Learning and tiny-scale work |
| Official or aggregator API | Fast | High | Paid or approval-gated | Bookable fares when access fits |
| Managed scraping API | Fast | High | Credits per request | Multi-source production scraping |
Captured comparison; verify current provider terms before implementation.
Frequently asked questions
Is there a Google Flights API?
No official public version exists. Google discontinued QPX Express in 2018 and did not replace it. Developers therefore reconstruct a site's own request, render pages through a scraping service, or use an aggregator API where access fits.
Is it legal to scrape flight prices?
Publicly visible fares may be lawful to collect in some places, but this is not legal advice. A site's terms may prohibit collection, rules differ by country, and login-protected areas should not be scraped. Check terms and robots.txt, keep rates polite, and prefer an official API when one fits.
Why does my flight scraper return no data?
Flight pages often load fares with JavaScript, so a plain request can return a shell without prices. Anti-bot systems can also block datacenter addresses. Rendering in a real browser and using residential or stealth routing addresses both causes.
What is the fastest way to scrape Google Flights?
Reconstructing the underlying protobuf request, as fast-flights does, avoids a full browser render and is faster for Google Flights alone. It is unofficial and can break when the private schema changes, so a rendered multi-source approach is more durable.
How do I track flight prices over time?
Run the scraper on a schedule, append each result with a timestamp to a database or CSV file, compare it with the previous run, and alert when a fare falls below your threshold. Cron or a scheduled workflow can run the job.
Can I scrape Skyscanner or Kayak?
Yes. The same rendered-request pattern works, but Kayak and Skyscanner use demanding anti-bot protections, so stealth residential routing and a real browser render are usually needed. Check each site's terms first.
{
"flights": [
{"airline": "JetBlue", "price_usd": 184, "departure_time": "8:05 AM", "arrival_time": "11:08 AM", "duration": "6 hr 3 min", "stops": 0},
{"airline": "JetBlue", "price_usd": 184, "departure_time": "9:30 AM", "arrival_time": "12:39 PM", "duration": "6 hr 9 min", "stops": 0},
{"airline": "JetBlue", "price_usd": 184, "departure_time": "3:15 PM", "arrival_time": "6:19 PM", "duration": "6 hr 4 min", "stops": 0},
{"airline": "JetBlue", "price_usd": 168, "departure_time": "11:29 AM", "arrival_time": "6:27 PM", "duration": "9 hr 58 min", "stops": 1},
{"airline": "Alaska", "price_usd": 176, "departure_time": "9:55 AM", "arrival_time": "4:22 PM", "duration": "9 hr 27 min", "stops": 1},
{"airline": "JetBlue", "price_usd": 181, "departure_time": "1:00 PM", "arrival_time": "7:00 PM", "duration": "9 hr", "stops": 1}
]
}
{
"flights": [
{"airline": "JetBlue", "price_usd": 181, "departure_time": "9:30 am", "arrival_time": "12:39 pm", "duration": "6h 09m", "stops": 0},
{"airline": "JetBlue", "price_usd": 181, "departure_time": "8:05 am", "arrival_time": "11:08 am", "duration": "6h 03m", "stops": 0},
{"airline": "JetBlue", "price_usd": 161, "departure_time": "11:29 am", "arrival_time": "6:27 pm", "duration": "9h 58m", "stops": 1},
{"airline": "JetBlue", "price_usd": 181, "departure_time": "6:00 am", "arrival_time": "8:59 am", "duration": "5h 59m", "stops": 0},
{"airline": "Delta", "price_usd": 269, "departure_time": "11:00 am", "arrival_time": "1:51 pm", "duration": "5h 51m", "stops": 0},
{"airline": "JetBlue", "price_usd": 181, "departure_time": "7:00 am", "arrival_time": "10:02 am", "duration": "6h 02m", "stops": 0}
]
}
{
"flights": [
{"airline": "Delta", "price_usd": 264, "departure_time": "2:45 PM", "arrival_time": "5:43 PM", "duration": "5h 58", "stops": 0},
{"airline": "jetBlue", "price_usd": 175, "departure_time": "3:15 PM", "arrival_time": "6:19 PM", "duration": "6h 04", "stops": 0},
{"airline": "jetBlue", "price_usd": 175, "departure_time": "9:30 AM", "arrival_time": "12:39 PM", "duration": "6h 09", "stops": 0},
{"airline": "Alaska Airlines", "price_usd": 156, "departure_time": "9:55 AM", "arrival_time": "4:22 PM", "duration": "9h 27", "stops": 1}
]
}