Developers December 27, 2019

Best way to get live flight price data?

ajobforme

I've been building a service with a core feature including flight prices.

  1. I first wanted to use Skyscanner API, especially since they have an affiliate program I would want to join for added revenue, but they never responded to my request (need to request the API key).
  2. I now use Amadeus API but the free limits are really low (something like 2000 requests per month...) and I could pay eventually, but another big problem I see is the rate limiting I've been experiencing. I send maybe 15 simultaneous requests and only like 5 go through while the rest return 429s. And this is with only me testing; imagine more users.
  3. I am thinking of implementing a custom scraping solution like this blog (https://towardsdatascience.com/scraping-flight-data-using-python-e71b97e859d3) but I know that it will be so much more work to debug and maintain and I think I'll have the same rate-limiting problem as in number 2, so I'd need to use a proxy service or something, which would cost more too.

I'd appreciate any advice you have for surmounting this :)

  1. 2

    You could ask around (or them) what API are www.adioso.com are using, cheers! Let us know what you decide and implement.

    1. 2

      Thanks! Am trying out Kiwi for moment. Love the feature of selecting a flexible date range and the limits are much more generous than Amadeus. I found this half-answer on HOW Adioso assembles results, but it doesn't say the source of the data.

  2. 2

    Kiwi have an API which is easily accessible

    1. 1

      Woah Thanks! This thing looks much better!

  3. 2
    1. Implement an internal queue and adapt yourself to their rate limit. If it affects the user experience and there's no alternative data source for you, then re-design your product around this.
  4. 1

    Just shallow thought you can proxy and multi user the API just as much as you can scrapping

    Also like 50% sure, should verify, I believe Google bought them a while back skyscanner and it's used in Google flights... (Could be talking ravish but..)

    1. 1

      I don't think so because the service tracks and limits the usage by API key. With scraping, no API key to track so they resort to source tracking, which is overcome by changing the source.

      1. 1

        Can you not get 10,000 keys/users?

        1. 1

          I think I've gotta manually provision each one in their UI so it wouldn't be automatic. Also I'd guess they have a limit for number of keys per account as well.

          1. 1

            Same as scrapping you can automate user and key creation...

  5. 1

    This comment was deleted 2 months ago.

    1. 1

      Thanks. I might have to look into that eventually (hopefully if I grow enough ;)) so I'll keep it in mind, but I really can't tell how much the benefit will be since I'm generating quite a few unique flight queries per user input.

Recommended Posts