What if your product depends heavily on another company's service?
I'm working on a web forum for real estate properties, and the UX with the property information fetched from via Zillow API, is far superior than, say, just using a link to Zillow listing.
However, Zillow poses a rate limit of 1000 calls per day, and storing the data from the API is against their policy.
Although they "may approve higher daily call limits if you adhere to these API Terms of Use and pass a Zillow audit and review", 1000 calls a day is way too small.
Also, I'm a mere developer, not a VC-backed company, and don't know how to get their approval or whether they will approve me or not.
They may see my product as a potential threat to them. Of course, Zillow is a giant company, but if this is successful, it can take a small portion of their revenue from them.
I know a lot of companies have such risks, and I've worked for a startup, the product of which completely hinged on the contact export feature of LinkedIn.
How do you eliminate such risk in the long run?
What is it like to contact the service provider (Zillow in my case) and get approval for API use or even partnerships? Did you need a business development person?
How do giant companies like Zillow or LinkedIn view indie hackers like us?
I worked for a startup once and they business model was entirely dependent on their ability to scrape the websites of competitors. The product was a better interface (and a combined view of searches across multiple other vendors). They weren't using official APIs, but in many cases the sites were locked behind a login. The strategy was to use the login credentials of their users. In your case, i wonder if you could convince users to give you their Zillow login credentials, and then you query Zillow on their behalf with their creds, thus getting around the 1000 request limit. I mention this to give you confidence that however hacky your solution ends up being, its quite likely that someone else out there with a ton of money also thought that hacky solution was worth trying. So don't abandon hope!
I realize this is not an elegant suggestion, but unfortunately i think you've answered your own question already, which is if your product is not in Zillow's interest to exist, they're not going to help you. Big companies do not view indie hackers kindly. The only exception to that rule is if they offer an app market place. Salesforce/Shopify and others thrive off of indie hackers building applications on top of their platform. Zillow not so much.
Also, I'm fairly certain that Zillow itself is built on top of other APIs that are closer to the original source of the data. So that could be another avenue for you to explore, try and pull your data closer to the original source, don't even bother with Zillow. It will be much harder, but could potentially avoid the issues you raised here.
Does the Zillow API even allow you to fetch listing data? I looked into it recently, and when signing up, the only API options I see are:
from: https://www.zillow.com/webservice/Registration.htm
I took this to mean that the API does not give you access to the real estate listings themselves.
Hmm.. I was looking at https://www.zillow.com/howto/api/GetSearchResults.htm, but although it shows up on google search results, there's no link in Zillow.
I don't need the listings. I just need to users to type in the Zillow URL of a property when they create a list, and get some information (photos, address, etc.) in the property in that link. Unfortunately, I cannot scrape it easily because the property information page is behind the capcha wall. I should probably email them to ask about this.
It's strange because when I slack someone a link to a zillow property, the link preview includes the metadata including the main photo and a short description, but I can't even get that.
When slack/messenger apps generate a preview they usually use a special user-agent setting and Zillow could well be detecting this and skipping the captcha.
Slack: https://api.slack.com/robots
Facebook: https://developers.facebook.com/docs/sharing/webmasters/crawler/
You could try set that and see what happens.
Thanks a lot!
Don't worry about it. If they're threatened by you, then you've got traction! Good job!
They have an API for a reason, try to figure out their intentions there.
You could always have a backup API that you can use. Take a peek at https://rapidapi.com/. You don't have to implement it, but it will encourage you to abstract the underlying API. Let's say you have a
findPropertyNearMemethod. Can you design a toggle so that you can switch to your backup API? I bet you could.BUT - don't overthink it. Your goal is really to find someone who will pay you! That's the more important goal.
Good luck
I would each out to Zillow and see what they say.
However, based on common sense and past experiences, the reality is if your product will be disadvantageous to Zillow they will not want to grant you access. It might be worth exploring alternative methods of data collection.
Even if they say they are okay today,... you can't take it for granted.. If your product gets in their way.. they will instantly crush it. Twitter started with a open API for any devs to write and build on their API.. And when they become popular they screwed most devs and they purchased a most popular client which indirectly killed the remaining products. I feel sad about those who spent years of their time and money and then backstabed by twitter.
From my experience this is not the best option to build a product which completely relays on others. I always had problems with keeping up to date with external changes by the providers. Also limits aren't the fun here. In general to much dependences.
Now I have another approach - I try to balance between 3/4 of internal solutions, and 1/4 external. This reduces the risk and greatly increases the independence.
I've been testing the model here https://pagemtr.com and have to say it works (by now).