Hi,
I am building a platform for skincare enthusiasts.
One core functionality includes searching for products and writing reviews for them.
I would like to have a skincare product database that is as accurate and comprehensive as possible.
It should at a minimum include the brand name, product name, and the category (cleanser, moisturizer, etc.).
Ideally, I would like to also have data for Asian skincare products.
For now, I just scraped data off some skincare shop websites.
However, the product offerings from even a large website like ULTA are a fraction of all popular skincare products.
In addition, a lot of websites including Sephora block scraping websites programmatically, and getting data from multiple sources leads to the stressful problem of de-duplicating.
One option is to just do my best to scrape as much as possible on my end and let users submit the product they want if it doesn't exist.
However, I was wondering if there's a simpler or easier way to acquire these product data. I want to know how others successfully did this.
Is there a product database I can buy for cheap? Is there some public API? Or some library that makes deduplication super easy?
Did you end up figuring this out? I'm interested in building something in skincare too.