4
1 Comment

How can I build a comprehensive database of skin care products?

Hi,

I am building a platform for skincare enthusiasts.
One core functionality includes searching for products and writing reviews for them.

I would like to have a skincare product database that is as accurate and comprehensive as possible.
It should at a minimum include the brand name, product name, and the category (cleanser, moisturizer, etc.).
Ideally, I would like to also have data for Asian skincare products.

For now, I just scraped data off some skincare shop websites.
However, the product offerings from even a large website like ULTA are a fraction of all popular skincare products.
In addition, a lot of websites including Sephora block scraping websites programmatically, and getting data from multiple sources leads to the stressful problem of de-duplicating.

One option is to just do my best to scrape as much as possible on my end and let users submit the product they want if it doesn't exist.

However, I was wondering if there's a simpler or easier way to acquire these product data. I want to know how others successfully did this.
Is there a product database I can buy for cheap? Is there some public API? Or some library that makes deduplication super easy?

on December 27, 2021
  1. 1

    Did you end up figuring this out? I'm interested in building something in skincare too.