Thoughts on the best recommendation systems?

Hi Devs,

We're ideating on our recommendation system for Showwcase to help deliver better content for users on our Discover page.

We've discussed things like:

  1. Wilson Score
  2. TikTok Algorithm
  3. More ideas using wikipedia as a starting point: https://en.wikipedia.org/wiki/Recommender_system

Anyone have ideas on the best way to approach this?

For context, you can visit the app at https://app.showwcase.com


  1. 4

    Etsy has a good whitepaper on this: https://codeascraft.com/2014/11/17/personalized-recommendations-at-etsy/

    It ended up performing a bit better than Top 4 in category when I was in e-commerce.

    How much data do you have? If you don't have enough usage to reach statistical significance with an A/B test, it is probably best to stick with Top N (or some variation like a random 10 in the Top 100). It is easier to implement and may actually perform better since most recommendation engines need a lot of data to be effective anyway.

  2. 2

    I used to work on the AI team at LinkedIn. Some quick thoughts:

    1. It's important to be clear on what you want to optimize for: DAU, likes, comments, a combination of them etc.
    2. Build out simple ML pipeline with a focus on ensuring you have high quality tracking and labeled data. Simpler the model the better to get started.
    3. Ensure that you have a good experimentation setup, this will help with model iteration.
    4. Iterate on the model itself, using your experimentation setup to monitor your key biz metrics for improvements.

    As for the kind of algorithm, most will still get great milage from good old regression models to start, you can try out deep learning technique with time. Or ofcourse you could training a quick deep learning model with something off-the-shelf on tensorflow.

    Hope this helps :)


  3. 2

    There are a lot of ways you can make a recommender system, I would recommend using a hybrid approach.

    These are the items you can show:

    • Top n items
    • Items from people/pages the user has interacted with before
    • Items from sources the user is following
    • Items based on user's interests

    Use a mix of various techniques and monitor which ones work best for each user show more itmes from that technique.

    Hope this helps.

    1. 1

      Yes I think this can be good for us. Since we do want to eventually be tailoring each person's feed to their interest, it makes sense for us to have a multi-regression model as per @luthfur advice.

      We've also been considering using an external service like Algolia which is very popular in the e-commerce space, and they have a bunch of off the shelf products to help with AI, customisation etc, @ProfessorBeekums. But we are careful to not share our user data with 3rd party providers, which makes this tricky. We will have to combine Algolia's results to our own custom built algorithm to deliver results.

Recommended Posts