18
17 Comments

How I Built A Craft Beer Search Engine For Free

submitted this link to Icon for group Developers
Developers
on November 6, 2021
  1. 2

    @jerrynsh Cool!

    I dropped you a line via email to ask a few questions. :)

    1. 1

      Just got back to you :D

  2. 2

    As someone who’s trying to escape tutorial hell and figuring out how to plan my own side projects, your thought process was really helpful. Thanks for sharing!

    1. 1

      I'm glad you find it helpful! Feel free let me know if you have any questions!

  3. 2

    Woow. This came exactly when I needed it the most. I got this idea a few days ago and the concept is 90% similar. This helped a lot thank you

    1. 1

      No problem! I'm glad it can be helpful!
      Feel free to send me a DM if you have any more questions or need any help! I'd be more than happy to provide some thoughts into it

  4. 1

    Great read thanks for share
    On question
    Why u use kaffeine if you already use uptimerobot which pings your website in every 5 mins

    1. 1

      Yikes. I meant to say that I am using only UpTimeRobot now. I don't use Kaffeine anymore haha

      I gave the wrong idea there :')

      1. 1

        ok thanks it is awesome

  5. 1

    Hi Jerry. Nice article and product. I also faced the same challenges with webscraping and decided to go with a scraping provider just like you after meddling for hours with chrome browsers..
    Did you also implement full text search functionality and if so how?

    1. 1

      Hey Caspi! Great question. To interact with the DB, I am actually using SQLAlchemy (ORM).

      Prior to using Postgres's similarity function, I was actually using the full-text search.

      In SQLAlchemy, I was using the plainto_tsquery method (which is a wrapper around the plainto_tsquery function in Postgres) to perform a full-text search. Here's a code snippet from my previous implementation:

      def get_product_based_on_query(search_string: str) -> Optional[list[Product]]:
          logger.info(f'Searching database with user query: "{search_string}".')
      
          try:
              tq = func.plainto_tsquery('english', search_string)
      
              return session.query(Product) \
                  .filter(
                  or_(
                      Product.brand.op('@@')(tq),
                      Product.style.op('@@')(tq),
                      Product.name.op('@@')(tq),
                  ),
                  and_(Product.updated_on >= datetime.utcnow() - timedelta(days=LAST_N_DAY_DATA)))  \
                  .order_by(Product.price_per_quantity) \
                  .limit(50) \
                  .all()
      
          except Exception as exception:
              logger.exception(exception)
      
          finally:
              session.close()
      

      I eventually switched to the' similarity' function to cope with typos, providing a better search experience for the users. Here's the code snippet of the actual (also current) implementation.

      Here's a suggestion though:

      If cost is not much of an issue, I would use Elasticseach (ES) or any Seach as a Service (i.e. Algolia or Appbase), whichever makes more sense for you) rather than making these queries directly to your DB. The advantage of this is that ES could act as a caching layer for our app while also making the search much faster with a tonne of added search-related features.

      1. 2

        Hi Jerry.

        Yes I agree elastic search and algolia are the kings on search engines.
        I was just curious of it because I have not implemented search before. I have setup a poc of a elastic search server, but haven't really used it properly because it is so time consuming to work with.

        1. 1

          I'm looking forward to what you are building!

          1. 2

            @jerrynsh Thanks. I am building a website to analyze sentiments from Twitter and Reddit.

          2. 2

            Hey guys. Shameless plug but we are building an Algolia/ES alternative: https://www.relevance.ai/discovery if you're keen on an Indiehacker alternative :)

            Jerry, great article. Enjoyed reading it, and have been working on my own scraped search engine project too (for startup podcasts). The value add of aggregation is fun to play with in a field you're passionate about.

            1. 1

              @danp It seems promising. I will have a look if I need to implement a search at some time. How does your service compare to Algolia?

              1. 1

                Our basic search out of the box is very similar to Algolia. Rapid search as you type (even works with their InstantSearch components). Bit easier than Algolia for filtering and sorting (no need to re-index). We also offer 100K requests on our free plan rather than 10K.

                Where we differ though is in how we handle more complex search. We use AI to handle synonyms/slang/content where Algolia uses manual rules and mappings.

Trending on Indie Hackers
Why Indie Founders Fail: The Uncomfortable Truths Beyond "Build in Public" User Avatar 138 comments Your AI Product Is Not A Real Business User Avatar 86 comments The Clarity Trap: Why “Pretty” Pages Kill Profits (And What To Do Instead) User Avatar 34 comments I built an enterprise AI chatbot platform solo — 6 microservices, 7 channels, and Claude Code as my co-developer User Avatar 28 comments I got let go, spent 18 months building a productivity app, and now I'm taking it to Kickstarter User Avatar 17 comments I went from 40 support tickets/month to 8 — by stopping the question before it was asked User Avatar 16 comments