2
2 Comments

Content crawler for Content & SEO Marketers

Coded a very small MVP for product research purposes. This app is supposed to be a web version of the Screaming Frog SEO Spider desktop tool.

Potential advantages over a desktop tool:

  • Save results in a database and these results will be available to the user on a web interface with interactive tables - no more clumsy spreadsheets.
  • Run on the cloud and be sent an email when the crawl is complete, no need to keep your computer running while doing web scans.

Pretty please check it out here, https://thecrawlbot.netlify.com/ and let me know if this project is worth pursuing.

  1. Will you use this product over other tools?
  2. Will someone else you know use this product?
  3. How much will you pay for this tool?
  4. What other features would you like to see on the app?
  5. Are you a Content or SEO Digital Marketer?

Thank you all!

on October 11, 2019
  1. 1

    Hi,
    I could be on the target because I'm using screaming frog everyday because it's a very powerful product (seo, crawler, scraper, error...)
    So I can be interested if your tool is as powerful + cheaper than the cost of a windows vps+annual licence fee.
    There is a competitor of screaming frog for a web version, but this tool doesn't have all the features of screaming frog.

    1. 1

      That sounds great, Tim.

      May I know what's the web version competitor of screaming frog.
      Also what features would you like to see that is not in screaming frog? Thanks!