April 19, 2019

Anybody Interested in A Real-Time Bot Filter


How many indie hackers out there would support or know somebody who would support the development of a new technology? I've been developing a real-time filter for HTTP requests that have an IP address registered to several proxy sources around the web. The filter works very easily: Simply put a JavaScript tag in the header of the web-pages you want to filter proxified web requests from, and my remote filters scan that request, and redirects to a 404 if its IP is either missing (tor) or belongs to a proxy IP registrar. Adding it to your site is like adding Google Analytics or CallRail, but it scans the IPs as they come in.

I am a great engineer. But an absolute rubbish marketing person and don't even know if there are those out there who would care about or want such a technology for free, let alone pay for it. Any advice from the community on how to get early adopters and subscribers off the ground? Any help would be greatly appreciated!

  1. 1

    I don't think that injecting a js into a website to prevent scanning is a good idea. It'll make the website slower. I would suggest some more passive way, we parse logs real time and block bots ip with max 3 seconds delay. This enabled us to build complicated algorithm that could check if it's a bot, fake bot, a whole class C scan. My websites were protected with this tools for many years and only botnet scans could bypass that.

    1. 1

      Makes sense to me and that is a thought I had also. I'm not a huge fan of adding JS to a site when it isn't necessary but it seemed the most marketable idea.

      The other implementation I have for it is to filter at the web-server level. With NGINX, for example, we include a second .conf file that updates every 10 minutes with known proxy and blacklisted IPs. The entire file is a series of explicit disallows. I believe that could offload the work to the server rather than relying on the browser and would allow for more stable performance levels. What is your opinion on that?

      1. 1

        I use firewall for it, it's the cheapest way to block, but it's not a marketable solution.

Recommended Posts