Ideas and Validation February 13, 2020

Bot, or not?

Alister Bulman @alister

One of the frequent banes of my life as a sysadmin, and developer, that runs his own servers, are all the bots that crawl and outright attack my servers. I wondered - who else finds it takes time and effort to try to avoid the comment-form spam, and all the other work involved in keeping things clean and working.

So, if you do run servers, I'd like your thoughts of this (currently skeleton) idea. I'll be filling in some more details in the next few days as to what, and how it does it.


  1. 1

    I've thought about various forms of this (identifying spam or malicious anything, I come from an insurance compliance background). The challenge I always run into is that as soon as you offer a filter as a service and incoming data/definitions it becomes possible for a malicious user to abuse your service.

    Worst case scenario: you become massively popular and everyone relies on your service to identify spammers. Malicious users identify this and send a bunch of false reports that XYZ is a spammer. Now XYZ has been blocked by outside, malicious entities.

    As @ehacke said, I think this is valuable but challenging.

  2. 1

    I think if you could do it, it would be valuable.

    But, it feels like if this was something that WAS doable, it would already exist.

    I think most websites rely mostly on heuristics and behavioural data to do this stuff, and that most of that is pretty specific to an individual site.

    That said, one way you may be able to make this work is to target specific platforms and cater to that narrower problem. Then you don't have to solve bots everywhere, just on Etsy (for example).