2
1 Comment

ML Web-Crawling

I've built an app which would benefit immensely from integrating ML webcrawling.

But I'd like be able to intelligently navigate of webpages toward an objective. This'd unlock a lot of useful applications. And it's feasible.

Background: I've done a lot of procedural webcrawling (puppeteer, selenium, straight up js - particularly on sites that seek to be crawl-proof). And I've solved captchas autonomously within a webcrawling app using w/ pytesseract. I'm also aquatinted w/ pytorch + brain.js + kaggle.

Any data scientists/devs who'd fit in collaborating to solve this problem (or a subset of it), hit me up. Looking to work with those with a track record of executing/getting things done (rare). :)

-Blake

ethdevs@gmail.com
discord: deepb#4778

on February 19, 2020
  1. 1

    Not related, but what are you thoughts on scraping linkedin? feasable at scale?