Analytics project
I want build analytics project. Got idea for specific niche and I know what I want to build. Problem is that I do not know where to start in terms of technology. I am web analyst and got experiences with website coding, html, css, bit of js, php and mysql.
Am I correct that Web Scraping and Web Crawling methods are used on build projects like wsbsentiment.io, brandwatch.com, explodingtopics.com?
Some scraping (which is usually against terms of services), but probably some percentage of:
Also note that scraping/crawling is getting harder nowadays since services like cloudflare automatically throwing captchas for unusual traffic.
You could use the web archive dumps if you can handle a lot of data.
Thanks Jehna, this will help