My agent kept hitting the same wall. I'd ask it to track what's trending on TikTok and X, or monitor product mentions on Amazon, and it just couldn't get there. The data is all technically public, but agents can't read it natively.
So I built a skill for it. https://monid.ai/
Your agent can then read from X, Reddit, TikTok, LinkedIn, Google Reviews, Facebook, and Amazon. Works well for things like:
Morning briefings that pull what's actually trending
Tracking mentions of a product or topic across platforms
Market research before making a decision
Still early and would love to hear how it fits into people's existing setups and what breaks.
I would be really nice if as a user I could test it out. That said, I think API and scraper way you're building should work just fine for you. You're gonna need some sales muscle though.
While struggling with a similar issue, I built this open source MCP to create a persona with real accounts and do web hunting with persistent sessions. I would love your feedback: https://github.com/fellowship-dev/navvi
Have you validated this yet?