Show IH: A Twitter search engine to find relevant Indie conversations
Hi everyone!
I wanted to share with you what I’ve been building this month.
IndieTweets is a search engine for the indie maker community on Twitter. I built it out of my own desire to discover new people outside of my basically inexistent Twitter presence (what a lonely place it is when starting out!).
You can find more about how it works here https://indietweets.com/how-it-works.html , but basically it creates a social graph of Twitter accounts belonging to indie makers and then lets you search what they are talking about! No need to login with Twitter, follow people, create lists... Just type what you are interested about (your product, your market, whatever!) and see if any fellow indie maker has talked about it in the last few days. From there you can take the action directly to Twitter and engage with the community!
I decided to open it to other people because in the last month it has been extremely useful in helping me find interesting makers and researching/marketing my own ideas.
As you can see it is very early stage and the index is far from complete (around 2K people at the moment), but I am curious to have other people play with it and help me improve the tool!
I am actually thinking of making a product out of it, but there will be time for that.
Feedback is more than welcome and happy indie searching!
I almost forgot: you can check if you are already part of the indexed social graph at this link https://search.indietweets.com/am-i-in . If you are not and do not want to wait for the algorithm to discover you (the Indieverse is in constant expansion and IndieTweets’ social graph is as well!) just let me know in the comments or write @gtardini on Twitter!
This is great, always looking for ways to connect with other Indie Makers. Thanks for building, bookmarked!
Thanks! Many improvements on their way, that bookmark deserves more reasons to be clicked :)
Forgot to say, right now I am using Twitter API (very limited, unfortunately...), but I am all for scraping so if you feel like you would like to use the data just contact me and I can either open an API account for you on my Elasticsearch cluster (Appbase.io, amazing tech) or give you my blessing for web scraping the page! Just use some timeout like 1min between requests :) Actually, you already have my blessing for scraping, go for it.
This comment was deleted 3 years ago.
The whole download&processing is done in Python, with the evergreen Pandas as main workhorse and Spacy for the more NLP-intensive parts. NetworkX is the library I use to do the graph modeling. So, Python up to here. The results are then REST-sent to Elasticsearch by the Next.js application which powers the web app reachable at the URL. To recap, Python for the API interaction and the data preparation/modeling + Next.js (Node and React) for the actual webapp! I would like to streamline the whole thing to Node but Python is just unbeatable for data wrangling
This comment was deleted 3 years ago.
I basically use everything I could find to get around Twitter rate limits, so a combination of both standard search and streaming API + some experimentation with the lists as you suggest, I was so frustrated when I hit the undocumented limit (and it took at least 48hrs to unlock, but I admit having been pretty pedal to the metal)! At the moment my plan is to cap the Twitter API in the way I am using it right now, then go deeper down the lists route to free some buffer while I look for a better way (maybe premium APIs or straight to scraping). Thanks so much for the suggestion, it's always a pleasure to find another rate-limit struggler :D
This comment was deleted 3 years ago.
This comment was deleted 3 years ago.
Thanks! Would have written to you some time later but thanks for commenting, your Connect list was very useful for seeding initial users to the graph-building algorithm :)
This comment was deleted 3 years ago.
Amazing, I'm glad it's out of my apartment and out in the wild. Keep up the good work on the Data group, I just joined and am planning to follow it closely!