Why do people even like Product Hunt?
In October 2020, as part of our launch of OpinionX on Product Hunt, we opened a dynamic survey to understand what Product Hunters love about PH and what they would change about their beloved community if they were given the chance.
If Reddit and Typeform had a baby, it would be a dynamic survey. People respond to open-ended questions and vote on other people's opinions to surface the statements that matter most to everyone. It's a giant big blend of survey and community.
We facilitated hundreds of votes on opinions submitted by Product Hunt users. Here are some of the highlights...
Declining Engagement
The majority of participants acknowledged a recent decline in products posted to Product Hunt. This aligns with Exploding Topic's keyword search data on Product Hunt which shows a 57% decline over 5 years in searches for the platform. Keep this in mind as we explore other opinions that surfaced in this OpinionX survey.
User Intent
78% of voters on the statement below signed up to Product Hunt to launch their own product. This proves the significance of Product Hunt as a milestone in a startup's lifecycle, however it also brings to light a potential lack of general visitors engaging on Product Hunt outside of those launching a product.
Considering this opinion was the second most important to participants, Product Hunt team should consider trying to rejuvenate the community beyond those actively launching their product. On the other hand, 80% of participants agreed that they scroll on Product Hunt when they're bored, so there are still some general users of the platform.
Discovery
There's a positive sentiment overall about how Product Hunt helps users to discover innovative new tools and resources. Multiple opinions mentioned the enjoyment of this discovery process and the possibility of finding something cool and useful.
Community
The third most important opinion to participants highlighted the value Product Hunt users place on community, a topic that was mentioned throughout the opinions submitted.
An opinion about the Product Hunt forums was one of the few statements with unanimous consensus. While its lower importance score indicated a possible low rate of existing awareness of or engagement in the PH forums, participants gave a clear indication of their view of this Product Hunt feature.
Exposure
100% of participants agreed that Product Hunt offers a big opportunity to gain early exposure, traction and users for their new products. As we already recognised, this is Product Hunt's primary driver of user acquisition
Each of these insights are positive indicators that Product Hunt is achieving what it originally set out to achieve - a community for sharing and discovering products.
Criticisms
As we have seen, participants covered many of their favourite aspects of Product Hunt which had high levels of agreement across the board. The OpinionX survey also highlighted areas that participatns felt Product Hunt could improve on.
The statement with the highest importance (below) revealed that many people feel Product Hunt has 'diverged from its core' because of the number of larger companies using the site to promote their launches. 'Diverged from its core' suggests that users feel Product Hunt was originally a community for startups but that the move to prioritise larger companies launching new features and products has been an unwelcome change. With 78% of survey participants agreeing, it's clear the majority would prefer to see Product Hunt prioritise smaller companies.
The lack of transparency around the homepage ranking algorithm was voted the 5th most important opinion by participants. Product Hunt acknowledges that votes alone do not contribute to your product rising to the top of the homepage, however this statement indicates that users want to know more about how this algorithm works in order to promote more fairness.
Issues relating to the Product Hunt iOS app were highlighted by one participant. This opinion ranked poorly in importance, which could mean that it was not seen as a significant problem by many users. A segmented research approach could be used to dig deeper into this claim.
Crowdsourcing Solutions
OpinionX enabled participants to submit ideas as well as feedback about Product Hunt. 80% of participants wished they could find their network easier through a Twitter integration. While this former PH feature was removed some time ago, it is closely linked to earlier comments on community.
Upvoter Lists
75% of users would like to be able to see all of the people who upvoted their product. At the moment, Product Hunt only allows you to see a few people who voted at any one time.
Product Listing Dates
This participant added their opinion right before the survey closed, so we didn't get enough votes to know what others thought of it, but we're including it because they made a great point. Product Hunt serves as a great source of SEO for products (we still get a vote or two on our October 2020 Product Hunt listing for OpinionX), so it can be hard to know if something is new or old, dead or alive, when you search on the platform. This could easily be solved by an 'Created on:' label like most social media sites.
Asking the Community
Providing Product Hunt users with an opportunity to share their perspective through an OpinionX survey format reveals a wealth of insights about how users think about Product Hunt today and into the future.
The praise, criticisms and suggestions are point to Product Hunt's strong community of users that would love to have more of a role in setting the direction of the platform. High ranking opinions in this OpinionX survey show that community, transparency and supporting startups are core principles of the Product Hunt community.
This survey is still open, so feel free to vote on all the opinions you saw above and more here if you'd like to check the dynamic survey tool out - no email required.
I write new content every week about the future of UX research on The Full-Stack Researcher blog and newsletter. You can find out more about OpinionX, the dyamic survey tool behind this case study, here.
Favorite thing about Product Hunt: Free promotion for my product, got Jamform about 200 more users though none paying.
Second favorite: Free ideas for what to work on next, find a good idea but executed poorly and do it better.
Least favorite thing: So. Many. Bots. It's ridiculous how many of the comments are obviously not real people. On silly/joke products you'll still see tons of comments like "Wow been looking for this for years!", "Who's your competitors?", "This will save me so much time", etc. when it doesn't even fit what the product does. Even on my own launches so it's not like they're paid for bots.
Interesting your comment on no paying users.
A lot of articles recommend going through a big process to post on PH - find a maker, post at the right time of day, animated GIF logo, special deal for PH users, etc. However, what they usually don't mention focusing on a goal:
Hearing that a lot of upvotes and comments are from bots is concerning and makes the effort of posting on PH possibly a distraction. You're not going to get paid sign ups from bots!
I tried for 2 month the Product Hunt ShipIt product and there were a lot of fake signups. However, one paid user did come from it.
TLDR;
#3 is important to have real users, and not just commenters. I'm obsessed with our paid user feedback and do everything I can to do what they ask. Non-paid user I collect and if hear multiple times it means it is important.
General comments by people who didn't try the product are useful to understand what is roadblocking them - could be truly something missing or they might not be your target audience. For example, if you get a lot of comments from 20 year olds on your retirement product, is that take-to-heart feedback?
This comment was deleted 3 years ago.
Having read so many blog posts and articles on Product Hunt launches, I was surprised that bits never came up at all. They weren’t really mentioned in anyone’s post-launch tips or notes. When we were launching I even sent PH an email with screenshots showing one of the top products that day was 80%+ bits and they just ignored me 🙃
This comment was deleted 3 years ago.
Thanks for sharing your survey. We've been looking at launching at PH for some time now, but haven't had the time to do it. Because we ended up focusing on product development and cold outreach campaigns.
We already have a working product with thousands of users though, so it's not like really "launching". More like trying to get more exposure and awareness through PH.
You survey and research here will definitely help!
This is very interesting! The one issue I've had with Product Hunt myself is that it is very tech-community focused. Nearly all the apps on their target that demographic, and I've found that this leads to very similar products being launched all the time.
Although it's a great platform for up-and-coming products, as someone who is a Product Hunt watcher & not someone posting there, I'd love to see more different kinds of ideas showing up there.
This comment was deleted 3 years ago.
Hey Anil, wikisurveys are a research method with significant academic study. The voting materials are crowdsourced and appear in a different order for each participant.
Participants don't just engage based off of agreement votes. Pair choice voting drives the importance metric, which again is a research method with significant academic weight.
The vast majority of our own votes on our Product Hunt launch came from our personal networks, based on a huge promo that we did to drive traffic to the Product Hunt home page to vote for us. We never really managed to break into the PH community on the day unfortunately, although it seemed like there wasn't much organic traffic there anyway. The few upvotes that weren't from our own network were indeed quite spammy looking (they were just upvoting every account). I also saw 2 products that were just blatently paying for bots to upvote them.
The statement referenced in the post above about upvoter lists, as with all statements in the survey, were submitted by survey participants (we had email verification on for the duration of the survey too btw, I just removed it now that we finished looking at the results). These are all user generated opinions, not pre-populated questions by us.
This comment was deleted 3 years ago.
AllOurIdeas is indeed an example of a wikisurvey. I did a literature review of wikisurveys while in university, including the Salganik and Levy paper you linked (it’s a great read!)
There are many, many types of wikisurveys, including examples of ones with agreement-based voting mechanisms that are supported by academic review. I’ve studied these tools and the research papers associated, but I’ve also met members of their advisory boards, people who have used the tools at scale and the creators of the tools themselves. It’s a new concept to most people, so I understand that it might seem like it goes against many established research principles.
As I mentioned in my last comment, we removed participant validation so that anyone reading the case study post could play around with the data. Also, wikisurveys aren’t just designed to be single use tools like normal surveys - they’re designed for you to join back in multiple times. Even with participant validation removed, your still rejoining the same ‘user account’ unless you try to avoid it (eg via incognito).