I launched 2 Months ago, Product Hunt put me on #7 for the day over 100+ upvotes insane for someone who was just building something to solve my own troubles.
I found evaluation of prompts tiring job, run the same prompt across mulitple models and keep tweaking it everytime to get the optimal results. From a developer lens I saw this as there has to be a way to do test cases for prompts, and test it against multiple models to check reliability and accuracy. As everytime a model gets deprecated we need to make the switch quick.
So I built promptperf.dev
While building the product I might have hung onto the thought of this product will make me my first million haha. On Launch Day seeing 100+ upvotes I started to see dreams of getting acquired for couple hundred thousand atleast lol.
Soon enough, I started to note that users were just signing up not sure how to use the app. So I iterated, and iterated again my metric for success became from signups to number of tests/evaluations run, so the goal shifted from getting people onboard to getting them to run the first testcase.
I now have 26 users who have ran their first test case. But only 7 have done it more than once. Its clear, users are not coming back, use it and dont need to keep coming back.
What this means, I should not target monthly pricing as clearly I'll be robbing the customer if they only have to use the app every so often. I've currently got Lifetime pricing at $49 which I think is fair given I need to calculate possibilities of the runs a user might make and keep the site up and running. I might need to implement $10, 10 runs plan or so I guess but since theres only been 7 actual users who decided to come back, Im putting it on the back burner as seems like I need to do a better job at explaining the problem to the users or clearly need to target more specific groups.
Really keen on some advice. (I tried Apollo for cold emails -> But you need to warm up your accounts before emails get sent out hence still not executed that step)
i checked out your website, i think i would find it really valuable if you would provide some "use cases"/"case studies" of what it can do to help get people's imagination flowing. like, when i look at your website - i get it (on an abstract level) and i don't get (on a concrete use case example level). maybe the top 3 use cases you envision.