I run aitrainer.work, a career platform for people who do AI evaluation work. Most visitors arrive when they need a new project and leave once they find one. That works for them, but a site that only gets visited during job hunts has a hard time growing.
Today I launched a daily AI Vibe Check game on the same site. Every day two marionettes, Calca and Brina, answer the same prompt. Both are AI models told to sound like a person. You read both answers and place them on a 7-point scale. It runs from "Calca sounds much more human" to "Brina sounds much more human", with "No difference" in the middle. After that you see how everyone else voted, and you can tag what felt off if you want to.
Today's prompt asks each of them to “Write a text to your flatmate asking them to stop leaving dishes in the sink, without starting a fight.”. Players can also suggest their own prompts to be added in the future, once per day after they play.
Four goals drove the build. I want people to come back daily, I want data nobody else has, I want the site to offer more than listings, and I want to stand out from the crowd, since my site has suddenly lots of competitors and copycats (good luck "stealing" this).
The daily loop is the part I am least sure about. If you play today's round, tell me whether anything on the final screen would bring you back tomorrow.
What made you pick this stack over the alternatives?
Appreciate the honesty here, most people only share the wins.
This is useful. How are you finding your first users so far?
Interesting. How are you measuring whether it is working?
Interesting approach. What was the hardest part to get right?
Interesting approach. What was the hardest part to get right?
The measurement shift is the real insight here. A job board's core metric used to be "jobs filled" or "visits during job search season." But that's a transactional metric—users show up when they have a specific need. The daily game reframes it to "habit strength" and "non-transactional return rate." Once you switch metrics from "did they find a job" to "do they come back tomorrow," you unlock retention that competitors can't copy with listings. The data loop you get from the game (which model sounds human) is actually a side benefit.
Giving people a reason to visit between job hunts is a smart fix for a site people only need now and then. The 7-point scale keeps it light and easy to come back to.
Played it. The sink prompt is good — that “without starting a fight” bit is exactly the kind of thing people actually write.
Final screen is the weak spot tho. Leaderboard of votes is fine for 20 seconds, then I’m out. Nothing there makes me go “aight I’ll come back tomorrow.” If you flashed tomorrow’s prompt teaser or a streak (“you’re 1 day in”) I’d hit it again. Right now it’s just “cool, other people also thought Brina was mid.”
Data play is smart. Daily loop still needs a reason besides curiosity. Streak or “your prompt might drop tomorrow” would do more than the results page.
I actually love the idea of the prompt teaser for the day after. Guess what? I am implementing this right now. thank you!
Thanks for sharing the numbers, that makes it much easier to follow.
Thanks for sharing the numbers, that makes it much easier to follow.
Great progress. How did you get your first few users?
Thanks for sharing the numbers, that makes it much easier to follow.
Gamification for a job board is such an underrated growth hack. The Wordle-style daily hook is perfect for AI trainers because it directly taps into their professional curiosity.To maximize retention, you should definitely add a 'Share your streak' button with a clean visual grid (like Wordle does). If AI trainers start flexing their vibe-checking streaks on LinkedIn or X, your organic traffic will absolutely skyrocket. Genius move!
From the SEO side, the thing I'd think hard about is whether each day's round gets its own permanent URL. A daily game mostly brings people back, which you want, but if the prompt, both answers and the final vote split stay up on their own page, each day also adds an indexable page with original data nobody else has. "Which of these sounds more human," with the vote split from real people, is exactly the kind of fact that gets quoted. The risk is thin pages when a round only gets a handful of votes, so maybe only keep rounds past a minimum vote count. Are you keeping past rounds, or replacing the page each day?
Yes I did think about this! I am not looking of the SEO perks of this as "add more random content" but rather "do something new and original" and collect data nobody else has. I have a research background myself and I am looking forward to conduct more original research studies with findings I can publish that hopefully can lead to increased authority and backlinks from other sites.
That's the stronger version of the idea. Original research earns links in a way daily content doesn't, because other people cite a finding, not a page. One practical thing: a single clean number travels much further than a dataset. Once you have enough rounds, a headline stat like "people rated the more polished reply as less human X% of the time" will get quoted far more than the full results page, as long as the page behind it shows the method and sample size. We have the opposite problem: no original data of our own, and a domain authority of 3 to show for it. How many votes per round do you think you need before a finding is publishable?
This is great work — reminds me of some of the calls I've had to make building Xstream4K. What would you do differently if you started over?
I always like to lauch things quickly and improve them later, hopefully this launch is solid enough I won't have to re-touch it too much in a few days.
What behavior would convince you the game is strengthening the job board rather than creating a separate traffic loop—daily returns, job searches after playing, or something else?
If people actually play and we get enough interesting responses so that I can draw some insights for my research, that'd be enough!
If the goal is research, what would tell you the game is helping the job-board side rather than just generating interesting responses?
well my site isn't just a job board, it also has lots of learning material and an academy for ai training and data annotation, i think of this as an ancillary product to that rather than the job board itself.