We turned off paid ads. Daily sign-ups fell about 90% within two days. Here's what we're trying instead.
I'm building Mythex (https://mythex.ai/?ref=ih), an AI app builder: you describe an app in chat and it writes, runs and publishes it.
While the ads ran, sign-ups looked healthy and steady. Then we switched the paid ads off, and within two days daily sign-ups fell by roughly 90%. So nearly all of that "steady growth" was the ads. We hadn't noticed, because we weren't looking at sign-ups by source. That's the first lesson, and it's an embarrassing one.
What we've tried in the week since, and what happened:
- Directories: slow. One free launch site scheduled us for January 2027. Two others have had us "pending review" for about a week. Worth doing once, not worth waiting on.
- A community profile got suspended an hour after a product post from what looked like a brand account. Fair enough. Lesson: on personal-profile platforms, help first, and only mention the product where someone is choosing a tool.
- X replies: a lot of work for few views, unless the thread is someone actually asking what to use.
- Long comments on posts here and on Dev.to with 0-1 comments: the only channel that reliably talks back. People reply, ask follow-ups, and one maker asked us to look at her app.
- Tutorials: one long, practical post a couple of times a week (this week's is building a client portal and testing it with two fake clients before a real one logs in).
What we shipped this week, because the product still matters more than the posts:
- a rebuilt phone workspace (chat, preview and publish on a phone)
- exports that no longer fail when one file in a project can't be read
- load balancer access logs, so any 5xx during a deploy can be traced afterwards
The question I'd love real answers to: if you went from paid to organic, what was the first channel that brought sign-ups you could actually attribute, and how long did it take to show up?
That January 2027 directory slot 😅 Apparently even getting discovered has a waiting list. Did the maker who asked you to look at her app end up trying yours, or was it more of a feedback swap?
The first channel I would trust here is not daily sign-ups, it is a published app. Ads can fill the signup chart in two days and still be people who never hit publish. Split the week before you turned ads off versus the week after by one event: signed up and published something within 24 hours. If that count barely moved when sign-ups fell 90%, the leftover organic (the maker who asked you to look at her app, plus the client-portal tutorial) is the base worth waiting on. Directories queued out to January 2027 will not answer this quarter, so tag the tutorial and that follow-up with their own refs and judge them on published apps over two or three weeks.
No paid-to-organic story of my own to share yet, but two things that could make attribution show up faster:
Give every organic touch its own ?ref= (you already do it for IH) and add a one-line "How did you hear about us?" field at sign-up. Self-reported source catches comments and word of mouth that analytics usually files under "direct".
Track AI assistant referrals (chatgpt.com, perplexity.ai and similar) as their own source. For an AI app builder, that's where a lot of "which tool should I use" questions now happen, and it rewards exactly what you're already doing: specific tutorials and being mentioned in threads where people are choosing a tool. Shopify said on its Q2 2026 call that AI-driven traffic to its merchants tripled year over year, so it deserves its own line in the dashboard.
And your lesson #1 (sign-ups by source) is the one most people learn too late. Thanks for being open about it.
The 90% drop is the most useful number in this post, because it tells you the real CAC. When you were running ads, every sign-up felt like growth. It was spend. The organic number after switching off is your actual baseline.
We’re at DA 3 with four visits a month from search, so we skipped ads entirely and went straight to community comments. 401 comments across IH, PH, and Reddit over five weeks. Reply rate is low — 31 real replies from 401 — but those 31 have been more useful than any other channel. Two asked direct product questions, one changed their product framing after the exchange.
Your finding about long comments on low-comment posts matches exactly. High-comment posts bury you. Low-comment posts treat every reply like a conversation.
First attributable channel hasn’t arrived yet. But comment-first builds context that shows up when someone does search your name.
Appreciate the honest breakdown! That 90% drop is painful, but catching it early is a huge win.
In my experience, technical tutorials and build logs targeting specific workflows tend to compound the best over time. They take longer to rank or gain traction, but the intent of visitors coming from those channels is night and day compared to quick directory hits. Looking forward to seeing how your client portal tutorial performs!
Have the channels generating replies actually produced attributable signups or product usage yet, or is the signal still mostly engagement and conversations?
Community replies may be doing more than last-click analytics shows. I’d turn recurring follow-up questions into focused tutorials, then track assisted conversions over 30 days—organic often creates trust across several visits rather than producing a same-session signup.
The 90% drop is a strong sign that the paid traffic was carrying most of the acquisition, while organic demand and source attribution had not yet been built up. One useful move today is to add clean first-touch and last-touch UTM tracking to every tutorial, community post, directory listing and product link, then compare sign-up conversion by source instead of total traffic. I also work on technical backlogs, tracking and frontend layouts, and can help clean up the acquisition side while you keep shipping Mythex.
Brave experiment, and a useful truth to find early. One thing worth adding while you test organic channels: ask every new signup "how did you hear about us?" as a free-text field. With paid off, it's the only way to know which of the new channels is actually working.