2
5 Comments

I promised to report what each channel converted. 18 days in, I can't, and that's the post

Said I'd come back with real numbers, so here they are.

First, the promise I can't keep: I said I'd report what each channel converts. I can't. I never instrumented attribution before launch, so I have placements and I have signups, and no honest line connecting the two. That's the most expensive mistake of the two and a half weeks and it cost me nothing to avoid. If you're pre-launch reading this, that's the free lesson: an hour of attribution work before launch day might be worth more than anything you'll do on launch day. I got so busy just trying to get content online I skipped some steps.

What I do have is the activation funnel, which turned out to be the more interesting half anyway.

- 31 signups since launch

- 18 of them opened a lesson

- 8 of those at least wrote a line of code in it

- 4 actually finished a lesson

- 0 paying

So 13 people signed up and never opened anything, and another 10 opened a lesson and stopped before typing. The whole business is downstream of that second group. I spent two and a half weeks optimizing the top of the funnel while the leak was in the first five minutes of the product, which I could have measured on day one for free.

Costs: $0 on ads. Total inference cost of every lesson anyone has run since launch, on my own key, is 31 cents. The free tier is capped at 10 lessons a month and nobody has come close to hitting it, which kills the "raise the limits" theories.

Channel placements, for whatever they're worth without conversion data: Show HN got 4 points. Product Hunt finished 83rd. A directory called Uneed that I hadn't heard of a month ago put us 3rd for the day. The pattern I did not expect is that the twenty-minute directory listings did more for me than the two launches every founder optimizes for. I'm not a social media native and it shows, so take that as one founder's data point rather than advice.

The other honest thing: on the Team tier question I asked in this thread on day 3, I still don't have an answer, because I have no upgrades of either kind to learn from. What I have instead is a hypothesis I'm now going to test directly. Through September I'm taking a few engineering teams as onboarding pilots, hands-on setup from me, in exchange for real feedback that isn't filtered through politeness. If self-serve was never going to produce Team accounts at this volume anyway, better to find out from ten real teams than from another month of waiting.

Next 30 days: fix the first lesson, instrument activation properly, and stop treating launch days as the job.

Question for anyone further along: how did you instrument activation early without building an analytics project instead of a product? I keep debating myself about a week of tooling I don't think I can afford right now.

posted toAvatar for product CodeTrain
CodeTrain
  1. 2

    One thing I like here is how quickly the bottleneck changed once you had real users.

    Before launch, the obvious question is "How do I get people in?" After launch, the evidence can point somewhere completely different. That's one of the few advantages small products have—you can let reality replace assumptions before you've invested months optimizing the wrong constraint.

    1. 1

      That is a good way of thinking about it, and I am trying to handle this more like a love project than a commercial product, even though it has huge enterprise/teams value

      1. 1

        That's an interesting tension. Sometimes keeping something as a love project protects exactly what makes it valuable, while enterprise demand can gradually pull it toward a very different product.

        Curious how you're thinking about that trade-off right now.

        1. 1

          A deep question for sure, and the genuine answer is probably no idea! lol I honestly don't mind solving that bridge once I hit it. I will say, I am passionate enough about helping others use AI in ways that is good for your brain that if I could replace my day job with taking car of and maintaining CodeTrain it would be like a dream job for me.

          1. 1

            That’s actually the interesting part — not having the answer yet is probably where the real tension starts.

            I’d be interested in continuing this conversation outside the thread. What’s the best email to reach you on?