Should AI builders ask questions before they start building?

Probably deciding when the AI should ask another question and when it should just start building.

Too many questions and it starts feeling like filling out a form. Too few and the tool can confidently build the wrong version of the idea.

We recently gave the exact same booking-app prompt to nine different builders, and only one actually asked questions before building. Seeing how different the outputs became made this problem feel much more real.

I wrote up the full test on built.new Learn if you want to see it.

Appreciate it. I’m still thinking through where the right balance is between “help me think” and “just let me build.” I think AI tools are going to have to get much better at knowing when each one is appropriate.

Yeah, especially if the starting point is vague.

If someone already knows the exact users, workflow and rules, I wouldn’t make them answer ten unnecessary questions. But something like “build me a booking app” hides a surprising number of product decisions.

We saw that pretty clearly when we tested the same booking prompt across nine builders. Some produced polished screens, but the actual booking logic behaved very differently.

That test is here if you’re interested in the details.

I actually didn’t share any numbers in this one 😄 This post was more about whether AI should understand the product a little better before it starts generating it.

But numbers would make an interesting follow-up once we have enough real usage to learn from.

Still too early for me to honestly call one channel the winner.

Right now I’m more interested in founder communities and build-in-public than chasing raw traffic. The conversations tell you a lot more when the product is still taking shape.

We actually analyzed 57 posts from Base44’s founder recently, and one thing that stood out was how much better lessons, real numbers and things that went wrong performed than normal feature announcements.

We’re still early, so I’d rather not pretend we’ve cracked distribution yet.

What seems most useful right now is being around people who are already building products and hearing the same problems come up repeatedly. At this stage I value ten useful conversations more than a random spike in traffic.

We’re actually still pre-launch with built.new, so I’d describe them more as potential early users than a big active user base right now.

Most of it is coming from conversations with founders and people already experimenting with AI/no-code tools. That’s useful because we’re not only finding people who might try the product — we’re also learning what they’re frustrated with before we open it up.

Mostly very manually so far.

Talking to builders, sharing what we’re learning, and trying to make the research useful even before the product is fully open.

I’m becoming much more interested in build-in-public as “teach what you learned” rather than just posting feature updates. We recently studied how Base44 did this across 57 founder posts and that distinction was surprisingly obvious.

We’re still pre-launch, so most of the feedback right now comes from conversations and testing rather than a large live user base.

One theme keeps coming up though: people don’t only struggle with building the app. They struggle with defining what they actually need to build first.

That’s a big reason I wrote this post. Sometimes helping someone remove three unnecessary decisions before the build is more valuable than generating three extra features for them.
https://built.new/learn/build-in-public-playbook/?utm_source=chatgpt.com

This post actually wasn’t about choosing a specific tech stack.

I’m more interested in what happens before that decision: does the tool understand enough about the user, workflow and scope before it starts generating?

The stack obviously matters later, but you can make a technically good build of the wrong product too.