I Thought Choosing the AI Model Would Be the Hard Part. I Was Wrong.

I've been building Observer, an AI companion for meetings, and one thing has surprised me more than anything else.

I went into this thinking the biggest challenge would be choosing the right AI model.

It wasn't.

The biggest challenge has been latency.

When someone asks a question during a meeting, you don't have 10 seconds.

You have maybe 2–3.

That means the model is only one piece of the puzzle.

Every stage matters:

• Capturing the screen

• Compressing images

• Streaming and transcribing audio

• Building context

• Model inference

• Rendering the response

I've found that shaving 300ms from multiple parts of the pipeline often improves the experience more than switching to a newer model.

Users don't tell me, "The reasoning is amazing."

They tell me, "That was instant."

For anyone building AI products:

Have you found latency to be a bigger challenge than model quality? Or has your experience been the opposite?

I'd love to compare notes with other builders.

Note: Waitlist open at getobserver.app. Launching soon.