I'm building a meeting assistant with no bot in the call. Audio was the hard part
Most AI meeting tools send a bot into the call, record the whole thing, then email a summary afterwards. I wanted help while people are still talking. Transcription runs live, and suggestions show up on your own screen: a clarification, a question that might be worth asking, a word that's missing because the meeting isn't in your language. The rest of the call doesn't see them.
No bot in Zoom or Teams. The app runs in the browser and only uses audio chosen in the share dialog.
I didn't get stuck on the LLM. The weeks went into something dumber: where the audio actually comes from.
Browsers don't treat the three sharing options the same way:
- A Chromium tab has audio on any desktop OS, and the checkbox is usually already ticked.
- Full screen can include system audio, but only on Windows and ChromeOS, and only if that box gets ticked by hand.
- A window: no audio. Ever.
That last one is the trap. Share the Zoom window, the preview looks right, the stream shows up, the session is silent. The browser doesn't warn. Nothing in the code can detect it either. Mac and Linux have no system audio at the browser level. Run the meeting in a Chromium tab and share that tab, or install a virtual audio driver. I'm not going to ask a first-time user to do that.
I request system audio explicitly when I call getDisplayMedia. The flag doesn't create a capability the platform doesn't have. A lot of the remaining work is UX: which button to click in a dialog I don't control, and saying that Chrome or Edge on desktop is the setup that actually works.
I also got the trigger wrong at the start. A suggestion on every finalized transcript is too eager. Conversation doesn't arrive in tidy chunks. Someone says "I think the real issue here is..." and the point only shows up in the next sentence. Generate too early and the model answers half a thought. Do it on every transcript and the screen is just noise. It now waits until enough new information has shown up since the last suggestion. The first one of a session has a lower bar, because an empty screen looks like a broken product.
Still pre-traction, so no revenue number to post here yet. Shared audio does make it all the way to a suggestion on screen. I'm spending the time on when to generate, and on telling people which OS does what, before anyone hits a silent session.
It's at livesuggest.ai.
If you've shipped anything that depends on getDisplayMedia, how did you handle the Mac case? I haven't found an answer I'm happy with.
The trigger timing problem is change detection, not scheduling. A timer fires whether or not anything meaningful happened. A change threshold fires when enough new semantic content has accumulated. The difference matters because pauses in conversation carry information — someone thinking is not the same as silence, and generating during a thinking pause makes the tool feel tone-deaf.
We hit a parallel in SEO auditing. A page that looks indexed but gets zero impressions produces no error. The site owner checks Google's index report and assumes everything works. We ended up front-loading the "this is broken and here is why" message before any data, because users who see green metrics first never scroll down to the red ones.
Your documentation burden is the same shape. The person who needs the warning will only see it if you show it before they hit the silent failure, not after.
The core problem here is visibility - not of audio, but of capability. You've discovered that the system can't tell you what it can't do (the silent share trap), so you have to design backwards from what users will actually see on screen. Most AI tools hide this - they assume the platform works the same way everywhere. Your UX solution (explicit os-level guidance) is really measurement made visible: "here's what this setup can actually measure," which becomes the signal for which path to take.
The silent window share would make me close the tab and blame the product. Preview looks fine, so I think the model is dumb, not the browser. I hit the same wall capturing a call in Chrome on a Mac. System audio just isnt there unless you set up a virtual cable, and nobody is doing that. Are you telling people to share the meeting tab only, or is window share still in the picker? If window is still an option I'd hide it. Users pick the thing that looks right.
The trigger timing seems more consequential than the audio plumbing. Have early users acted on suggestions often enough to validate the current generation threshold?