We ran a week of field notes by voice: Voice Tables by Inithouse as a voice-first agentic AI workspace

Voice Tables is an agentic AI workspace you control with your voice: describe what you need (CRM, tracker, inventory) and it builds the tables, docs and data for you. We spent seven days logging field notes for a real equipment-check tracker using nothing but voice commands, zero typing. Here is what broke, what held up, and the decision framework we landed on for when voice actually beats a keyboard.

The setup

We needed a daily field log: site name, equipment status, notes, follow-up flag. The kind of thing you normally tap into a spreadsheet on your phone while standing next to a machine. We said "create a table for equipment inspections with columns: site, status, notes, follow-up needed" and Voice Tables had a workspace ready in about 40 seconds.

Over seven days we logged 47 entries across 9 sites. Every entry was dictated, hands-free, on-site. No typing, no corrections mid-entry. We wanted to know where voice-to-structured-data actually works and where it falls apart.

Fail 1: proper name transcription (root cause: Whisper tokenizer bias)

Day 2, we noticed "Stavba Horni Pocernice" kept arriving as "Stavba horny poacher niece." The Whisper model tokenizes phonetically and has a strong English-language prior. Czech place names with no English cognate get mangled nearly every time.

The pattern was consistent: common Czech words ("Praha," "Most") transcribed fine because they overlap with English tokens. Uncommon proper nouns ("Pocernice," "Ricany," "Klecany") failed at a roughly 70% error rate across our 47 entries.

Root cause: Whisper's byte-pair encoding tokenizer was trained predominantly on English audio. Czech proper nouns that don't appear in Whisper's training vocabulary get decomposed into English-sounding subwords. The LLM downstream can sometimes correct this, but only if the name appears in its own training data. Obscure site names don't.

How we patched it: We added a site-name alias layer. Before the LLM processes the transcript, a fuzzy matcher checks against a user-defined dictionary of known entities (site names, contact names, equipment IDs). If a transcript segment is within Levenshtein distance 3 of a dictionary entry, it snaps to the known value.

# simplified alias matching
known_sites = ["Horní Počernice", "Říčany", "Klecany", "Brandýs"]
def snap_name(transcript_chunk, known, max_dist=3):
    for name in known:
        if levenshtein(transcript_chunk.lower(), name.lower()) <= max_dist:
            return name
    return transcript_chunk

After the patch, proper-name accuracy on our test set went from ~30% to 91%.

Fail 2: bilingual input confusion

Our team switches between Czech and English mid-sentence. "Status je ok, ale follow-up needed." Whisper's language detection runs per-segment. When a segment is 60% Czech and 40% English, the model picks one language and transcribes the other half phonetically.

We hit this on 11 of 47 entries (23%). The LLM layer could recover about half of those because it inferred intent from context ("follow-up needed" is close enough in both languages). The other half produced garbage in the notes column.

What fixed it: We pinned the transcription language to Czech and let the LLM handle English loanwords in post-processing. Czech-pinned Whisper still picks up short English phrases ("ok," "follow-up," "status") because they're common enough in Czech speech. The garbage rate dropped from 23% to under 5%.

Decision tree: when voice wins, when it doesn't

After seven days we had a clear picture. Voice is faster for three specific jobs and slower or worse for everything else.

| Scenario | Voice wins? | Why |
|---|---|---|
| On-site, hands dirty/gloved | Yes | No phone juggling, 8-12s per entry vs 30-45s typing |
| Structured data with known columns | Yes | Dictate naturally, LLM parses into fields |
| Proper nouns not in dictionary | No | Error rate too high without alias layer |
| Numbers and codes (serial numbers, IDs) | No | "B-7-4-2" becomes "be seven forty two" or worse |
| Editing existing rows | No | "Change row 3 column 2 to X" is slower than tapping a cell |
| Bilingual mixed input | Depends | Works if pinned to dominant language, breaks if 50/50 |

The pattern: voice is strong for append-only logging with natural language and weak for precision edits or data with non-dictionary tokens.

Timeline

• Day 1: Setup, first 5 entries. Everything looked fine.
• Day 2: Proper name errors surfaced. We logged them and kept going.
• Day 3-4: Bilingual confusion appeared in 4 entries. We pinned language to Czech.
• Day 5: Deployed the alias dictionary with 12 known site names.
• Day 6-7: Clean runs. 91% name accuracy, under 5% bilingual garbage. 47 total entries logged.

The whole cycle from "this is broken" to "this works for our use case" took 5 days. Most of that was identifying the pattern, not writing the fix.

Cross-portfolio learning

We saw a version of the proper-name problem in Magical Song, our AI song generator. Users dictate a story ("make a song about my friend Věra's birthday") and Whisper mangles the name. The alias-dictionary approach from Voice Tables is now on the roadmap for Magical Song's input pipeline.

Same underlying issue, different product surface. Whisper's English bias is a portfolio-wide concern for us at Inithouse whenever voice input touches non-English proper nouns.

What we would do differently

Three things:

1. Ship the alias dictionary on day zero. We knew our sites had Czech names. We should have pre-loaded a dictionary before the first entry, not after 15 broken transcriptions.

2. Pin the transcription language from the start. Auto-detection is a nice default for demos. For production use where you know the primary language, pin it. We lost two days of clean data to a setting we could have flipped in config.

3. Set expectations about serial numbers. Voice is bad at alphanumeric codes. We should have designed the table to keep serial-number fields as manual-entry-only from the beginning, rather than discovering it mid-week when "B742" became "be seven forty two" in the database.

The takeaway

Voice-first data entry works, but only for a specific shape of work: appending natural-language observations into structured tables, in a known language, with a pre-loaded dictionary of proper nouns. Outside that envelope, you are fighting the transcription model more than you are saving time.

Voice Tables handles the workspace part well: 60 seconds from idea to a working table with docs and chat. The voice-to-data pipeline is where the sharp edges live, and they are fixable edges, not fundamental blockers.

We are 14 products into the Inithouse portfolio. Voice input keeps showing up as a feature request across multiple products. The field-notes experiment gave us a reusable playbook for when to say yes to voice and when to keep the keyboard.