Onira

I shipped an Android app that generates a personalised hypnosis session on the

device — no server, no account, nothing uploaded. Here is what the on-device part

actually cost, in case you are considering it.

Memory is the constraint, not speed.

I expected inference to be slow and it is: a mid-range phone writes 300-500 words

in tens of seconds. What I did not expect is that speed stops being the problem

almost immediately. The runtime loads a 2.6 GB model into the native heap and

does not mmap it. Measured on a Pixel 7 (7.6 GB RAM) mid-generation: ~6 GB PSS,

zram 100% full, kswapd at 72% CPU. The process thrashes, generation crawls, and

the low-memory killer takes the app the moment it leaves the foreground.

Two things fixed it, both structural rather than clever:

1. Close the model as soon as the last block is generated. The session then

   narrates for 30 minutes with nothing resident — which is exactly the window

   where the user backgrounds the app.

2. Preload the ad interstitial in one place only, on the home screen. Loading it

   spins up a WebView in its own process, and every other moment I had tried

   overlapped with the model being resident.

Generate in blocks, not in one call.

The first version asked for the whole script in one generation. Time to first

audio was minutes, and if it failed, it failed entirely. Now a session is an

ordered list of blocks, each 300-500 words, each its own turn on the same

conversation so later blocks stay thematically consistent. The opening and the

closing are not generated at all — nothing in them depends on the user's input,

so they are templates, which means narration starts as soon as the model is

loaded rather than after the first generation. Block N+1 is written while block

N narrates. A block that fails gets a short pre-written bridge instead of killing

the session.

Sizing by duration, not word count.

I sized sessions at a fixed word count and shipped ~90-minute sessions to anyone

who had slowed the narration down. Narration speed and pause length swing real

duration by more than 3×. The fix was to budget from a target duration, convert

to words at the user's own settings, and subtract the templates.

The honest trade.

On-device buys a privacy claim that is structurally true rather than a promise —

there is no server that could read what you type, so there is nothing to audit.

It costs a 2.6 GB first-run download, which is the biggest drop-off in the

funnel by a distance, and it rules out the small server-side moderation layer

you would otherwise put in front of a model writing psychologically-framed

content for someone who might be in distress. That last one is handled with a

keyword gate before the model is ever invoked, which is deliberately

high-recall — I would much rather block a benign message than miss a real one.

Would do it again for this app. Would not do it for anything where the model is

the product rather than a privacy argument.