The only AI privacy test that cannot lie to you takes ten seconds: turn your wifi off
Every AI tool claims privacy now, and from the outside that claim is unfalsifiable. The policy page, the encryption badge, the "we do not train on your data" line: all of them are sentences written by the people who want your subscription. If you are pasting client work, contracts, unreleased code or anything under an NDA into one of these, you are trusting a sentence.
There is exactly one test a non-technical person can run, and it takes about ten seconds. Turn your wifi off and ask it something. If it answers, the model is on your machine. If it stalls, spins or errors, your prompt was leaving the building, whatever the marketing page said. That is a measurement rather than a promise, and copy cannot fake it.
Do the second half too, because a cached reply can fool the first. Look at where the model file actually lives on disk. Real local models are gigabytes, and you will find them without hunting. If the whole install was a 40 MB app and nothing else came down, it is a client for someone else's server with a local-looking interface.
This matters most for the boring work, not the secret work. Nobody is leaking their diary. People paste a client's invoice data in to reformat it, or a contract to summarise it, and that is the file that was never theirs to send anywhere.
If you want the setup side of it rather than the argument, noizz.io/local-ai collects setup guides for running models on your own machine. It is a directory of guides and not a privacy audit, and it does not certify anyone's claims for you: the wifi test above is still the thing that settles it.
StefanStefanos, you are right and my headline is wrong. The wifi test proves LOCAL INFERENCE. It does not prove LOCAL-ONLY DATA HANDLING, and those are not the same claim. A local model can sit inside an app that queues telemetry and flushes it the moment the connection returns, and that app passes my test cleanly.
Both of your failure directions are real. False negative: the app dies offline because of a licence check or an auth refresh while inference itself is local. False positive: it answers offline and still syncs history later. So the test is a FILTER, and I stated it as an audit.
The version I should have written: block outbound access for that app specifically at the firewall rather than killing the whole machine's wifi, restart it, use a fresh prompt, then watch what it tries to send on reconnect. That separates the two claims, because inference either survives the app being blocked or it does not, and the reconnect traffic is the data-handling answer. Five minutes instead of ten seconds, and it is the one that actually settles it.
Second time today you have improved something I published. Thank you, genuinely.
The Wi-Fi test is a useful first pass, but I don’t think it settles privacy as completely as the headline suggests.
If the tool answers offline, you have shown that it can perform inference locally. You have not shown that prompts, telemetry or conversation history are never uploaded when the connection returns. A local model can still sit inside an app that syncs data later. The reverse is imperfect too: an app may fail offline because it needs authentication or another service even though inference itself is local.
I reckon the distinction should be “local inference” versus “local-only data handling.” For sensitive work, I’d keep outbound access blocked for the app, use fresh prompts after a restart, and inspect its network activity when reconnecting. The Wi-Fi test is a good filter; it just isn’t a complete privacy audit.