What On-Device Processing Would Look Like
If a companion app genuinely runs its model on your phone, several things follow that you can observe: it works with the network off, it consumes a large amount of storage, it warms the device and drains the battery during replies, and its answers are noticeably less capable than a server-backed equivalent. If an app claims local processing but none of those hold, the claim is describing something narrower than it sounds.
This is worth checking because “on-device” and “private” get used interchangeably in marketing, and only one of them is a testable property.
The four things that would have to be true
Local inference has physical consequences that are hard to hide.
It works offline. This is necessary and it is the first test, described in what airplane mode tells you about an app. An app that cannot reply without a connection is not generating replies locally. There is no exception to this.
The app is large. A model that fits on a phone still occupies substantial storage, and it has to arrive somehow — either in the download or as a separate fetch on first run. Check the app’s storage footprint in your phone’s settings. A conversational app occupying a few tens of megabytes is not carrying a model.
Replies cost battery and heat. Local inference uses the device’s processing units hard. You should be able to feel it, and you should see the app appear prominently in your battery usage breakdown. Server-backed apps use almost no local processing for generation; their battery cost is network and screen.
Replies are slower to start and shorter on capability. Local models are smaller than data-centre models. That shows up as less coherence over long conversations, weaker recall, and simpler language. An app producing data-centre-quality output instantly on a phone is not doing it on the phone.
The narrower claims that are usually what is meant
Most legitimate uses of “on-device” in this category refer to something real but limited, and it is worth knowing which.
On-device speech recognition, converting your voice to text locally before sending the text. This is a genuine privacy improvement — audio never leaves — and it is a completely different claim from local generation.
On-device wake-word detection, so the microphone is only opened locally until a trigger. Standard in voice products.
Local storage of history, which means your conversations live on your phone rather than only on a server. Also real, also unrelated to where generation happens, and it has its own consequences for backups and device residue.
Local encryption at rest, meaning the app’s local database is encrypted. Worth having; says nothing about transmission.
A local fallback model used only when offline, with the main experience served remotely. Some apps do this and the offline voice is recognisably different.
None of these are dishonest. They become misleading only when compressed into a headline that a reader will take to mean their conversations never leave the phone.
Why full local generation is uncommon here
Two constraints, both structural rather than about any particular company.
The models that make companion apps feel like companions are large, and running one on consumer hardware means accepting a substantially smaller model with visibly reduced capability. In a category where the entire product is conversational quality, that is a hard trade to make.
And the business model usually depends on the server. A subscription is easier to enforce when the service is remote, features can be updated without shipping an app release, and usage is measurable. Local inference gives all of that up. These are not sinister reasons; they are ordinary product reasons, and they explain why the honest default assumption is remote generation.
What local processing would and would not buy you
Even a genuinely local app is not automatically private, and this is the misconception worth dismantling.
Local generation means the text of a reply was computed on your phone. It does not mean the prompt was not also transmitted, that no telemetry was sent, that history is not synced, or that crash reports do not contain conversation fragments. All of those are separate decisions.
A local app can still phone home, for licensing, analytics, model updates, and diagnostics. Watch the per-app data usage counter; a truly local app should transfer very little in normal use.
The exposures on your own device get larger, not smaller. If everything is local, everything is in your backups, in your device residue, and available to anyone with your unlocked phone. See what ends up in your phone backup and what a shared device exposes.
That last point is the genuinely useful one. Local processing does not remove risk; it moves it from an operator to your device, and the correct precautions change accordingly.
What this does not tell you
None of these observations prove anything about what an app transmits. Storage size, battery draw, and offline behaviour are consistent with a hypothesis; they do not inspect traffic, and treating them as proof would be exactly the overreach this post is arguing against.
They are, however, enough to catch the common case, which is not deception so much as compression — a real but narrow technical property described in language that implies a much broader one. If an app’s storage footprint is small and it fails offline, you can be confident that whatever “on-device” refers to, it is not where your conversation is being written.